Skip to main content

Exploitation of 2D binary source correlation using turbo block codes with fine-tuning


This article proposes a joint source-channel coding technique for two-dimensional (2D) binary Markov sources by using concatenated turbo block codes composed of two Bose, Chaudhuri, Hocquenghem (BCH) codes, of which output is followed by a rate-1 recursive systematic convolutional code. The source correlation of all rows and columns of the 2D source is well exploited by using a modified Bahl–Cocke–Jelinek–Raviv (BCJR) algorithm for the decoding of the BCH codes. Simulation results show that the proposed technique outperforms in terms of bit error rate the codes that exploits one-dimensional (1D) source correlation using the modified BCJR algorithm, and obviously the conventional system without source correlation exploitation. In order to further improve the performance, this article aims to make fine-tuning of the code parameters, given the source correlation property, that can achieve performance even closer to the theoretical limit than without the fine-tuning. Finally, results of image transmission simulations using two images, one having strong and the other weak 2D correlation, are presented to demonstrate the effectiveness of our proposed technique.

1 Introduction

According to the Shannon’s separation theorem, the optimal design of the source and channel codes can be sought for independently, so far as the source entropy rate is lower than the channel capacity [1]. However, this theorem assumes infinite code length, which imposes, in theory, infinite latency to the communication systems. Furthermore, in many existing source coding schemes there remains residual redundancy left after source encoding, especially for the sources having multi-dimensional memory structure such as image and/or videos, and in fact, the residual redundancy can well be utilized to achieve further error protection. Such inferiority of the separation theorem-based design have motivated a considerable number of work towards establishing efficient joint source-channel coding (JSCC) techniques, especially towards best exploiting the source correlation in channel coding.

Since the discovery of the turbo codes by Berrou et al. [2], which achieves the bit error rate (BER) threshold at only 0.7 dB away from the Shannon limit, the research community is experiencing significant paradigm shift in many areas of information theory and applications. So is the case of JSCC design, and it has been quite common that JSCC techniques utilize the turbo or turbo-like codes. The JSCC techniques shown in [36] characterize the source by a Markov model and exploit the source statistic during decoding process at receiver. Source statistic is taken into account in encoder design in [710], and correspondingly decoding algorithm modification is also proposed. Recently, exploitation of the memory structure inherent within the source using serial concatenated convolutional codes has been proposed in [11], which demonstrates superiority in BER performance without requiring heavy decoding complexity. However, the JSCC methods that have been proposed so far operate at relatively low code rate (equal or less than a half rate), and furthermore they considered the exploitation of the source correlation in only one direction, such as line-by-line scanning of image, which can be referred as one-dimensional (1D) source correlation.

In our recent work presented in [12], we introduced a novel JSCC system which exploits the two-dimensional (2D) source correlation using turbo block codes (TBC) utilizing relatively high rate Bose, Chaudhuri, Hocquenghem (BCH) codes. The Bahl–Cocke–Jelinek–Raviv (BCJR) algorithm [13] used in decoding the two component codes of TBC is modified based on the algorithm proposed in [11]. The proposed 2D technique achieves relevant performance gain in terms of BER compared to the system that exploits 1D source correlation only and apparently much significant gain over the conventional system without source correlation exploitation. Obviously, the stronger the source correlation, the larger gain can be obtained, in general. In fact, the multi-dimensional source correlation imposes restrictions on the values of neighboring symbols in multiple dimensionality, and hence 2D source correlation is stronger than 1D correlation. The technique shown in [12] exploits this principle, and the technique proposed in [12] allows the use of high rate codes. Furthermore, the technique presented in [12] utilizes excellent property of a memory-1 recursive systematic convolutional (RSC) code, referred to as accumulator, that follows the 2D TBC via random interleaver; with this structure, clear BER threshold can be achieved.

The aim of this article is to provide a clear and complete description of the proposed 2D source correlation utilization technique presented in [12]; since [12] is a conference publication, the shortage in space did not allow us to provide the algorithm details. On the top of the complete descriptions of the technique shown in [12], it is also shown in this article that further performance improvement is possible by replacing the symmetric TBC and accumulator used in [12] with asymmetric TBC and a longer memory RSC code, respectively. For the stone-by-stone comparison, this article applies the proposed new techniques we found after the publication of [12] to the cases where only 1D source correlation is utilized.

The remainder of the article is organized as follows. The 2D source model we assume in this article is described in Section 2. The proposed technique and the modified BCJR algorithm are detailed in Section 3. Section 4 presents the results of simulations for the utilization of the 1D and 2D source correlation as well as for the case where no correlation information is taken into account. The results of the image transmission simulations are presented in Section 5. Finally, concluding remarks are provided in Section 6.

2 2D Markov sources

The model that we assume to characterize the source behavior is a first-order binary Markov source model. The 1D Markov source is described first, and then, we extend it to 2D Markov source. For 1D case, the state U t at a time t is determined by its value at t−1 according to the conditional probability P r{U t =i|Ut−1=i} where i,i{0,1}, and the behavior of the model can conveniently be expressed by the state transition matrix:

A= a i , i = p 1 1 p 1 1 p 2 p 2 ,

where p1 and p2 are the state transition probabilities; p1 and p2 stand for the probabilities that the state at t is the same as the previous state, as shown in the state diagram in Figure 1 where state S0 emits binary “0” while S1 emits binary “1”.

Figure 1
figure 1

Two-state Markov chain. State S i emits binary output i, i{0,1}.

The entropy rate of the stationary first-order Markov source [14] can be calculated by

H(S)= i , i { 0 , 1 } μ i a i , i log a i , i ,

where μ i represents the stationary state distribution. With p1=p2=p, the Markov source is symmetric, and hence μ i =0.5. In this case, Equation (2) can be simplified to

H(S)=p log 2 p(1p) log 2 (1p).

For the 2D Markov source, the output Un,t of the current state is determined by two factors; the previous state in the horizontal direction, Un,t−1 and that in the vertical direction, Un−1,t, where t and n are the timing indexes of the 2D source in the horizontal and vertical directions, respectively. This 2D source correlation model can be considered as a two-dimensionally coupled first-order Markov chains and the corresponding transition matrix of the 2D source can be represented by using a coupled Markov chain (CMC) model [15]. Following the CMC model, we can use the 1D source transition matrices AH and AV for the horizontal and vertical directions, respectively, in order to obtain the 2D source transition matrix B, as

B = b i , j , k = Pr { U n , t = k | U n 1 , t = i , U n , t 1 = j } = a i , k V · a j , k H f = 0 1 ( a i , f V · a j , f H ) , i , j , k { 0 , 1 } ,

where a i , k H and a j , k V are the elements of the matrices AH and AV, respectively. The entropy rate for the 2D source can be calculated as

H ( U n , t | U n 1 , t , U n , t 1 ) = H ( U n 1 , t , U n , t 1 | U n , t ) + H ( U n , t ) H ( U n 1 , t , U n , t 1 ) ,

which, by assuming Un,t−1 and Un−1,t, are independent given Un,t, can be simplified to

H ( U n , t | U n 1 , t , U n , t 1 ) = H ( U n 1 , t | U n , t ) + H ( U n , t 1 | U t , n ) + H ( U t , n ) H ( U t , n 1 , U t 1 , n ) = H V ( S ) + H H ( S ) + H ( U n , t ) H ( U n 1 , t , U n , t 1 ) .

H H (S) and H V (S) can be calculated by using the entropy rate from (2) for the horizontal and vertical directions, respectively, while H(Un,t) and H(Un−1,t,Un,t−1) are obtained empirically via measurements. It is worth to mention that in the case of symmetric Markov chain of both directions, H(Un,t)=1.

3 Correlation exploitation

3.1 2D coding technique

The block diagram of our proposed JSCC scheme at the transmitter side is shown in Figure 2. The source U is a 2D binary source, and the correlation of each row and column of the source is indicated by the state transition probabilities, p1 and p2, respectively. Therefore, for a 2D source with a size K×K, there are 2×K×K transition probabilities representing the 2D correlation of the source since the probability varies horizontal and vertical line-by-line.

Figure 2
figure 2

Block diagram of the proposed system at transmitter.

In our proposed system, the average value for the source correlation, p ̄ , is required since this value will be used later to determine whether to fed or bypass the rate-1 inner code. For any value of p<0.5, the correlation is defined as 1−p, whereas for p≥0.5, the correlation is equal to p. Hence, p ̄ has a value between 0.5 and 1, with which p ̄ =0.5 indicates no correlation and p ̄ =1 indicates the full correlation, both in average sense.

The coding scheme shown in Figure 2 can be viewed as a serial concatenation of 2D TBC and the rate-1 inner code, where a random interleaver is in between. The TBC consists of two components of block codes, C1 and C2, and in this article, we use BCH codes as the component codes of TBC. BCH code C1 has a parameter set (N1,K1,D1), where K1 refers to the information bit length, N1 the code word length and D1 the minimum Hamming distance of C1. In the same way as in C1, C2 has a parameter set (N2,K2,D2) where K2 refers to the information bit length, N2 the code word length and D2 the minimum Hamming distance of C2. Following the TBC structure, the information bit U is directly fed into C1, while C2 receives the interleaved version of U. Block interleaver π B is used in the proposed scheme in order to maintain the source correlation in the vertical direction since the use of random interleaver breaks the correlation. However, this has a side effect that the extrinsic log-likelihood ratios (LLRs) are correlated, and the use of correlated LLR does not strictly follow the turbo concept. Therefore, to break the correlation, we need random interleaver, followed by an inner code C3, which is a rate-1 RSC code. However, this process breaks the 2D correlation. Hence, we need a trade-off of either we use C3 or not, and the threshold depends on the correlation of the source itself. Theoretical derivation of the optimal threshold is, however, out of the scope of this article and is left as future study.

The threshold value indicated by p T is determined empirically since, as indicated in [12], the inclusion of C3 only yields benefit to the source having strong correlation and it will degrade the performance when source correlation is weak. If the average source correlation, p ̄ , is equal or larger than the threshold value p T , the parity sequence together with information sequence U is multiplexed and interleaved by using random interleaver π R . The π R -interleaved sequence is then fed into C3. In order to avoid the performance degradation due to this observation, C3 is bypassed when p ̄ < p T . Assuming p ̄ p T , the parity sequence from C3 is multiplexed with the parity sequence from C2, modulated by using binary-phase shift keying (BPSK) and transmitted over additive white Gaussian noise (AWGN) channel where the variance of the complex noise is σ2. The overall code rate of the system is, therefore, K/(2NK).

The block diagram of our proposed system at receiver side is shown in Figure 3 where C 1 1 , C 2 1 , and C 3 1 are the decoders of C1, C2, and C3, respectively. All three decoders are activated if p ̄ p T with activation order C 3 1 , C 1 1 , C 2 1 , C 1 1 , C 3 1 , C 1 1 , C 2 1 , C 1 1 ,; on the other hand, if p ̄ < p T , only two decoders are activated with activation order C 1 1 , C 2 1 , C 1 1 , C 2 1 ,. All decoders iteratively exchanged information in the form of extrinsic LLR, L e i ,i{1,2,3} where i represents the component decoder indexes. The BCJR algorithm is used for soft-input soft-output decoding of each component code. In order to exploit the source correlation during the decoding process, the modified BCJR algorithm is used in C 1 1 and C 2 1 . The correlation of each row of the 2D source is utilized by C 1 1 while the correlation of each column of the 2D source is utilized by C 2 1 . The modified BCJR algorithm is based on the 1D JSCC decoder modification shown in [11] for convolutional codes; however, we have made additional minor modification to the algorithm in order to make the algorithm better suited with block codes. In this article, we assume that the source correlation property for the horizontal and vertical directions are known to the receiver. In practical, the source correlation property can be forwarded with the help of higher order protocol.

Figure 3
figure 3

Block diagram of the proposed system at receiver.

It is worth to mention that our 2D technique can also be used in serial concatenation of BCH codes with block interleaver separating the codes. We conducted a simple experiment of using our technique with serial concatenation and compared the results with parallel concatenation. Code parameters are the same in both cases and the only difference is that in the case of the serial concatenation, “parity-of-parity” is transmitted and decoder uses it. There was no significant improvement observed by using serial concatenation, despite the decrease in code rate and the increase in decoding delay. This finding is consistent to the result presented in [16] for conventional TBC without source correlation exploitation.

3.2 Modified BCJR algorithm

The source correlation can be well utilized by making modifications of the BCJR algorithm. In this section, we will show how the information of the source transition probability can be included in the standard BCJR algorithm [13] in the channel decoding process. Referring to the work in [11] for convolutional codes, a similar modified BCJR algorithm can be derived. The Markovinity inherent in the model yields the a posteriori LLR, corresponding to the state transition from i’ to b i’ where i,i{0,1}, as

L( U t | Y t )=ln ( s , s ) R t 1 i , 1 α t 1 ( s , i ) γ t ( s , i , s , i ) β t ( s , i ) ( s , s ) R t 0 i , 0 α t 1 ( s , i ) γ t ( s , i , s , i ) β t ( s , i ) ,

where Y t is the received signal at the timing index t, and R t 1 is the set of the transitions from trellis state s’ to s corresponding to U t =1 and likewise, R t 0 for the set corresponding to U t =0. The forward and backward state probabilities α t and β t are obtained by, respectively, the recursions Equations (8) and (9), and the state transition probability γ t is given by (10) as

α t (s,i)= s , i α t 1 ( s , i ) γ t ( s , i ,s,i),
β t 1 ( s , i )= s , i β t (s,i) γ t ( s , i ,s,i),


γ t ( s , i , s , i ) = Pr { S t = s , U t = i , Y t | S t 1 = s , U t 1 = i } , = Pr { S t = s , U t = i | S t 1 = s , U t 1 = i } Pr { Y t | U t = i } ,

respectively. The joint probability of the first term in (10) is influenced by the transition probability of the Markov source and the input bit U t corresponding to the state transition from St−1=s’ to the state S t =s. Since the Markov source output is equivalent to the input U t to the outer code C1 and/or C2, the first term can be approximated by

Pr { S t , U t | S t 1 , U t 1 } Pr { U t = i | U t 1 = i } Pr { U t = i } , = a i , i Pr { U t = i } .

It is worth being noticed here that there is no difference in terms of the calculation for the terms α t and β t as shown in (8) and (9), respectively, from the standard BCJR algorithm; the difference of the modified algorithm from the standard algorithm arises when calculating the branch metric γ t , where a new term a i , i is added as shown in (11) in order to take into account the source correlation information. For block codes, the state and state transition probabilities associated with the trellis diagram are evaluated until t=N, which can be separated into two parts; the first part is the uncoded information data bit sequence and the second part is the parity bit sequence. The uncoded information data bits are corresponding to the index tK while the parity bits to the index K<tN. Accordingly, the γ t calculation is also separated into two parts; the first part is for tK where it is affected by the value of a i , i obtained from the Markov source transition probabilities. The second part is for K<tN, where the term a i , i =0.5 for i,i{0,1}, since there is no correlation for the parity sequence, and hence for the second term in (11), P r{U t =i} for i{0,1} is equal to 1 because the a priori LLR feedback for the parity sequence is zero. The extrinsic information can be obtained by subtracting the a posteriori LLR calculated by (7) from the a priori information and the soft channel output of Y t , according to the turbo principle.

4 Numerical results

4.1 BER performance evaluation

A series of simulations was conducted by using BPSK modulation over AWGN channel to evaluate the BER performance of our 2D technique. The source correlation property of each row and column of the 2D source was assumed available at the receiver and the source correlation in the horizontal and vertical directions was exploited by the decoders C 1 1 and C 2 1 , respectively. For simplicity, this article only considers the results for symmetric Markov sources and hence, the strength of the source correlation is indicated by the transition probability p, where p=0.5 corresponds to uncorrelated source due to equal and random appearance probability of binary values 0 and 1, regardless of the previous source value. The source correlation becomes stronger as the value of p approaches 1 or 0, however, as stated before, in this article, we only consider the value of p between 0.5≤p≤1. In order to observe the effect of the different correlation strength, the same p value was used for each row and column of the 2D source. Memory-2 RSC (7,4)8 was used as the inner code C3, while BCH (127,120,3) was used for C1 and C2. With this setting, the overall code rate of the system is R c =0.90. The BER performance is shown in Figure 4 and it is found that by setting the threshold value p T =0.71 makes significant difference in performance.

Figure 4
figure 4

BER performance of the proposed 2D system with and without C 3 .

It should be reminded that the threshold for selecting either the C3 be used or not depends on the pros-and-cons trade-off due to LLR correlation: block interleaver aims to keep the source correlation, while the correlation makes negative impact on the iterative decoding. The threshold value is obtained empirically, and its theoretical derivation is out of the scope, even though we recognize that it is a very important open question. In this simulation, we used 240×240 block interleaver π B and random interleaver π R with a length of 60960.

Figure 5 shows the comparison of BER performance between the proposed 2D system, the 1D system, and conventional TBC system [16, 17]: the 1D system utilizes the modified BCJR algorithm in C 1 1 , while the standard BCJR algorithm for C 2 1 ; the conventional TBC system does not utilize the source correlation at all. BER performances were evaluated after 20 iterations for all the cases. It can be observed in the 1D and 2D BER curves that the stronger the source correlation, the better performance can be achieved, and hence, the gain from the conventional system increases. It is found from Figure 5 that larger gain can be achieved by exploiting the 2D source correlation. With p=0.7, the 2D system outperforms the 1D system by 0.29 dB and as the source correlation becomes stronger, larger improvement can be achieved, for example, with p=0.9, the 2D system outperforms the 1D system by 1.46 dB.

Figure 5
figure 5

Comparison of BER performance of the system utilizing 2D source correlation and 1D source correlation. Both systems used BCH (127,120,3) in component codes C1 and C2, while RSC memory-2 (7,4)8 used in C3, R c =0.90.

The theoretical limit based on the constellation constraint capacity (CCC) [18] is used to evaluate the gap in E b /N0 values between that required to achieve a low enough BER (=10−5) and the theoretical limit CCC for a set of N possible transmitted symbols a is given by

C = log 2 ( N ) 1 N · k = 0 N 1 E log 2 i = 0 N 1 exp | a k + w a i | 2 | w | 2 2 σ 2 ,

where w is the complex Gaussian noise with variance of σ2. For the source with memory, (12) is subjected to the constrains of H·R c C where R c is the code rate of the system and H is the entropy rate defined by (6). The H values for the several state transition probability p values and their corresponding theoretical E b /N0 limits for the BPSK transmission over AWGN channel are shown in Table 1. Based on the simulation results shown in Figure 5, the gaps at BER 10−5 in E b /N0 to the theoretical limit were determined and tabulated in Table 1. The gain over the conventional system increases, as the source correlation becomes strong. However, the stronger the source correlation the larger is the gap to the CCC limit; for example, with p=0.7, gap =2.88 dB, while with p=0.9, gap =4.22 dB.

Table 1 Gaps to the theoretical limit and gains from the conventional system in E b / N 0 at BER 10 −5 of the proposed 2D system

4.2 Fine-tuning of code parameters

In the previous section, the main focus has been on the technique that well exploit the 2D source correlation. This section aims to further adjust the code parameters to achieve performance closest to the theoretical limit, based on the concept described in the previous section. In order to reduce the gap to the limit especially for strong source correlation case, we have evaluated the BER performance by using different BCH codes for C1 and C2, and different RSC codes for C3. Within the Hamming distance three codes, BCH (N,K,3), we conducted a brute-force code search towards the performance enhancement for C1 and C2 optimization with maximum N=255. We have limited our search for C3 within the inner rate-1 RSC codes having two-tap feedback polynomials that achieve for all 1’s input binary sequence result in better performance than any other generator polynomials, according to the results in [19]. We only consider the total memory size of C3 up to 8. We have tested all possible code combinations from our code list and the code combination with performance closest to the theoretical limit was selected. Figure 6 shows the BER performance for p=0.7,0.8 and 0.9 after 30 iterations, all with the code parameters obtained by the brute-force search. Table 2 summarizes the code parameters obtained by the brute-force search for p=0.7,0.8 and 0.9 together with its gap to the theoretical limit. Different combinations of codes have been found for different p values. With p=0.7, gap = 0.71 dB and as the p-value increases, the BER performance of the 2D system tends to be further away from the theoretical limit and for p=0.9, the BER performance is 3.45 dB away from the limit, which is smaller than without the fine-tuning of the code parameters, i.e., 4.22 dB in Table 1. Further improvement may be possible by using C3 with a longer memory and, C1 and C2 with BCH codes having minimum Hamming distance larger than 3. However, apparently, this approach suffers from exponentially increasing computational complexity.

Figure 6
figure 6

BER performance of the 2D system with code parameter optimization after 30 iterations.

Table 2 Gaps in E b / N 0 at BER 10 −5 of our 2D system after code parameter optimization

Previously, we have proposed a TBC-based technique where the knowledge about the 2D correlation is exploited in the decoding process, resulting in better performance than the conventional techniques. This invokes another question of how much improvement can be exploited by the proposed 2D techniques when the source has indeed only 1D correlation, i.e., H(Un,t|Un,t−1)≠1 and H(Un,t|Un−1,t)=1. This is the case of, for example, conventional fax machine using line-by-line scanning only. Based on the same search technique as in the 2D system, the optimal code parameter search for 1D system was performed. Figure 7 depicts the BER performance with the optimal code parameters for different p values for the 1D system. Table 3 summarizes the optimal code parameters for p=0.7,0.8, and 0.9 together with their gaps to the theoretical limits. With different p values, the optimal code parameters are also different, nevertheless, the gaps to the theoretical limit are around 0.6 dB in all the cases tested.

Figure 7
figure 7

BER performance of the 1D system with code parameter optimization after 30 iterations.

Table 3 Gaps in E b / N 0 at BER 10 −5 of the 1D system after code parameter optimization

Only a few techniques have been proposed for the utilization of 1D binary Markov source correlation and these techniques are designed for low code rate applications, say, R c ≤0.5. Zhu and Alajaji [3] have proposed a JSCC system using turbo convolutional codes and the technique is known as joint source channel turbo codes (JSCTC). Meanwhile, Zhou et al. [11] have proposed serial concatenated joint source-channel codes (SCJSCC), where it is shown that superior BER performance can be achieved with less decoding complexity than JSCTC. The comparison in terms of the gap to the theoretical limit between our 1D system with JSCTC and SCJSCC is summarized in Table 4. The 1D system proposed in this article for the case H(Un,t|Un,t−1)≠1 and H(Un,t|Un−1,t)=1 outperforms the other two reference systems.

Table 4 Comparison of gaps to the theoretical limit at BER 10 −5 between our 1D system with SCJSCC and JSCTC

5 Image transmission simulation

In this section, we demonstrate the effectiveness of our proposed technique through image transmission simulations. Two images were tested: one with strong correlation and the other with weak correlation, and the simulations were conducted over AWGN channel. The horizontal and vertical state transition probabilities, line-by-line, were evaluated before performing the simulations, and were used in the modified BCJR algorithm, as stated in Section 3.2.

The first image example was a 240×240 pixels binary image of Lenna, shown in Figure 8a, with an average source correlation p ̄ =0.92. We used commonly BCH (127,120,3) code as the TBC’s component codes C1 and C2, and a rate-1 RSC code with generator polynomial (7,4)8 for C3 throughout the simulations. The image was transmitted over the same channel (even with the same noise samples) before decoded by using different techniques. We chose specific E b /N0 value at which turbo cliff happens in the BER performance with the 2D correlation-exploited case. From the BER curves shown in Figure 5, the turbo cliff for the first image example ( p ̄ =0.92) is estimated to happens at E b /N0=−3.8 dB. The result of the decoded image by using the proposed 2D and 1D systems after 25 iterations at E b /N0=−3.8 dB are shown in Figure 8b,c, respectively. It is found that there is no degradation in image quality [indicated by pixel errors and peak signal-to-noise ratio (PSNR)] with the 2D system, while 4.70% pixel errors remains with the 1D system. The conventional TBC system without exploiting the source correlation results in the lowest image quality, with pixel errors of 19.36% and PSNR 7.14 dB as shown in Figure 8d. In order to give good consistency of the results due to the random channel state, we calculated the average PSNR (APSNR) over 300 samples (different noise sample sequences were used, transmission-by-transmission, and the same sets of the noise sample sequences were commonly used for different decoding techniques). The corresponding APSNR for the 2D, 1D, and conventional TBC systems are 32.23, 13.36, and 7.13 dB, respectively.

Figure 8
figure 8

Image source 1 with p ̄ = 0 . 92 at E b /N 0 =−3.8 dB (a) transmitted image, (b) proposed 2D system (0% pixel errors, PSNR dB), (c) proposed 1D system (4.41% pixel errors, PSNR 13.49 dB), (d) conventional TBC system (19.31% pixel errors, PSNR 7.14 dB).

The image used exemplifying relatively low p value is a gray-scale image of Lenna with size 240×240 pixels where each pixel of the image is represented by eight binary digits. Before encoding this sample image, the gray-scale image shown in Figure 9a was converted into a binary matrix with p ̄ =0.62. The second image example has p value close enough with p=0.7 (there is no BER curve for p=0.6 in Figure 5). From the BER curves in Figure 5, it is estimated that for the 2D correlation-exploited case with p=0.62 the turbo cliff is achieved around E b /N 0 =3.2 dB. The results of the decoded image at E b /N 0 =3.2 dB after 25 iterations are shown in Figure 9b–d for various systems where eight transmissions in total were required to decode the full image. It is found from the results that the proposed 2D system achieves the highest image quality, with no pixel errors, similar to the previous image example in Figure 8b. The corresponding APSNR for the 2D, 1D, and conventional TBC systems are 52.11, 39.87, and 38.24 dB, respectively. From the two image transmission simulations, we can clearly see the advantage of our proposed systems over the conventional TBC system without utilizing source correlation especially in the case of source with strong correlation. As expected, the 2D system achieves better image quality than the less complex 1D system due to the extra source correlation information required in 2D system.

Figure 9
figure 9

Image source 2 with p ̄ = 0 . 62 at E b /N 0 =3.2 dB (a) transmitted image (b) proposed 2D system (0% pixel errors, PSNR dB) (c) proposed 1D system (9.39% pixel errors, PSNR 39.97 dB) (d) conventional TBC system (11.52% pixel errors, PSNR 38.98 dB).

6 Conclusions

In this article, a JSCC technique utilizing the correlation of 2D binary Markov sources using high rate codes has been proposed. A rate-1 RSC code and two BCH codes were deployed, where the BCH decoders use the modified BCJR algorithm in order to exploit the source correlation. The threshold for selecting whether or not to use the rate-1 RSC code depends on the trade-off due to LLR correlation: the block interleaver aims to keep the source correlation, while the LLR correlation causes negative impact on the iterative decoding. It has been shown through a series of simulations that significant BER performance improvement can be achieved by using the proposed technique over the 1D and conventional systems. In order to further enhance the performance, the code parameters were fine-tuned for different source correlation strengths. It has been shown that the utilization of the 1D source correlation, as exemplified by line-by-line scanned image, with our proposed technique achieves a BER threshold of only about 0.6 dB away from the theoretical limit, and outperforms the other 1D techniques, JSCTC and SCJSCC. This new technique has many potential applications and in this article, we have demonstrated the effectiveness of the proposed technique in image transmission application.


  1. Shannon CE: A mathematical theory of communication. Bell Syst. Tech. J 1948, 27(3):379-423.

    Article  MathSciNet  Google Scholar 

  2. Berrou C, Glavieux A, Thitimajshima P: Near, Shannon limit error-correcting coding and decoding: turbo-codes. In Proceedings of the IEEE International Conference on Communications (ICC) 1993. Geneva, Switzerland; (1993:1064-1070.

    Google Scholar 

  3. Zhu G, Alajaji F: Joint source-channel turbo coding for binary Markov sources. IEEE Trans. Wirel. Commun. 2006, 5(5):1065-1075.

    Article  Google Scholar 

  4. Garcia-Frias J, Villasenor JD: Combining hidden Markov source codes, parallel concatenated. IEEE Commun. Lett 1997, 1(4):111-113.

    Article  Google Scholar 

  5. Garcia-Frias J, Villasenor JD: Joint turbo decoding and estimation of hidden Markov sources. IEEE J. Sel. Areas Commun 2001, 19(9):1671-1679. 10.1109/49.947032

    Article  Google Scholar 

  6. Belhadj H, Zaibi S, Bouallègue A: Iterative joint source-channel decoding with source statistics estimation: application to image transmission. Signal Process. Image Enhance. Multimed. Process. Multimed. Syst. Appl. Ser 2008, 31: 301-311. 10.1007/978-0-387-72500-0_26

    Article  Google Scholar 

  7. Zhu G, Alajaji F: Turbo codes for nonuniform memoryless sources over noisy channels. IEEE Commun. Lett 2002, 6(2):64-66.

    Article  Google Scholar 

  8. Zhu G, Alajaji F, Bajcsy J, Mitran P: Transmission of nonuniform memoryless sources via nonsystematic turbo codes. IEEE Trans. Commun 2004, 52(5):855.

    Google Scholar 

  9. Cabarcas F, Souza R, Garcia-Frias J: Source-controlled turbo coding of non-uniform memoryless sources based on unequal energy allocation. In Proceedings of the IEEE International Symposium on Information Theory (ISIT) 2004. Chicago, Illinois, USA; 2004:164-164.

    Google Scholar 

  10. Cabarcas F, Souza R, Garcia-Frias J: Turbo coding of strongly nonuniform memoryless sources with unequal energy allocation and PAM signalling. IEEE Trans. Signal Process 2006, 54(5):1942-1946.

    Article  Google Scholar 

  11. Zhou X, Anwar K, Matsumoto T: Serially concatenated joint source-channel coding for binary Markov sourcesin. In 6th International ICST Conference on Communications and Networking (CHINACOM) 2011. Harbin, China; 2011:53-60.

    Chapter  Google Scholar 

  12. Izhar MAM, Fisal N, Zhou X, Anwar K, Matsumoto T: Utilization of 2-D Markov source correlation using block turbo codes. In Proceedings of the 7th International Symposium on Turbo Codes and Iterative Information Processing 2012. Gothenburg, Sweden; 2012:56-60.

    Google Scholar 

  13. Bahl L, Cocke J, Jelinek F, Raviv J: Optimal decoding of linear codes for minimizing symbol error rates (corresp.). IEEE Trans. Inf. Theory 1974, 20(2):284-287.

    Article  MathSciNet  Google Scholar 

  14. Cover TM, Thomas JA: Elements of Information Theory. USA: Wiley; 2006.

    Google Scholar 

  15. Elfeki AMM, Dekking FM: A Markov chain model for subsurface characterization: theory and applications. Math. Geol 2001, 33(5):569-589. 10.1023/A:1011044812133

    Article  MathSciNet  Google Scholar 

  16. Hanzo L, Liew TH, Yeap BL: Turbo Coding, Turbo Equalisation and Space-Time Coding for Transmission Over Fading Channels. Wiley-IEEE Press; 2002.

    Book  Google Scholar 

  17. Hagenauer J, Offer E, Papke L: Iterative decoding of binary block and convolutional codes. IEEE Trans. Inf. Theory 1996, 42(2):429-445. 10.1109/18.485714

    Article  Google Scholar 

  18. Ungerboeck G: Channel coding, with multilevel/phase signalling. IEEE Trans. Inf. Theory IT-28 1982, 55-67.

    Google Scholar 

  19. ten Brink S: Code characteristic matching for iterative decoding of serially concatenated codes. Ann. Telecommun 2001, 56(7):394-408.

    Google Scholar 

Download references


This research was supported in part by the Research Management Center (RMC), Universiti Teknologi, Malaysia, under GUP research grants No. Q.J130000.2523.02H91, and in part by the Japan Society for the Promotion of Science (JSPS) KIBAN (B) No. 2360170.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Mohd Azri Mohd Izhar.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ original submitted files for images

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Mohd Izhar, M.A., Fisal, N., Zhou, X. et al. Exploitation of 2D binary source correlation using turbo block codes with fine-tuning. J Wireless Com Network 2013, 89 (2013).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


  • Convolutional Code
  • State Transition Probability
  • Additive White Gaussian Noise Channel
  • Entropy Rate
  • Source Correlation