A survey of three-dimensional turbo codes and recent performance enhancements

2.1 Properties of three-dimensional turbo codes

where 0 ≤ ρ ≤ 1 is the fraction of the surviving bits in y₁ and y₂ after puncturing. In fact, we have k systematic bits, P = 2 × λ × k post-encoded parity bits and 2 × (1 − λ) × ρ × k surviving parity bits after puncturing. For example, if the overall coding rate is $R=\frac{4}{5}$, then it is necessary to have a permeability rate $\lambda \le \frac{1}{8}$. Moreover, the authors in [2] show that the probability of error at each decoder intrinsic input has risen by a factor $\frac{1+\left(1+\lambda \right)R}{1+R}$, inducing a loss in convergence. In fact, the more redundancies are post-encoded, the less redundant information at the first iteration the decoder will have, then causing more errors at its output. On the other hand, a large value of λ means a higher MHD. Therefore, λ is a key parameter to choose the compromise between distance and convergence.

The choice of the post-encoder influences also the performance in both the waterfall and error floor regions. It has to meet different requirements already detailed in [2, 3, 6]. In [7], we justified the choice of the post-encoder by means of EXIT analysis. A four-state binary convolutional encoder is used in practice. The RSC code with feedback polynomial 1 + D² and parity generator polynomial 1 has been selected to be the post-encoder in different simulations of the 3D TC.

The 3D TC is characterized by two permutations denoted by π and π′, as shown in Figure 1. π is the internal permutation of the TC, and π′ is used to spread a fraction λ of the parity bits before feeding them to the post-encoder. Let i and j be the address in the natural order and in the permuted order, respectively. For the regular permutation, we assume π′ to be defined by the following congruence relation: i = π ' (j) = (P₀j + i₀) P, where i ₀ is the starting index, and P ₀ is an integer relatively prime with P. For each block length, these parameters have to be carefully chosen to guarantee a large spread. It was observed through the different simulations that the important property is the spread. In the sequel, π′ corresponds to the regular permutation achieving a spread of $\sqrt{2P}$[12], where P is the size of the frame to be post-encoded. The 3D TC performs better with regular π′ than with a random interleaver in terms of MHD and convergence. Details are available in [7].

2.2 Performance of 3GPP2 three-dimensional turbo codes

We have investigated the distance gain and the effect on TC convergence threshold for different block sizes, coding rates and permeability rates. Similar to the case of double-binary codes in [2, 3], we have observed that the addition of the post-encoder improves the asymptotical behaviour of the 3GPP2 TC in many cases. Table 1 presents examples of MHD values obtained with this code for different block sizes and coding rates, using the all-zero iterative decoding algorithm [13]. We can observe that the direct application of the third coding dimension to the existing code leads to an increase of its minimum distance, except in the case of high coding rates. The authors in [14] analyzed the asymptotic weight distribution of 3D TCs and showed that their typical minimum distance may, depending on certain parameters, asymptotically grow linearly with the block length. Table 1 shows that it is attractive to increase λ, since larger minimum distances are obtained. However, this will be paid in terms of loss in convergence threshold, as explained above. Also, the increase in complexity is not negligible. In [2, 3], the computational complexity increase was estimated to be less than 10% with respect to classical two-dimensional TC. This first estimation of the complexity increase was optimistic. In fact, 3D TC hardware implementation issues have been thoroughly investigated in [8]. We proposed an appropriate hardware architecture of a 3D turbo decoder and the corresponding complexity analysis. When high throughputs are required for a given application, several processors can be placed in parallel, thus decreasing the relative additional complexity of the 3D coding scheme. For example, the computational complexity increase is 36% when only two processors are placed in parallel, for k = 1,530 bits, $R=\frac{1}{2}$ and $\lambda =\frac{1}{8}$. It is 25% with four processors. An increase of less than 10% in the memory requirements is also necessary for 3D TCs.

3.1 Convergence threshold of three-dimensional turbo codes

The convergence thresholds of 3GPP2 3D TCs are estimated through an EXIT chart analysis [15] on the additive white Gaussian noise (AWGN) channel and on the Rayleigh fading channel as well. To generate the EXIT chart of a TC, we have to consider the transfer characteristics of the extrinsic information for each soft-input soft-output (SISO) decoder. In the case of 3D TC, the two SISO decoders exchange extrinsic information about the systematic part of the received codeword, like for classical turbo decoding. However, both of them exchange also extrinsic information about the post-encoded parity bits with the four-state SISO predecoder, and we have to take into account in the EXIT chart that the extrinsic information about these parity bits is changing from an iteration to the other. Consequently, the curves of mutual information exchange between the two decoders change every iteration. The convergence threshold of the code is the minimum SNR where the only intersection point between the two transfer characteristics curves is the point (1,1). Different convergence thresholds of 3GPP2 3D TCs, for several code rates and values of λ, are given in Table 2. For a fixed code rate R, the best convergence threshold is achieved by the classical TC (λ = 0). The loss in the convergence threshold increases with λ. In Table 3, we compare the loss in convergence due to the use of 3D TCs for transmissions over the Rayleigh fading channel and over the AWGN channel at coding rate $R=\frac{1}{2}$ and $\lambda =\frac{1}{8}$. Table 3 shows that the loss of convergence is more significant over the Rayleigh fading channel. This result represents a drawback of 3D TCs. In fact, for applications such as the terrestrial mobile radio communications where the simulations are carried out over fading channels, the loss of convergence can be significant. However, the 3D TCs proved to be suited to transmissions over Gaussian channels. Thus, they can be used in applications such as fixed satellite communications. In the following subsections, we propose two techniques to remedy these convergence issues.

3.2 Time-varying three-dimensional turbo codes

So far, we have only considered fixed or time-invariant (TI) convolutional codes. TV convolutional encoders have generator polynomials which periodically vary with time. The idea appeared for the first time in the paper of Costello [16]. The author conjectured that TV convolutional codes have larger free distances, d_free, than fixed convolutional codes with the same rate and constraint length. This conjecture led to the search for periodic TV convolutional codes that are better than TI convolutional codes with respect to d_free. We have investigated a simple time-varying post-encoder in order to increase locally its minimum distance. This improves the level of the extrinsic information provided by the predecoder to the two SISO decoders, which reduces the loss of convergence threshold for 3D TCs. The proposed TV encoding technique consists in alternating two redundancy polynomials in time, instead of having only one. In the case of the four-state selected post-encoder, the octal polynomials W₁ = 4 and W₂ = 7 are alternated for the parity. Also, we replace some redundancies W₁ = 4 by other redundancies W₂ = 7. The replacement period is denoted by L. In Figure 2, time-varying 3GPP2 3D TCs have been simulated over a Rayleigh fading channel. Figure 2 shows the bit error rate (BER) performance for blocks of 2,298 bits at code rate $R=\frac{1}{3}$ and $\lambda =\frac{1}{8}$. In this case, the loss of convergence due to the hybrid structure is reduced by 35% from 0.23 to 0.15 dB. Even if the Rayleigh fading simulation in Figure 2 appears to show that the classical TC performance is the best, this is not the case at high error rates since the minimum distance of the 3D TC is higher: d_min = 36 compared to d_min = 25 for the classical TC.

Among all the simulations carried out, it was observed that the time-varying parity construction reduces the loss of convergence by 10% to 50% of the value expressed in decibels. We have systematically checked that the asymptotic performance is not degraded. In fact, the choice of the post-encoder does not influence a lot the minimum distance of the 3D TC for a fixed code memory and for a given permeability rate λ. For the time-varying post-encoder, there is no reason to improve or degrade the asymptotic performance since we introduce few local modifications. However, the higher the local minimum distance of the post-encoder, the better the level of the extrinsic information which the predecoder supplies to the two SISO decoders. Therefore, the TV technique acts as a convergence accelerator of the 3D TC. In addition, we have observed that the EXIT of a time-varying 3D TC changes with the value of L. This tool can be used to choose an optimal value of L. The selected value can be the L that produces the lowest convergence threshold.

3.3 Three-dimensional turbo codes for high spectral efficiency transmissions

We have also investigated the association of the 3D TC with high-order modulations. This structure is used for applications where high data throughputs are required such as the transmission of high-definition television (HD TV). In the most recent transmission systems, high bit rates require using high-order modulations, such as 16-QAM for 3GPP2, 64-QAM for LTE and 256-QAM for DVB-NGH.

We consider a coded modulation scheme based on the so-called pragmatic or bit-interleaved coded modulation approach [17]: a turbo encoder and a modulator that follows an interleaver and a Gray mapper. It is known that among the bits forming a Gray-labelled symbol in M-QAM or M-PSK modulations for M > 4, the average probability of error is not the same for all the bits [18]. Therefore, three constellation mappings, all compliant with Gray labelling, were investigated. First, the mapping is uniformly distributed on the entire constellation. In a second configuration, the systematic bits are mapped to better protected places as a priority, and all the other bits are uniformly distributed. For the third mapping, the systematic bits are first mapped to better protected places as a priority. Then, if it is possible, the post-encoded parity bits are better protected by the considered modulation than the other non-re-encoded parity bits. This choice is made because the systematic bits as well as the post-encoded parity bits are used by both decoders during the decoding process. Thus, protecting them is expected to reduce the loss of convergence. At the receiver side, the demapper computes the log likelihood ratio (LLR) related to each bit of the information sequence. This symbol-to-bit LLR calculation is followed by a 3D turbo decoder using the MAP algorithm. The application of the proposed transmission scheme in two particular cases of the 3GPP2 3D code is presented below. In both cases, we consider a transmission over a Rayleigh fading channel that has been shown to be a challenging type of channel for 3D TCs.

3.3.1 Example 1: 3D TCs associated with a 16-QAM modulator

Among the four bits forming a symbol with Gray labelling in 16-QAM, the average probability of error is smaller for the first and the third bits than for the second and fourth bits. To explore this property, frame error rate (FER) performance of 3GPP2 3D TCs has been simulated with $\lambda =\frac{1}{8}$ at code rate $R=\frac{1}{3}$ for k = 2,298 bits, as shown in Figure 3. It was observed through these simulations that the use of the 16-QAM modulation, where the systematic bits as well as the post-encoded parity bits are more protected than the other non-re-encoded parity bits, allows the loss of convergence of the 3D 3GPP2 TC to be reduced and even be transformed into a gain in the waterfall region compared with the 3GPP2 standardized TC.

3.3.2 Example 2: 3D TCs associated with an 8-PSK modulator

Among the three bits forming a symbol with Gray labelling in 8-PSK, the average probability of error is smaller for the first and the second bits than for the third bit. We have simulated the FER performance of 3D TCs with $\lambda =\frac{1}{8}$ at code rate $R=\frac{4}{5}$ for k = 1,146 bits. For this coding rate, the third configuration cannot be adopted, and we are obliged to implement the second configuration. Simulations in Figure 4 show that the use of an 8-PSK modulation, where the systematic bits are protected as a priority, allows a significant gain in the waterfall region of 0.5 dB compared with the 3GPP2 standardized TC. This gain is more significant than the one obtained in Figure 3.

The investigation of the previous three Gray mappings allows some rules of conception to be defined. In fact, a loss of convergence is still observed when the bits in a symbol are uniformly distributed on the entire constellation. However, when the second configuration or the last one is used, the loss in the convergence threshold disappears and a gain in the waterfall region is observed. When these both configurations can be implemented, the configuration 3 must be used as far as possible. Otherwise, it is necessary to implement at least the configuration 2, as in the example presented above for the 8-PSK. When the systematic bits are first protected as a priority and, if possible, the post-encoded bits, a significant gain is obtained. The technique is all the more interesting as the coding rates are higher for the same value of λ, even for transmissions over fading channels. In other words, the problem related to the loss in the convergence threshold of 3D TCs can be solved.

To conclude, we notice that the notion of irregularity plays a major role to build 3D TCs having good performance at low and high SNRs. In fact, the optimization method, first presented in [6], is based on the use of a non-regular pattern of post-encoding and allows increasing the minimum distance. Besides, the time-varying post-encoder with a little irregularity and also the association of the 3D TC with high-order modulations, where both the systematic bits and the post-encoded parity bits are more protected than the other parity bits, create a sort of irregularity in the Gray mapping. Both represent a success in reducing or even eliminating the problem of convergence loss. The next step of the study concerns the investigation of irregular TCs. The aim is to obtain an irregular TC which performs well in both the waterfall and the error floor regions.

4.1 Another representation of turbo codes

In most cases, the two (or more) constituent RSC encoders of a parallel TC are identical. For this reason, the authors in [19] proposed a ‘self-concatenated’ turbo encoder, depicted in Figure 5. It consists of the concatenation of a repetition code and an RSC code separated by an interleaver. In fact, it is possible to merge the two trellis encoders and replace the initial interleaver with a double-sized interleaver preceded by a twofold repetition (d-fold repetition in general). The interest of this second equivalent encoding structure of a classical TC lies in its simplicity and in the opportunity of introducing an irregular structure. By adopting the terminology used for the low density parity check (LDPC) codes, a regular TC is related to the use of a uniform repetition in the equivalent encoding structure (see Figure 5). On the other side, when the repetition degree d is not the same for all the information bits, the TC is said to be irregular. Note that the irregular TC implemented in Figure 5 is a generalization of repeat-accumulate codes presented in [20].

The performance of the self-concatenated regular TC is identical to the one we get using the standard turbo encoder when the interleaver length is sufficiently high. For short and medium block sizes, a few additional iterations are necessary to achieve the same performance. This is due to the decoding process because no extrinsic information is available at the first iteration. It is possible to implement a shuffled iterative decoding [21, 22] in order to achieve the same performance as for the classical turbo encoder and avoid adding iterations in practice. In shuffled decoding, the decoder updates the extrinsic information as soon as possible, without waiting for all the copies of a given data to be processed. In other words, the decoder does not wait until the next iteration to send extrinsic messages. However, as it does not represent a major problem, we have chosen to increase the number of iterations in our simulations by two additional iterations for the classical sequential decoding.

For instance, the BER performance of the 3GPP2 TC has been simulated for blocks of 2,298 bits at coding rate $R=\frac{1}{3}$. Figure 6 shows that the use of the equivalent encoding structure for a regular TC requires two additional iterations (i.e. 12 instead of 10) to reach the performance of the classical parallel TC.

An irregular TC consists of the concatenation of a non-uniform repetition, an interleaver and a RSC code. In fact, the introduced irregularity makes it possible to improve the performance of a TC by inserting some bits inside the RSC trellis with a degree d > 2. These high-degree bits are commonly called pilot bits: they have reliable forward and backward metrics in the decoding process, and they propagate on both directions and influence the other bits with degree d = 2. However, making the code irregular leads to an increase in the rate of the constituent codes. Therefore, for usual coding rate values, only a small fraction of information bits is repeated d > 2 times. Thanks to their higher degree, the pilot bits include d extrinsics instead of two and are thus extremely well protected.

4.2 Selecting the degree profile

For irregular TCs, the information bits are divided into classes, each class j having a specific degree d _j = j. The fraction of bits of degree d _j in a class j is denoted by f _j , where f _j  ∈ [0, 1]. A degree profile consists of all the degrees d _j and their corresponding non-zero fractions f _j . In the sequel, we represent a degree profile by the vector (f₂, f₃,…, f_max) or the vector (2, 3,…, max). The average information bit degree is $d_{\mathit{Average}}={\displaystyle \sum _{j=2}^{\max }{d}_j{f}_j}$. We keep the minimum degree equal to 2 in order to refer to the classical TC. The maximum degree is d_max.

The convergence threshold as well as the asymptotic performance of an irregular TC strongly depends on its degree profile. The best profile depends on the interleaver and the generator polynomials of the RSC code. For codes with very large block lengths, the optimization of the previous parameters can be done using the density evolution method developed by Richardson and Urbranke [23, 24]. Using this approach, irregular LDPC codes with performance at 0.0045 dB from the capacity were obtained [25]. The Gaussian approximation can be used to speed up the search for good parameters. This sub-optimal method leads to quite accurate results and was defined in several different ways [26–29]. Although the density evolution method and the Gaussian approximation approach can be used to select a good degree profile for codes with large block sizes, Monte Carlo simulations of the bit error probability are carried on for finite length. The main drawback is that they are time-consuming, and only profiles with two non-zero fractions can be considered. In fact, the method consists in fixing a degree d_Irreg and varying its fraction f_Irreg. A fraction that achieves the best performance can be found. The next step involves changing the degree d_Irreg, while the fraction f_Irreg is fixed to the value already selected. We can then find optimal values for both d_Irreg and f_Irreg. However, this profile is not automatically the best one, since the optimization does not take into account all the possible combinations (d_Irreg, f_Irreg). Also, better performance may be attained when the profile is not restricted to two non-zero fractions. A simple method, based on the EXIT diagrams to select a good profile without resorting to extensive and long simulations, was introduced in [9]. This method allows many degree profiles to be compared at the same time. Comparisons between different degree profiles in terms of convergence behaviour as well as asymptotic behaviour are made, and the selection of the best degree profiles can be progressively refined.

The application of this technique led to the choice of two degree values d = 2 and d = 8 with the following degree profile: $\left(f_2=\frac{5}{6},f_8=\frac{1}{6}\right)$. For reasons of simplicity, only two degrees have been considered, and d = 2 is fixed with reference to the classical TC. In the sequel, we adopt this profile as working assumption. In fact, compared to the regular TC, the irregular TC with degree profile (f₂, f₈) offers a significant gain in the waterfall region of 0.3 dB. However, this code has a poor performance in the floor. The following subsections focus on the improvement of performance at high SNRs.

4.3 Design of suitable permutations for irregular turbo codes

The first intuitive idea investigated was to design an interleaver where all the groups of eight bits are uniformly distributed. The objective is that they spread their reliable forward and backward metrics along the frame. On the other hand, the spread between the pilot groups should be large enough to avoid correlation between them. High correlation may dramatically degrade error correction performance and even ruin the possible gain due to a large minimum distance. An empirical value for the spread is to be at least equal to 2 × (ν + 1), where v is the memory length of the simulated code. The condition on the spread between the pilot groups imposes a constraint on the fraction f₈ (f_max in general) of the pilot bits: $f_{\max }\le \frac{d_{\mathrm{av}}}{2\times d_{\max }\times \left(\nu +1\right)}+\frac{1}{k}$, where d_av is the average information bit degree and k is the total number of information bits. For usual block size values, the term $\frac{1}{k}$ is negligible, and the constraint above is a relation between f_max, d_max and v that can be expressed as follows: $f_{\max }\le \frac{d_{\mathrm{av}}}{2\times d_{\max }\times \left(\nu +1\right)}$. For $f_8=\frac{1}{6}$, d_av = 3 and ν = 3, this condition can never be satisfied.

In the cases where this condition is not satisfied, we proposed an algorithm in order to jointly spread the groups of eight bits along the frame and maximize the MHD [10]. This idea was inspired by a procedure described in [30] where the author focuses on the optimization of TC permutation design with the so-called almost regular permutation model. In [31], Kraidy et al. used a progressive edge growth-based interleaver for irregular TCs to lower the floor in the case of binary erasure channels. The algorithm we proposed is based on Dijkstra's algorithm [32] and on the estimation of the minimum distance for each permutation. To apply Dijkstra's algorithm in our specific context, we first represent the interleaver by a graph. At the beginning, the graph exists and it is empty. As we consider a tail-biting code, the graph has the form of a ring. The nodes, empty at first, are connected two by two in the ring. Thus, each node v _i is connected to only a predecessor and a successor v _j in the graph. The weight of each connection w _i,j is equal to 1. The purpose of the algorithm is to put addresses in the nodes. Before interleaving, an appropriate repetition is applied to each bit among the k bits stemming from the source. The addresses that appear progressively in the graph are the interleaved addresses corresponding to the output of the interleaver. For each bit with repetition degree d, we choose a random interleaved address π(j) for the first copy. Then, we compute the distances from π(j) for any vertex in the actual graph by Dijkstra's algorithm. The interleaved address of the second copy is chosen at random such that the score for π(j) is greater than α times the best possible value. Then, a connection must appear in the graph between v_π(j) and v_π(j+1), represented by a crossbar of weight equal to 0: w_{π(j),π(j+1)} = 0.

The algorithm constructs the graph progressively. At the end, $\frac{d\times \left(d-1\right)}{2}$ connections are added to the graph for every bit with degree d. These crossbars, of weight equal to 0, connect different interleaved data of the same information bit. Besides, the parameter 0 < α ≤ 1 is used to implement a random variation in the selection. The value α = 1 corresponds to the original Dijkstra's algorithm. We noticed that if we fix α = 1 in our algorithm, the obtained interleavers produce high values of girths but sometimes unacceptable values of minimum distances. The girth is the length of a shortest cycle contained in the graph. This criterion of selection was a priority for the author of [30]. In fact, his first purpose was to maximize the correlation girth while keeping an acceptable minimum Hamming distance. However, the minimum Hamming distance is our most important criterion of selection in order to improve the distance properties of irregular TCs. If the pilot bits are highly correlated, they may dramatically degrade error performance and even ruin the possible gain due to a large minimum distance. In order to reduce the auto-correlation effect, the interleaver should spread the pilot bits by increasing the correlation girth. However, the minimum distance can suffer since a long cycle has more probability to contain parity bits of the same data bit. Thus, we do not look for maximizing this criterion but only increasing the correlation girth and searching for higher minimum distances. Therefore, different values were tested for the parameter a, and we finally set a to 0.85, giving a reasonable space of search. Every time an interleaver is found, we estimate the minimum Hamming distance of the irregular TC using the all-zero iterative decoding algorithm [13]. Only interleavers that improve the asymptotic performance of irregular TCs are memorized.

The proposed algorithm allows suitable permutations to be designed that increase the minimum distance of irregular TCs. The pilot bits are now distributed along the frame in a way that they guarantee a good spread between the different groups, and the correlation effect between the pilot groups is reduced. This algorithm works very well for short block sizes [10]. For medium sizes and large blocks, the algorithm leads to a systematic search for an optimized interleaver in a wide domain of parameter values, due to the random selection introduced by parameter a. When the block size is some thousand bits, the algorithm may take an unacceptable computational time to find good interleavers, and we cannot be sure of detecting all the possible cases.

This algorithm was run for blocks of 1,146 bits with degree profile (f₂, f₈). As the average degree is d_av = 3, the interleaver length is equal to 3,438. Figure 7 compares the FER performance of the code under both random and optimized interleaving. A gain of two orders of magnitude in the error floor is observed, in favor of the optimal interleaver. Nevertheless, like for random interleavers, one additional drawback of this family of interleavers is the necessity to store the interleaved addresses as no equations are available for the permutation.

4.4 Adding a post-encoder to irregular turbo codes

The proposed algorithm in section 4.3 is only practicable for short to medium blocks. It is possible to investigate the interleavers provided by the proposed algorithm and explore them in detail in order to find structured interleavers, having similar properties, which can be described in an analytical way. However, as previously explained, devising permutations for turbo codes is not an easy task. In order to ensure large asymptotic gain at very low error rates, even with non-optimized internal permutation, we propose an irregular TC inspired by our work about 3D TCs in order to improve the distance properties of these codes [9]. A fraction 0 ≤ λ ≤ 1 of the parity bits are post-encoded by a rate-1 post-encoder. For the regular 3D TC, the increase in minimum distance is significant at the expense of a loss in convergence threshold and an increase in complexity. The same kind of behaviour is expected for irregular TCs.

Figure 8 shows the BER and FER performance of regular and irregular TCs for blocks of 2,046 bits and code rate $R=\frac{1}{3}$. Thus, the interleaver length is equal to 6,138. Compared with irregular TCs, the gain at high SNRs is nearly two orders of magnitude when the post-encoding is performed. Other simulations show that a gain of nearly 2.5 orders of magnitude can be observed for longer interleavers. Note that, unlike the method described in section 4.3, there is no limitation on the block size. The great advantage is that irregular TCs with post-encoding perform better than the regular TCs in both the waterfall and the error floor regions. It is possible to increase even more the minimum distance by the search of an adapted pattern of post-encoding. Here, the post-encoding is regular. However, it is possible to postcode only the pilot bits, or only the bits with the lowest degree, or to find a balance between both of them. This perspective is expected to give better results and will be investigated in a future work.