Karhunen-Loève-Based Reduced-Complexity Representation of the Mixed-Density Messages in SPA on Factor Graph and Its Impact on BER

2.1. General FG-Based Processing

The FG/SPA is unambiguously given by the FG structure, update rules and scheduling algorithm [1, 2]. To enable the processing, it is necessary to store the messages in iterations between updates. The implementation of the FG/SPA gives exact results for an arbitrary cycle-free FG with exact evaluation of the update rules, exact message representation and with an arbitrary scheduling algorithm, which considers all messages in the FG. When all of these assumptions are fulfilled, the FG/SPA provides an elegant optimal evaluation algorithm. As an example, the forward/backward Bahl-Cocke-Jelinek-Raviv algorithm [1] might be mentioned.

The FG/SPA, however, often works well also in cases when the mentioned conditions are violated. First of all, the FG might contain loops. In such a case, the FG/SPA works only approximately. Many of iterative algorithms such as iterative decoding are solvable utilizing the looped FG/SPA. Several works focused on the convergence criteria for the looped FG/SPA [3, 4]. The role of the scheduling algorithm becomes important for the looped FG/SPA. A number of results was proposed in [5].

In a contrast with these principal difficulties, the representation of the messages (and corresponding update rules) forms an implementation-related problem.

2.2. FG Processing with Mixed Continuous and Discrete Variables

The most straightforward representation is a discretization (sampling) of the continuous message. The message is represented by a piecewise-constant function. An exact (we mean exact with respect to the definition of the SPA) evaluation of the update rules is approximated by the numerical integration with the rectangular rule [6–8]. The discretization of the continuously valued message is straightforward but highly inefficient in terms of the number of coefficients required to obtain a given fidelity goal. This representation was adopted as a reference model in this paper (Section 3.4).

The continuously valued message in FG/SPA stands for a PDF up to a scale factor. Thus the message can be easily described by its moments. The main interest is focused on the Gaussian message, which is fully described by its mean and variance. The Gaussian representation is extremely suitable for all linear models (only superposition and scaling factor nodes are allowed). The update rules are then closed-form operations on the Gaussian messages. See [8] for details.

Nevertheless, the use of the Gaussian representation might bring good results also in nonlinear models (e.g., joint phase estimation and data detection [9]). A mixture Gaussian message (the message given by superposition of several Gaussian kernels) might be also used as a message representation. A common problem of the Gaussian mixtures is the increasing number of mixtures in the update rules. A mixture number reducing approach based on the approximation of the resulting PDF was considered, for example, in [10, 11].

Some authors consider alternative methods of the message representation such as a representation by a single point, function value and a gradient at a point [6, 7] or a list of samples [6, 8, 12].

2.3. Canonical Representation of Mixture Densities

A unified design framework based on the canonical distribution was proposed in [13]. This design consists of a set of kernel functions and related parameters describing the message. The sets of the parameter are passed through the FG/SPA instead of the continuous messages. Following this framework, the iterative decoding algorithms based on Fourier and Tikhonov parameterization were proposed in [14]. The parameterizations are suited for the channels affected with strong phase noise. The Fourier and Tikhonov parameterizations are, however, chosen only by inferring the suitable shape from the given particular application scenario. No systematic general procedure is developed.

2.4. Goals of this Paper and Contributions

3.1. Core Principle

This section summarizes the core principle in a "barebones" manner. The details follow in the sections below.

Let us assume an FG model with mixture PDF messages and an FN update algorithm (e.g., SPA). We assume the FG containing cycles with an iterative update evaluation (e.g., by a flooding algorithm). A particular shape of the message describing a given variable depends on (1) random observation inputs of the FG (the received signal) and (2) an iteration number of the network for other given parameters (SNR, preamble, etc.). Let us denote the true message evaluated in the FG/SPA without any implementation issues by . It is a randomly parameterized function (by ) in variable. As such, it can be approximated by a linear superposition of kernels (basis functions)

(1)

Expansion coefficients are random. The message is then fully represented by the vector .

A particular form of this expansion should provide an efficient (minimal dimensionality) representation of the message with well-defined fidelity criterion. The KLT can serve for this efficient representation. It provides orthonormal kernels based on the second-order message statistics. The resulting coefficients are uncorrelated. The second-order moments of the coefficients are also directly related to the residual MSE of the approximation.

The second-order statistics (the correlation function) of the true messages can be easily numerically approximated (by simulation) by an empirical correlation function. A reduced complexity approximation of the message is obtained by truncating the dimensionality of the original vector . Due to the orthonormality of the basis, the residual MSE will be purely additive as a function of the truncation length. Significantly contributing kernels are easily identified by the second moment of the corresponding coefficient. This gives an easy and direct relation between the description complexity and the approximation fidelity.

3.2. KLT Message Representation Details

The analysis is built on the stochastic properties of the message . We assume the message to be a real valued function of argument , where is an interval. Furthermore, we assume the existence of the integral (3) and the message being from space. The autocorrelation function of the message is given by

(2)

where and stands for the expectation over the set of iterations (we can consider an arbitrary subset of all iterations) and the observation vector.

Once the autocorrelation function is given, the solution of the characteristic equation

(3)

provides the eigenfunctions as a canonical basis of the message. We index the sorted eigenvalues such as for all : and the eigenfunction is indexed if and only if the pair forms eigenpair, that is, it solves (3).

Using the orthonormal property of the KLT-basis system, we obtain the expansion coefficients as

(4)

These coefficients jointly with the set of eigenfunctions describe the message by (1).

The complexity is reduced by omitting several components. We neglect the components with index , where stands for the number of used components (dimensionality of the message). Then we can easily control the MSE of the approximated message by the term .

Note that, as a result of the KLT-approximation, the message might become negative at some points, that is, there may exist such a number that . It violates assumptions of the almost all FN update algorithms and it must be rectified by a proper translation.

3.3. Empirical Correlation Function Measurement

The evaluation of the autocorrelation function is the key issue of the evaluation of the kernel basis functions. A direct calculation using (2) is sometimes difficult to be done, especially for complex models. If the continuous message is approximated by a piecewise constant function (see Section 3.4 for details), the autocorrelation matrix might be empirically estimated by

(5)

where stands for number of iterations, stands for number of realizations and is a discrete vector resulting from the discretization of the message . A discrete form of the characteristic equation (3) is given by , where denotes again the eigenvalues as in (3) and is the eigenvector, which is assumed to be the discretized eigenfunction . The evaluation of the expansion coefficients (4) might be done by

(6)

Finally, the message is represented by

(7)

Of course, the correlation evaluation requires small discretization steps. But since this operation is done only off-line during the system design phase, its complexity is not an issue at all.

3.4. Reference Message Representation Models

Our goal is to compare the capabilities of the message types (KLT against others) to represent the reference message as exactly as possible. We assume that we use a reference model without any implementation issues affecting the message representation and the update rules. Thus we are not interested in the update rules for particular representations. This is an important difference in contrast to other works (e.g., [6, 14]), where the authors try to obtain the message representation jointly with the update rules.

For our analysis, we need only an unambiguous relation between the reference (possibly highly complex) message and its approximation. In each iteration, the relation is used for the evaluation of the approximated message which is then inserted into the run of the reference model instead of the original reference message during the analysis. The results of this analysis might be interpreted as the ideal behavior of the particular message representation with an exact implementation of the update rules.

All considered representations in this section are only based on a deterministic description. Thus we might lighten the notation a little bit. The reference message is denoted by . We suppose the following representations.

3.4.1. Sample Representation

The discretization of the continuous message is a straightforward method of the practically feasible representation as it was discussed in Section 2.2 or [6, 12]. An arbitrary precision might be achieved using this representation (of course at the expense of the high complexity). Nevertheless, the representation offers a direct way to the empirical evaluation of the autocorrelation function (see Section 3.3). Thus it is a suitable option for our reference model.

The reference message is represented by the vector , where . And the approximated continuous message is then composed as a piecewise function from the samples

(8)

where , for , and otherwise.

The sample representation is considered in two cases. The first one is the reference model, where we select as many samples as the approximation of the message can be neglected. We also use the representation by samples to be compared with the proposed KLT-message for a given dimensionality.

3.4.2. Fourier Representation

The well known Fourier decomposition enables to parameterize the message by the Fourier's series as

(9)

where and . Dimensionality is given by . In [14], the authors have derived update rules for the Fourier coefficients in a special case of random-walk phase model.

3.4.3. Dirac-Delta Representation

The message is represented by a single number situated in the maximum of the message. In [7, 8] the authors present the gradient method to obtain , which is the main part of their work. Nevertheless, from our point of view, the message might be easily obtained from the reference message. Therefore it is selected to be compared with the proposed representation. The message is given by

(10)

3.4.4. Gaussian Representation

Gaussian representation is widely used in literature as a message representation (e.g., [10]). We consider the simplest possible scalar real-valued Gaussian message given by the pair with the interpretation

(11)

where and .

4.1. System Model

We assume several models situated in the signal space (Figure 1) and an MSK model situated into the phase space (see Figure 2).

4.2. Factor Graph of the System at Hand

The FG/SPA is used as a joint synchronizer-decoder (see Figure 3) for all mentioned models. Note that the FG for the considered models might be found in the literature (phase space model in [9] and signal space models in [6, 14]).

One can see, that for both models. The FG is depicted in Figure 3. We shortly describe the presented factor nodes and message types presented in the FG.

4.2.1. Factor Nodes

We denote and then we use as the phase distribution of the RV given by , where stands for the zero mean complex Gaussian RV with variance ; and .

1. (i)

   Factor Nodes in the Signal Space Models

    

Phase Shift (PS):

(12)

AWGN (W):

(13)

1. (ii)

   Factor Nodes in the Phase Space Model

    

Phase Shift (PS):

(14)

AWGN (W):

(15)

1. (iii)

   Factor Nodes Common for Both Signal and Phase Space

    

Random Walk (RW):

(16)

Other Factor Nodes

Other factor nodes such as the coder, CPE, and signal space mappers FN are situated in the discrete part of the model and their description is obvious from the definition of the related components (see, e.g., [1] for an example of such a description).

4.3. The FG/SPA Reference Model

The empirical stochastic analysis requires a sample of the message realizations. Thus we ideally need a perfect implementation of the FG/SPA for each model. We call this perfect or better said almost perfect FG/SPA implementation as the reference model. Note that even if the implementation of the FG/SPA is perfect, the convergence of the FG/SPA is still not secured in the looped cases. We call the perfect implementation such a model that does

not suffer from the implementation-related issues such as an update rules design and a messages representation. The flood schedule message passing algorithm is assumed. The reference model might suffer (and our models do) from the numerical complexity and it is therefore unsuitable for a direct implementation.

Prior we classify the messages appearing in the reference FG/SPA model (Figure 3) and their update rules, we found the following notation. We denote the message from variable node to factor node and the opposite message is denoted by and the factor node RW, which lies between -th and -th section according to Figure 4. Analogously, stands for the FN between -th and -th section.

.

4.4. Scenarios

We specify four scenarios for the analysis purpose. All of the scenarios might be seen as a special case of the system model described in the Section 4.1. All scenarios use the FG/SPA containing the loops, except the first one.

4.5. Eigensystem Analysis

The first objective is to investigate the eigensystem of the mixture messages. We demonstrate the analysis by numerical evaluation of the eigenvalues and eigenvectors for various scenarios mentioned before.

The main result of the eigensystem analysis consists in the observed fact, that the KLT of the messages in all considered models leads to the eigenfunctions very similar to the harmonic functions independently of the parameters of the simulation as one can see in Figure 5. It is also independent of the other parts of the scenario such as coder or mapper (see Figure 7).

The dimensionality of the message is upper bounded by number of samples in the reference message in our approach. The eigenvalues resulting from the analysis offer important information for the approximation purposes as it was discussed in Section 3.2. The eigenvalues resulting from the characteristic equation are shown in Figure 6 for the MSK modulation. The eigenvalues of the other models look very similar. The floor is caused by the finite floating precision. As one can see, the higher SNR, the slower is the descent of the eigenvalues with the dimension index. The curves in the plots point out to the fact that the eigenvalues are descending in pairs that is, .

4.6. Relation of the MSE of the Approximated Message with the Target Criteria Metrics

The KLT-approximated message provides the best approximation in the MSE sense. The minimization of the MSE of the approximated message, however, does not guarantee the minimization of the target criteria metrics such as MSE for the phase estimation or BER for data decoding. We have therefore performed several numerical simulations to inspect the behavior of the KLT-approximated message. We also consider the message types mentioned in the Section 3.4 into our simulation.

Few notes are addressed before going over the results. The MSE of the phase estimation is computed as an average over all MSE of the phase estimates in the model. The measurement of the MSE is limited by the granularity of the reference model. The simulations of the stochastic analysis are numerically intensive. We are limited by the computing resources. The simulations of the BER might suffer from this, especially for small error rates. The threshold of the detectable error rate is about for the uncoded QPSK model and for the BICM model.

4.6.1. Simulation Results for the Uncoded QPSK Modulation

We start with the results in cycle-free FG (see Figures 8 and 9). One can see several interesting points. First of all, the Fourier representation gives absolutely equal results as the KLT representation for both MSE and BER target metrics. Due to the shapes of the eigenfunctions, this result is not very surprising. One can see that evaluated according to (1) and (9) is equal when the set of basis functions in (1) is given exactly by the harmonic functions. However, it has a significant consequence. The harmonic function-based linear basis optimizes the MSE at least in the models considered in this analysis.

Another interesting point might be seen in Figure 9. Adding the sixth component to KLT (and also Fourier) canonical representation, the performance is slightly worse than the five-component approximated message. It means that the proportional relationship between MSE of the approximated message and BER does not work, at least in this given case.

The representation by samples does not seem to work well. It is probably caused by relatively high SNR. A few samples hardly cover the narrow shape of the message. The limitation of the Gaussian message is given by its incapability to describe the phase in vicinity of 0 and . Relatively good result is achieved using the Dirac-Delta message.

4.6.2. Simulation Results for the BICM

The last measurement was performed with the BICM model for SNR=8 dB. As it was mentioned, the randomness of the message is given not only by the iteration and the observation vector, but also by the position in the FG (of course only the messages and ).

The results of the analysis are shown in Figures 11 and 10. The first point is that the KLT message representation does not give the same results as the Fourier representation. The KLT-approximated message seems to converge a little bit faster than the Fourier representation up to approx. 45th iteration, where the KLT-approximated message achieves the error floor. There are two possible reasons for the error floor appearance (see Figure 11). First, the eigenvectors which constitute the basis system are not evaluated with a sufficient precision. Second, the evaluated KLT basis is the best linear approximation in average through all iterations and this basis is not capable to describe the messages appearing in the higher iterations sufficiently. If we focus on the issue discussed in the last model, where the 5-component message outperforms the 6-component one, we might observe this artifact again. A similar point might be seen in Figure 10 for both KLT and Fourier representations, where the 3-component messages outperform the 4-component messages. We can remind the point discussed in the eigenvalues section about the pairs of the eigenvalues. It seems (roughly said) the eigenfunctions work in pair so that adding only one of the pair might have a slightly negative impact for the target metrics.

Furthermore, we can observe a good behavior of the Dirac-Delta message in the BER measurement case. The MSE of the phase estimation, however, does not give such good results for the Dirac-Delta representation.