The properties of the proposed method are demonstrated using different models. First, the models are introduced, then the FG of the models including the reference case is discussed. Finally, the numerical results obtained from the models are figured and discussed.
4.1. System Model
We assume several models situated in the signal space (Figure 1) and an MSK model situated into the phase space (see Figure 2).
4.1.1. Signal Space Models
We assume a binary i.i.d. data vector of length as an input. The coded symbols are given by . The modulated signal vector is given by , where is a signal space mapper. The channel is selected to be the AWGN channel with phase shift modeled by the random walk (RW) phase model. The phase as the function of time samples is described by , where is a zero mean real Gaussian random value (RV) with variance . Thus the received signal is , where stands for the vector of complex zero mean Gaussian RV with variance . The model is depicted in Figure 1.
4.1.2. Phase Space Model
We again assume the vector as an input into the minimumshift keying (MSK) modulator. The modulator is modeled by the canonical form, that is, by the continuous phase encoder (CPE) and nonlinear memoryless modulator (NMM) as shown in [15]. The modulator is implemented in the discrete time with two samples per symbol. The phase of the MSK signal is given by , where is the th sample of the phase function, is the th state of the CPE and the sampled modulated signal is given by . The communication channel is selected to be the AWGN channel with constant phase shift, that is, , where stands for the received vector, is the constant phase shift of the channel and is the AWGN vector. The nonlinear limiter phase discriminator (LPD) captures the phase of the vector , that is, . The whole system is shown in Figure 2.
4.2. Factor Graph of the System at Hand
The FG/SPA is used as a joint synchronizerdecoder (see Figure 3) for all mentioned models. Note that the FG for the considered models might be found in the literature (phase space model in [9] and signal space models in [6, 14]).
Prior the description itself, we found a notation to enable a common description of the models. We define

(i)
, and , where for the signal space models with the RW phase model,

(ii)
, and , where for the phase space model with the constant phase model.
One can see, that for both models. The FG is depicted in Figure 3. We shortly describe the presented factor nodes and message types presented in the FG.
4.2.1. Factor Nodes
We denote and then we use as the phase distribution of the RV given by , where stands for the zero mean complex Gaussian RV with variance ; and .

(i)
Factor Nodes in the Signal Space Models
Phase Shift (PS):
AWGN (W):

(ii)
Factor Nodes in the Phase Space Model
Phase Shift (PS):
AWGN (W):

(iii)
Factor Nodes Common for Both Signal and Phase Space
Random Walk (RW):
Other Factor Nodes
Other factor nodes such as the coder, CPE, and signal space mappers FN are situated in the discrete part of the model and their description is obvious from the definition of the related components (see, e.g., [1] for an example of such a description).
4.2.2. Message Types Presented in the FG/SPA
The FG contains both discrete and continuous messages. The discrete messages are presented in the coder. There is no need for the investigation of their representation, because they are exactly represented by PMF.Several parameterizable continuous messages might be exactly represented using a straightforward parameterization (e.g., Gaussian message). These messages are presented in the AWGN channel model. The rest of the messages are mixed continuous and discrete messages. These mixture messages are continuously valued messages without an obvious way of their representation. The messages are situated in the phase model.
4.3. The FG/SPA Reference Model
The empirical stochastic analysis requires a sample of the message realizations. Thus we ideally need a perfect implementation of the FG/SPA for each model. We call this perfect or better said almost perfect FG/SPA implementation as the reference model. Note that even if the implementation of the FG/SPA is perfect, the convergence of the FG/SPA is still not secured in the looped cases. We call the perfect implementation such a model that does
not suffer from the implementationrelated issues such as an update rules design and a messages representation. The flood schedule message passing algorithm is assumed. The reference model might suffer (and our models do) from the numerical complexity and it is therefore unsuitable for a direct implementation.
Prior we classify the messages appearing in the reference FG/SPA model (Figure 3) and their update rules, we found the following notation. We denote the message from variable node to factor node and the opposite message is denoted by and the factor node RW, which lies between th and th section according to Figure 4. Analogously, stands for the FN between th and th section.
.
4.3.1. Discrete Type Messages
They are situated in the discrete part of the FG/SPA. As we have already said, their representation by PMF and the exact evaluation of the update rules according to the definition [1] are straightforward.
4.3.2. Unimportant Messages
The messages from PS factor node to the observation (, , , and ) lead to the open branch and neither an update nor a representation of them is required, because these messages cannot affect the estimation of the target parameters (data, phase).
4.3.3. Parameterizable Continuous Messages
The messages and are representable by a number meaning , and are representable by the pair meaning , , respectively. One can easily find the slightly modified update rules derived from the standard update rules. Examples of those may be seen in [12].
4.3.4. Mixture Messages
The representation of the remaining messages, that is, , , , , , and , is not obvious. These messages are thus discretized and the marginalization is performed using numerical integration with the rectangle rule [8, 12] in the update rules. The number of samples is chosen sufficiently large so that the impact of the approximation can be neglected. The mentioned mixture messages are real valued onedimensional functions for all considered models.
4.4. Scenarios
We specify four scenarios for the analysis purpose. All of the scenarios might be seen as a special case of the system model described in the Section 4.1. All scenarios use the FG/SPA containing the loops, except the first one.
4.4.1. Uncoded QPSK Modulation
The QPSK modulation is situated in the AWGN channel with RWmodel of the phase shift. The length of the frame is data symbols, the length of the preamble is 4 symbols and the preamble is situated at the beginning of the frame. The variance of the phase noise equals . This scenario is cyclefree and thus only inaccuracies caused by the imperfect implementation are presented. The information needed to resolve the phase ambiguity is contained in the preamble and, by a proper selection of the analyzed message, we can maximize the approximation impact to the key metrics such as BER or MSE of the phase estimation. We thus select the message to be analyzed.
4.4.2. Coded 8PSK Modulation
In addition to the previous scenario, the convolutional coder is presented. The length of the frame is data symbols, the length of the preamble is 2 symbols. The same message is selected to be analyzed .
4.4.3. MSK Modulation with ConstantPhase Model of the Phase Shift
The length of the frame is data symbols. The analyzed message is . These messages are nearly equal for all possible PS factor nodes (e.g., [12]).
4.4.4. BitInterleaved Coded Modulation
The model employs a bitinterleaved coded modulation (BICM) with convolutional code and QPSK signalspace mapper. The phase is modeled by the RW model with . The length of the frame is data symbols and 150 of those are pilot symbols. This model slightly changes our concept. Instead of the investigation of the single message, we analyze all and messages jointly. It means that all of the analyzed messages are approximated in the simulations and the stochastic analysis is performed over all investigated messages.
4.5. Eigensystem Analysis
The first objective is to investigate the eigensystem of the mixture messages. We demonstrate the analysis by numerical evaluation of the eigenvalues and eigenvectors for various scenarios mentioned before.
The main result of the eigensystem analysis consists in the observed fact, that the KLT of the messages in all considered models leads to the eigenfunctions very similar to the harmonic functions independently of the parameters of the simulation as one can see in Figure 5. It is also independent of the other parts of the scenario such as coder or mapper (see Figure 7).
The dimensionality of the message is upper bounded by number of samples in the reference message in our approach. The eigenvalues resulting from the analysis offer important information for the approximation purposes as it was discussed in Section 3.2. The eigenvalues resulting from the characteristic equation are shown in Figure 6 for the MSK modulation. The eigenvalues of the other models look very similar. The floor is caused by the finite floating precision. As one can see, the higher SNR, the slower is the descent of the eigenvalues with the dimension index. The curves in the plots point out to the fact that the eigenvalues are descending in pairs that is, .
4.6. Relation of the MSE of the Approximated Message with the Target Criteria Metrics
The KLTapproximated message provides the best approximation in the MSE sense. The minimization of the MSE of the approximated message, however, does not guarantee the minimization of the target criteria metrics such as MSE for the phase estimation or BER for data decoding. We have therefore performed several numerical simulations to inspect the behavior of the KLTapproximated message. We also consider the message types mentioned in the Section 3.4 into our simulation.
Few notes are addressed before going over the results. The MSE of the phase estimation is computed as an average over all MSE of the phase estimates in the model. The measurement of the MSE is limited by the granularity of the reference model. The simulations of the stochastic analysis are numerically intensive. We are limited by the computing resources. The simulations of the BER might suffer from this, especially for small error rates. The threshold of the detectable error rate is about for the uncoded QPSK model and for the BICM model.
4.6.1. Simulation Results for the Uncoded QPSK Modulation
We start with the results in cyclefree FG (see Figures 8 and 9). One can see several interesting points. First of all, the Fourier representation gives absolutely equal results as the KLT representation for both MSE and BER target metrics. Due to the shapes of the eigenfunctions, this result is not very surprising. One can see that evaluated according to (1) and (9) is equal when the set of basis functions in (1) is given exactly by the harmonic functions. However, it has a significant consequence. The harmonic functionbased linear basis optimizes the MSE at least in the models considered in this analysis.
Another interesting point might be seen in Figure 9. Adding the sixth component to KLT (and also Fourier) canonical representation, the performance is slightly worse than the fivecomponent approximated message. It means that the proportional relationship between MSE of the approximated message and BER does not work, at least in this given case.
The representation by samples does not seem to work well. It is probably caused by relatively high SNR. A few samples hardly cover the narrow shape of the message. The limitation of the Gaussian message is given by its incapability to describe the phase in vicinity of 0 and . Relatively good result is achieved using the DiracDelta message.
4.6.2. Simulation Results for the BICM
The last measurement was performed with the BICM model for SNR=8 dB. As it was mentioned, the randomness of the message is given not only by the iteration and the observation vector, but also by the position in the FG (of course only the messages and ).
The results of the analysis are shown in Figures 11 and 10. The first point is that the KLT message representation does not give the same results as the Fourier representation. The KLTapproximated message seems to converge a little bit faster than the Fourier representation up to approx. 45th iteration, where the KLTapproximated message achieves the error floor. There are two possible reasons for the error floor appearance (see Figure 11). First, the eigenvectors which constitute the basis system are not evaluated with a sufficient precision. Second, the evaluated KLT basis is the best linear approximation in average through all iterations and this basis is not capable to describe the messages appearing in the higher iterations sufficiently. If we focus on the issue discussed in the last model, where the 5component message outperforms the 6component one, we might observe this artifact again. A similar point might be seen in Figure 10 for both KLT and Fourier representations, where the 3component messages outperform the 4component messages. We can remind the point discussed in the eigenvalues section about the pairs of the eigenvalues. It seems (roughly said) the eigenfunctions work in pair so that adding only one of the pair might have a slightly negative impact for the target metrics.
Furthermore, we can observe a good behavior of the DiracDelta message in the BER measurement case. The MSE of the phase estimation, however, does not give such good results for the DiracDelta representation.