# A semi-analytical performance prediction of a digital communication system using Fourier transform inversion

- Fatima Ezzahra Naamane
^{1}, - Mohamed Et-tolba
^{2}and - Mostafa Belkasmi
^{1}

**2015**:236

https://doi.org/10.1186/s13638-015-0467-2

© Naamane et al. 2015

**Received: **28 March 2015

**Accepted: **9 September 2015

**Published: **29 October 2015

## Abstract

Analytical performance evaluation of a digital communication system remains a serious problem especially when a sophisticated digital signal processing is considered. Moreover, it is difficult to obtain the expected performance of such system using the Monte Carlo simulation method. In this paper, we propose a new semi-analytical approach for predicting error probability in a digital communication system. This approach is based on Fourier transform inversion formula to estimate the probability density function (pdf) of the observed soft sample at the receiver. Furthermore, we applied a bootstrap method for selecting the optimal smoothing parameter to make the proposed semi-analytical method more accurate. Simulation results show that the obtained semi-analytical error probability is close to the one measured using Monte Carlo simulation and provides a significant gain in terms of computing time. Besides, the use of the bootstrap method decreases the squared error between the true pdf and the estimated one.

### Keywords

Semi-analytical approach Fourier transform inversion Bit error probability pdf Smoothing parameter## 1 Introduction

Advanced wireless communication systems use sophisticated digital modulation schemes as well as space–time diversity in order to provide high data rates. The transmission quality of these systems is determined by the performance evaluation, which can be made using metrics such as the bit error probability (BEP), the block error probability (BLEP), or throughput. However, unified analytical expressions of these metrics are not available for several digital communication systems. The common method used to fix this problem is Monte Carlo simulation in which one has to simulate the transmitter, the transmission channel, and the receiver. Unfortunately, in complex systems, this method becomes very prohibitive in terms of computation time, and it requires a very large number of transmitted samples to estimate very low error probabilities. As a solution, semi-analytical performance prediction (SPP) has been proposed in recent years and it has been the subject of numerous studies. In [1], the authors have proposed the importance sampling (IS) method for BER prediction. It has been found that for simple memoryless systems (e.g., a BPSK modem [2]), the efficiency of the IS technique is high and its implementation is relatively easier. However, its accuracy can be severely degraded, especially when a complex system receiver is used. For this reason, Abdi et al. have proposed in [3] a low complexity prediction technique for turbo-like codes. It is based on estimating the probability density function (pdf) of the log-likelihood ratio (LLR) at the output of the decoder using a normal density as a reference. Nevertheless, it does not allow reducing the complexity of the iterative decoding algorithm. In [4], the authors have derived a semi-analytical expression of the bit error probability using a non-parametric estimation of the probability density of the observed samples. It has been shown that the accuracy of the pdf estimator is sensitive to the choice of the smoothing parameter. The method we have proposed in [5] considers the estimation of the pdf using kernel estimator [6] which uses an efficient technique for selecting the smoothing parameter. In [7], we have compared some methods to make up for the optimum smoothing parameter choice. The first is the minimum integrated squared error (MISE) [8], which exhibited a significant squared error between the true pdf and the estimated one. In the second method, the smoothing parameter is estimated using a cross-validation (CV) method [9, 10]. Simulation studies have concluded that the method called cross validation outperforms the other method in terms of squared error. Nevertheless, this technique can lead to inconsistent estimator and requires too much computing time.

In this paper, we propose a new semi-analytical approach based on Fourier transform inversion to derive a semi-analytical expression of error probability. In this method, the probability density of the decision variable at the matched filter output is estimated from the characteristic function via Fourier transform inversion. This is due to the fact that the characteristic function is defined as the Fourier transform of the probability density function. In addition, Fourier integrals can be numerically evaluated by the fast Fourier transform (FFT) algorithm. Furthermore, in order to control the behavior of the probability density estimator, we applied a bootstrap method for selecting the optimum smoothing parameter. This leads to an accurate semi-analytical error probability due to the bootstrap approach efficiency.

The remainder of the paper is organized as follows. In Section 2, we describe the system model considered in this work. In Section 3, a new semi-analytical expression of the error probability is derived, using Fourier inversion approach. Some methods for selecting the smoothing parameter are given in Section 4. Simulations and numerical results are given in Section 5. Then, concluding remarks are made in Section 6.

## 2 System model

**b**=[

*b*

_{1},

*b*

_{2},…,

*b*

_{ L }] and each has length

*L*. The sequences of bits are then passed to a digital modulation scheme which converts them into sequences of symbols, each has length

*M*and whose elements take values in constellation set

*Ω*. The digital modulation can perform binary phase-shift keying (BPSK), quadrature phase-shift keying (QPSK), or high order modulation such as 16-quadrature amplitude modulation (QAM) and 64-QAM. Other techniques, such as single carrier frequency division multiple access (SC-FDMA) or orthogonal frequency division multiplexing (OFDMA), can be included in the transmitter to improve the system reliability. After bit-to-symbol mapping, the modulation transforms the symbol stream into an analog signal suitable to be sent through the transmission channel which can degrade the signal quality.

At the receiver, the channel output is passed to a matched filter to reduce the noise effect. After that, the demodulation is performed for symbol-to-bit conversion. Finally, the receiver makes a decision to detect the information bits.

## 3 Semi-analytical error probability derivation

### 3.1 Bit error probability definition

*N*samples

*C*={

*x*

_{1},

*x*

_{2},…,

*x*

_{ N }} at the output of the matched filter and makes a decision to estimate the information bits. Due to the channel effect, this decision can be erroneous. So, it is important to measure the communication system efficiency in terms of bit error probability (BEP). According to the system model presented in Fig. 1, this bit error probability is defined to be the conditional probability that the receiver makes a wrong decision on a transmitted information bit. Assuming that the

*i*th bit is transmitted, the error probability is expressed as follows:

*X*be the random variable whose realizations are the observed samples at the matched filter output and define the decision region associated to the information bit

*b*

_{ i }as

*i*th information bit at the receiver end. The probability of error on the bit

*b*

_{ i }defined in (1) is then re-expressed as

*X*to get

*P*

_{ e }, we divide the set of the observed samples

*C*into two subsets

*C*

_{0}and

*C*

_{1}. The first subset contains

*N*

_{0}observed samples which corresponds to the transmission of

*b*

_{ i }=0. The second subset consist of

*N*

_{1}observed samples when the bit

*b*

_{ i }=1 is transmitted. In this manner, the probability density function of

*X*can be viewed as a mixture of two probability densities \(f_{X}^{(1)}(x)\) and \(f_{X}^{(0)}(x)\) of the observed samples corresponding to the transmitted information bits

*b*

_{ i }=1 and

*b*

_{ i }=0, respectively. Then, the average bit error probability is written as

where \(P_{k} = \frac {N_{k}}{N}, k=0,1\), is the probability that *b*
_{
i
}=*k* is transmitted.

Accordingly, for predicting the error probability *P*
_{
e
}, one has to estimate the probability densities \({f_{X}^{1}}(x)\) and \({f_{X}^{0}}(x)\). In this paper, we will focus on the use of Fourier inversion approach and its use for estimating error probability.

### 3.2 Probability density function estimation

*X*via Fourier transform inversion. This is expressed as follows:

*φ*

_{ X }is the characteristic function of a random variable

*X*, defined as

*N*observed samples {

*x*

_{1},

*x*

_{2},…,

*x*

_{ N }}, the expectation in (8) can be approximated by a finite sum. Hence, the characteristic function

*φ*

_{ X }can be written as

Consequently, the probability density function can be estimated according to (7) by using the approximation of *φ*
_{
X
}(*t*) given in (9). However, the Fourier integral in (7) can exhibit divergence for large values of the time variable *t*. To solve this limitation, the characteristic function estimator \(\widetilde {\varphi }_{X}(t)\) is multiplied by a damping function *ψ*
_{
h
}(*t*)=*ψ*(*h*
*t*) to control the smoothness of the estimated probability density function.

where *h* is a smoothing parameter.

*ψ*(

*t*) is the Gaussian function \(\psi (t)= e^{-\pi t^{2}}\). Then, the semi-analytical probability density function is done as

*x*

_{ i })

_{0}and (

*x*

_{ i })

_{1}are the observed samples corresponding to the transmitted bits

*b*

_{ i }=0 and

*b*

_{ i }=1, respectively.

*h*

_{1}(respectively,

*h*

_{0}) is the smoothing parameter which depends on the number of observed samples, i.e.,

*N*

_{1}(respectively,

*N*

_{0}).

*Q*(:) denotes the complementary unit cumulative Gaussian distribution, that is

From (14), it is clear that the accuracy of bit error probability estimation depends on the choice of the optimal smoothing parameter.

## 4 Smoothing parameter selection

*h*that minimizes the error between the estimated pdf and the true pdf. The most common metric to represent this error is the mean integrated squared error (MISE) which is expressed as [12]

*h*:

where \(R(g)= \int _{}^{} g^{2}(u) \, du\), \(\mu _{k}(g)= \int u^{k} g(u) \, du\), and *K*(.) represents the kernel function. Until now, it is difficult to measure *h*
_{opt} since it depends on the unknown quantity *R*(*f*
^{′′}). To solve this problem, several types of MISE-based methods have been suggested in literature. Hereafter, we detail the most popular ones.

### 4.1 Rule-of-thumb method

*f*, in (18) by a standard normal distribution that has mean

*μ*and variance

*σ*

^{2}, i.e., \(\mathcal {N}(\mu,\sigma ^{2})\). In this manner, we get

### 4.2 Cross-validation method

*R*(

*f*

^{′′}) in

*h*

_{opt}formula. Furthermore, CV approach considers the integrated squared error (ISE) to select the optimal smoothing parameter. This error metric is expressed as [14]

*h*is called least squares cross-validation (LSCV)-based method [15] expressed as

where, \(\widetilde {f}_{-i}(x_{i})\), *i*=1,…,*N*, is the estimated density using all the original observations except for *x*
_{
i
}.

*h*) is an unbiased estimator of \(\text {MISE}(h)-\int _{-\infty }^{+ \infty } f^{2}(y)\,dy\). This is expressed as :

*h*:

*R*(

*f*

^{′′}) by the estimator:

where \(\widetilde {f}_{X}^{\prime \prime }\) is the second derivative of the kernel density estimate and \(K_{h}(x)=\frac {1}{h} K\left (\frac {x}{h}\right)\). The operator ∗ indicates the convolution product.

### 4.3 Bootstrap method

Bootstrap procedures for selecting the smoothing parameter have been studied in previous work [20–22]. The idea is to estimate the MISE using the bootstrap and then minimize it with respect to *h*. Let \(\widetilde {f}_{X}(x;g)\) be the estimate of *f*(*x*) obtained from {*x*
_{1},…,*x*
_{
N
}}, with a pilot smoothing parameter *g*.

*h*→0 as

*N*→

*∞*, leads to an asymptotic approximation to MISE

^{∗}[24] as:

^{∗}(

*h*) as

*h*

_{opt,boot}is obtained by minimizing MISE

^{∗}(

*h*) with respect to

*h*:

*h*

_{opt,boot}obtained from (35) is given as (see proof in Appendix D):

*h*

_{opt,boot}value depends on the second derivative of the estimate pdf \(\int _{}^{}(\,\widetilde {f}_{X}^{\prime \prime }(x;g))^{2}\,dx\) where the pilot smoothing parameter

*g*is selected using least squares the cross-validation method [25]. This parameter is chosen so as to minimize

*f*

_{ N,−i }(

*x*

_{ i },

*g*) is the density estimate based on all of data expect

*x*

_{ i }. To justify the choice of the bootstrap method for selecting the optimal smoothing parameter, we have presented the integrated squared error as a function of the smoothing parameter

*h*. Figure 2 shows the obtained results with bootstrap, cross-validation, and rule-of-thumb methods. It is seen that the bootstrap method outperforms the other methods in terms of the integrated squared error between the true probability density and the estimated density.

## 5 Simulations and results

In order to verify the obtained semi-analytical expression of error probability, computer simulations were done using the system model presented in Fig. 1. We first validated the probability density estimation using Fourier inversion. We, then, used it to predict the semi-analytical bit error probability of several transmission scenarios. This probability is compared with the BER evaluated using Monte Carlo simulation which considers a 95 *%* confidence interval for all scenarios.

*N*=10,000 samples and estimates the probability density using Fourier inversion method. The obtained probability density is compared to the theoretical density as shown in Fig. 3. It is seen that the density curve corresponding to Fourier inversion method is close to theoretical density curve. Moreover, we have evaluated the semi-analytical bit error probability (BEP) using the expression given in (14) in terms of signal to noise ratio (SNR). The simulation results obtained from the semi-analytical method are compared with those from Monte Carlo simulation, as well as from the analytical method. Besides, the analytical BEP is expressed as

where \(erfc(x)=\frac {2}{\sqrt {\pi }} \int _{x}^{+\infty } {e}^{{-x}^{2}}\,dx\).

^{−4}, Monte Carlo simulation requires a number of 1,000,000 samples while Fourier inversion uses only 10,000 observed samples.

Computing time comparison. This table summarizes an experiment comparing the time (in seconds) to obtain bit error probability

Computing time (s) | |||
---|---|---|---|

BEP | Proposed SPP | MC simulation | |

BPSK | 10 | 2.106 | 154.179 |

10 | 1.760 | 14.001 | |

10 | 1.013 | 1.441 | |

QPSK | 10 | 2.554 | 86.933 |

10 | 1.734 | 9.013 | |

10 | 1.025 | 2.752 | |

4-PAM | 10 | 5.877 | 71.864 |

10 | 5.309 | 8.1315 | |

10 | 3.663 | 4.333 | |

SC-FDMA | 10 | 2.631 | 54.810 |

10 | 2.048 | 8.131 | |

10 | 1.671 | 1.453 |

^{−3}, Monte Carlo simulation requires a number of samples equal to 100,000 while the proposed method uses only

*N*=10,000 samples. In addition, the same performance in terms of bit error probability has been obtained when quadrature phase-shift keying (QPSK) modulation is considered. The simulation results are presented in Fig. 7.

*N*=20,000 observed samples. As for all scenarios, Fourier inversion curves are very close to Monte Carlo simulation curves and analytical method curves where its analytical expression is done by

where \(p=\frac {1}{2} - \frac {1}{2}.\left (1+\frac {1}{\text {SNR}}\right)^{-\frac {1}{2}} \). Also, it is observed that a reduced computing time is obtained. This presents a major strength of the proposed approach and very promising for many practical systems.

## 6 Conclusions

In this paper, we have considered a new semi-analytical method for estimating the error probability for any digital communication system. We have shown that the problem of error probability estimation is equivalent to estimate the conditional probability density function (pdf) of the observed soft samples at the receiver output. The proposed method is based on Fourier inversion approach for predicting the pdf. It has been shown that the accuracy of this approach is very sensitive to the optimum smoothing parameter selection. Furthermore, we have applied the bootstrap method for selecting the optimal smoothing parameter which makes the proposed semi-analytical method more accurate. The simulation results have concluded that with either the Monte Carlo (MC) simulation technique or the new proposed semi-analytical approach, we have the same performance. Moreover, the use of the bootstrap method can decrease the squared error between the true pdf and the estimated one.

## 7 Appendix A Proof of (11)

## 8 Appendix B Proof of (14)

*x*

_{ i })

_{1}and (

*x*

_{ i })

_{0}, respectively, which corresponds to transmitted information bits

*b*

_{ i }=1 and

*b*

_{ i }=0, respectively. By using the obtained semi-analytical probability density function in (13), we can define

*h*

_{1}(respectively,

*h*

_{0}) is the smoothing parameter which depends on the number of observed samples, i.e.,

*N*

_{1}(respectively,

*N*

_{0}). By substituting the estimated pdf \(\widetilde {f}_{X}^{(1)}\) and \(\widetilde {f}_{X}^{(0)}\) in (B.1), we get

## 9 Appendix C Proof of (18)

*K*(.) as any function satisfies \(\int _{}^{} K(x) \, dx =1 \) and:

**Estimation bias:**Let us consider that the expectation of kernel function can be written as integrals of the convolution of the kernel density and the true density function:

*f*(

*x*+

*h*

*u*) in the argument

*hu*and with

*h*→0. For a

*ν*

^{′}th-order kernel, we take the expansion out to the

*ν*

^{′}th-term to solve this integral:

where \(\mu _{\nu }(K)=\int _{}^{} u^{\nu } K(u)\,du\).

**Estimation variance:**Let us compute the variance of \(\widetilde {f}(x;h)\) for a density estimator:

**Mean-squared error**As defined, the mean squared error (MSE) is done as

*f*, we have

where \(R(\,f^{\prime \prime })=\int _{}^{} {f^{\prime \prime }(u)}^{2}\,du\)

*h*. The first derivative is given by

## 10 Appendix D Proof of (36)

^{∗}, and Var

^{∗}all involve expectations conditionally upon \(x^{*}_{1}, x^{*}_{2},\ldots, x^{*}_{N}\) and all

*x*

^{∗}are sampled from the smoothed distribution \(\widetilde {f}_{X}(x;h)\). Making a substitution followed by a Taylor series expansion, this assumes that

*h*→0 as

*N*→

*∞*, gives an asymptotic approximation:

*h*. The first derivative is given by

## Declarations

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## Authors’ Affiliations

## References

- A Bohdanowicz, in Proc. SCVT 2001, Delft. On efficient BER evaluation of digital communication systems via importance sampling (The Netherlands, 2001), pp. 61–67.Google Scholar
- JG Proakis, Digital Communications, 4th Edition (McGraw-Hill, New York, 2001).Google Scholar
- A Abedi, AK Khandani, A new method for performance evaluation of bit decoding algorithms using statistics of the log likelihood ratio. J. Franklin Inst. 345, 60–74 (2008). doi:http://dx.doi.org/10.1016/j.jfranklin.2007.06.002.MATHView ArticleGoogle Scholar
- S Saoudi, M Troudi, F Ghorbel,
*An Iterative Soft Bit Error Rate Estimation of Any Digital Communication Systems Using a Nonparametric Probability Density Function*(EURASIP journal on wireless communications and networking, Hindawi Publishing Corporation, 2009). doi:http://dx.doi.org/10.1155/2009/512192. - FE Naamane, M Et-tolba, M Belkasmi, in Proc ICWMC 2013. A semi-analytical performance prediction of turbo coded SC-FDMA, (2013), pp. 146–151.Google Scholar
- W Zucchini,
*Applied smoothing technique, part1: Kernel density estimation*, (Temple University, Philadephia, PA, 2003). http://isc.temple.edu/economics/Econ616/Kernel/astpart1.pdf. - FE Naamane, M Et-tolba, M Belkasmi, in
*Proc IEEE PIMRC 2013*. Performance prediction of a coded digital communication system using cross-validation, (2013), pp. 621–625. doi:http://dx.doi.org/10.1109/PIMRC.2013.6666211. - SJ Sheather, MC Jones, A reliable data-based bandwidth selection method for kernel density estimation. J. R. Stat. Soc. 53(3), 683–690 (1991). Series B.MATHMathSciNetGoogle Scholar
- SR Sain, KA Baggerly, DW Scott, Cross validation of multivariate densities. J. Am. Stat. Assoc. 89, 807–817 (1992).MathSciNetView ArticleGoogle Scholar
- OY Savchuk, JD Hart, SJ Sheather,
*An empirical study of indirect cross-validation. Nonparametric statistics and mixture models: a Festschrift in honor of Thomas P. Hettmansperger*(World Scientific Publishing, Singapore, 2010).Google Scholar - Z Zambom, R Dias,
*A review of kernel density estimation with applications to econometrics, International Econometric Review (IER), vol. 5, no. 1*, (2013). Available: http://EconPapers.repec.org/RePEc:erh:journl:v:5:y:2013:i:1:p:20-42. - I Horova, J Zelinka,
*Contribution to the bandwidth choice for kernel density estimates Computational statistics*(Springer-Verlag, 2007). doi:http://dx.doi.org/10.1007/s00180-007-0020-9. - S Saoudi, T Ait-Idir, Y Mochida, A novel non-parametric iterative soft bit error rate estimation technique for digital communications systems. Proc. IEEE Int. Conf. Commun. (ICC 11), 1–6 (2011). doi:http://dx.doi.org/10.1109/icc.2011.5.962691.
- D Agnew, C Constable,
*Chapter 9: Non parametric density function estimation, preliminary course outline for SIO 223A*(Fall, 2008). http://igppweb.ucsd.edu/~cathy/Classes/SIO223A/index.html. - E Mammen, M Miranda, JP Nielsen, S Sperlich,
*A comparative study of new cross-validated bandwidth selectors for kernel density estimation*, (2012). http://arxiv.org/abs/1209.4495. - A Bowman, An alternative method of cross-validation for the smoothing of density estimates. Biometrika. 71, 353–360 (1984).MathSciNetView ArticleGoogle Scholar
- OY Savchuk, JD Hart, SJ Sheather, Indirect cross-validation for density estimation. J. Am. Stat. Assoc. 105(489), 415–423 (2010).MathSciNetView ArticleGoogle Scholar
- SJ Sheather, Density estimation. Stat. Sci. 2004. 19(4), 588–597 (2004).MATHMathSciNetGoogle Scholar
- DW Scott, GR Terrell, Biased and unbiased cross-validation in density estimation. J. Am. Stat. Assoc. 82, 1131–1146 (1987).MATHMathSciNetView ArticleGoogle Scholar
- J Faraway, M Jhun, Bootstrap choice of bandwidth for density estimation. J. Am. Stat. Assoc. 85, 1119–1122 (1990).MathSciNetView ArticleGoogle Scholar
- B Grund, J Polzehl, Bias corrected bootstrap bandwidth selection. J. Nonpara- metric Stat. 8, 97–126 (1997).MATHMathSciNetView ArticleGoogle Scholar
- P Hall, Using the bootstrap to estimate mean squared error and select smoothing parameter in nonparametric problems. J. Multivariate Anal. 32, 177–203 (1990).MATHMathSciNetView ArticleGoogle Scholar
- JS Marron,
*Bootstrap bandwidth selection, exploring the limits of bootstrap*. (R LePage, L Billard, eds.) (Wiley, New York, 1991).Google Scholar - C Taylor, Bootstrap choice of the tuning parameter in kernel density estimation. Biometrika. 76, 705–712 (1989).MATHMathSciNetView ArticleGoogle Scholar
- W Hardle, JS Marron, MP Wand, Bandwidth choice for density derivatives. J. R. Stat. Soc. 52, 223–232 (1990).MathSciNetGoogle Scholar
- HG Myung, DJ Goodman, Single carrier FDMA a new air interface for long term evolution (John Wiley & Sons Ltd, 2008).Google Scholar