# Symbol Error-Rate Analysis of OSTB Codes and Linear Precoder Design for MIMO Correlated Keyhole Channels

- Pradeepa Yahampath
^{1}Email author and - Are Hjørungnes
^{2}

**2008**:571827

**DOI: **10.1155/2008/571827

© P. Yahampath and A. Hjørungnes. 2008

**Received: **19 July 2007

**Accepted: **17 March 2008

**Published: **6 April 2008

## Abstract

Analytical expressions for the symbol error rate (SER) of MIMO systems are important from both design and analysis point of views. This paper derives exact and easy to evaluate analytical expression for the SER of an orthogonal space-time block coded (OSTBC) MIMO system with spatially correlated antennas, in which the channel signal propagation suffers from a degenerative effect known as a keyhole. These expressions are valid for complex valued correlations between antenna elements. Numerical results obtained by simulations are presented to confirm the validity of the analytical SER expressions for several multilevel modulation schemes. Subsequently, a procedure for designing a linear precoder which exploits the knowledge of the channel correlation matrix at the transmitter to enhance the performance of the above system is given. To this end, the exact SER expression is minimized by using a gradient descent algorithm. To demonstrate the performance improvements achievable with the proposed precoder, numerical results obtained in several design examples are presented and compared.

## 1. Introduction

In previous work, the capacity of MIMO keyhole channels is studied in [3, 7–9]. In [10], expressions for the exact symbol error rate (SER) of orthogonal space-time block codes (OSTBC) over a double Rayleigh fading keyhole channel are derived for -PSK and -QAM modulation schemes. In a subsequent work [7], similar analysis is also carried out for keyhole channels with Nakagami- fading. In another related work, [11] derives expressions for the pairwise error probability (PEP) of MIMO systems with keyhole channels. However, in [7, 10, 11], the channel is assumed to be spatially uncorrelated. Similar analysis for MIMO keyhole channels with spatial correlations has not been reported so far. It is well known that if the channel correlation matrix is known at the transmitter (e.g., as channel side information (CSI)), a linear precoder matched to the channel correlation structure can be used at the MIMO channel input to improve the system performance (see, e.g., [12–16]). However, the aforementioned work assumes Rayleigh or Ricean fading and is not applicable to double Rayleigh fading as in the case of channels with keyholes. Formally, a keyhole MIMO channel can be modeled by a matrix which is the outer product of two complex Gaussian vectors [4]. This implies that each coefficient in the channel matrix is a product of two independent complex Gaussian variables (and hence the name double Rayleigh fading). Since keyhole effects lead to the degeneration of the channel matrix [3, 4], it is particularly important to exploit CSI available at the transmitter using a properly designed precoder to enhance the MIMO system performance.

In contrast to previous work, we consider in this paper the SER analysis and the design of linear precoders which minimize the average SER of double Rayleigh fading MIMO channels with spatial correlations. More specifically, we derive easy, to evaluate, exact analytical expressions for the average SER of systems with -PSK, -PAM, and -QAM signaling and maximum likelihood (ML) detection. These closed-form SER expressions are useful in performance analysis of such systems and can be easily evaluated with computing software, using readily available functions. Numerical results are presented in this paper to confirm that our analytical expressions agree with those obtained by simulation of the OSTBC-MIMO system. Based on the analytical expressions for SER, we then present a design of linear precoders which minimize the average SER by exploiting the knowledge of the channel correlation matrix (available at the transmitter either as prior knowledge or via a feedback channel from the receiver), under a constraint on the total average transmitted power. In our formulation, the antenna correlations are allowed to be complex valued [17]. In the general case, we consider it is difficult to find a closed-form solution for the minimum SER (MSER) precoder. As such, the MSER precoder matrix is found by a constrained gradient descent minimization method, based on the derivatives of the average SER. Experimental results are presented to demonstrate the performance improvements achieved by the proposed MSER precoder. In these experiments, MIMO keyhole channels with both real and complex correlation matrices have been considered.

The expressions and the optimal precoder design derived in this paper are novel in the following aspects. Different to [10], the SER expression derived in this paper applies to MIMO *keyhole* channels with *spatial correlations* at both transmit and receive antennas. While the SER expressions in [10], which apply only to uncorrelated channels, do not generalize to our case, we show that in the absence of spatial correlations the SER expressions we derive reduce to those in [10]. Note that, for the uncorrelated channel considered in [10], the precoding problem we consider is not relevant. Also, different to [14, 15], the SER expressions and the optimal precoders derived in this paper account for the *keyhole effect*, that is, each channel coefficient is a *product* of two complex Gaussian random variables. While there is no direct relationship between these expressions, they correspond to two extreme cases of a *double scattering channel* with
scatterers [4]. In particular, when
, the channel exhibits the keyhole effect considered in this paper. When
, the MIMO channel is the single-scattering case considered in [14, 15]. It should be noted that neither the results in this paper nor those in [14, 15] readily generalize to the case
.

The rest of this paper is organized as follows. Section 2 describes the keyhole channel model and the OSTBC-MIMO system model under consideration. Section 3 derives the SER expressions for this system, where -PSK, -PAM, and -QAM modulation schemes are considered. Section 4 then presents an algorithm for designing an MSER linear precoder for the system. In Section 5, numerical results are presented to demonstrate the accuracy of the given SER expressions and the performance gains achievable with the proposed precoder designs. Finally, concluding remarks are given in Section 6. Some proofs are provided in the appendices.

Notation

is matrix *stack operator*,
is matrix transpose,
is *Hermitian* operation,
is complex conjugate,
denotes the trace of a matrix,
is *Frobenius norm,* (
is vector norm),
denotes the Kronecker product,
denotes
identity matrix, and
defines new symbols.

## 2. Channel Model and System Description

Channel Model

The gain matrix of a double Rayleigh fading keyhole MIMO channel with transmit antennas and receive antennas can be represented by the outer product of two independent complex Gaussian vectors (size and (size representing transmit and receive Rayleigh fading, respectively, [4], that is, . However, each vector is assumed to be correlated so that and , where is the transmit antenna correlation matrix, is the receive antenna correlation matrix, and and are independent identically distributed (iid) complex Gaussian vectors of sizes and respectively, (here, and are the unique square roots of Hermitian positive semidefinite matrices and , resp., [18]). Note that each element of is a product of two independent Gaussian variables. Note also that the rank of is necessarily one.

System Model

where is the average power of modulation signal set, and is a constant that depends on the particular OSTBC in use. For example, for [21], and for in [19].

The codeword is transmitted through the MIMO keyhole channel whose output is a matrix , where is the complex channel noise matrix. The elements of are independent complex Gaussian variables with iid real and imaginary parts of variance . The receiver decodes channel output using a ML detector. It is assumed that the channel matrix remains constant for the duration of a codeword (quasi-static channel), and that the receiver has the knowledge of and to be used in ML decoding. While is not known to the transmitter, the antenna correlation matrices and are assumed to be known.

## 3. MGF of Output SNR and SER Expressions

*signal to noise ratio*(SNR). When the space-time block code is orthogonal, the MIMO system is equivalent to a SISO system, in which the SNR at the input of the ML detector is given by [10]:

This integral is readily available as a built-in function in software packages such as Matlab and Mathematica. Given this result, the MGF of the output SNR, can be obtained as .

Note

where
is the *confluent hypergeometric function* [26, (9.211-4)],
is the gamma function, and
,
are partial fraction coefficients (procedures for finding which are well known). This expression can now be used to generalize the SER expressions (21)–(23). It can be shown that when
for all
(
) and
for all
(
), (24) reduces to (16). Other interesting cases are when either transmit antennas or receive antennas are uncorrelated, in which case either
or
is the identity matrix. Also, when both
and
are identity matrices (no channel correlation), then it is easy to show that (24) reduces to [10, (6)].

## 4. Precoder Optimization

This equation yields two positive values for for which the updated precoder in (26) satisfies the power constraint, provided that the step size is chosen sufficiently small (see Appendix B). The choice of the smaller value for out of the two possible solutions results in the proper convergence of the gradient descent algorithm (see Figure 8). In each iteration , we first find by solving (28) and then using (26). The convergence of the solution can be decided when becomes less than a prescribed threshold. In the following, we derive closed-form expressions for for different modulation schemes.

## 5. Numerical Results and Discussion

*Alamouti code*) for which . In this case, , , that is, rate is one. Figure 3 shows the results for a MIMO system based on the OSTBC from [20] for which and , , , that is, rate is 3/4. Note that no precoding has been used in these cases ( set to a scaled identity matrix). Since spatial correlation in the channel degrades the SER performance, it is also of interest to compare the SER of an OSTBC over a correlated keyhole channel with that over a keyhole channel with no spatial correlation. Such a comparison for the MIMO system is shown in Figure 4 which shows that the degradation of performance due to spatial correlation in the given keyhole channel can be quite significant at high SNR (exponential antenna correlation model is used). Also included here is the SER over an6 Rayleigh fading channel (no keyhole effect and no antenna correlation). In particular, these curves clearly highlight the loss of diversity (as indicated by the slope of the curves [1]) in the MIMO system due to the combined effect of the keyhole propagation and the antenna correlation.

*beamforming*in the direction of . Note that if the MSER precoder matrix is known to have equal elements, it can be directly determined by using the power constraint.

## 6. Concluding Remarks

Exact analytical expressions were derived for SER of -PSK, -PAM, and -QAM modulated OSTBCs over a MIMO spatially correlated keyhole channel. A general complex correlation matrix has been assumed in the derivations. These expressions are easy to compute using numerical software and have been verified by Monte Carlo simulations. The given analytical expressions have been used to design MSER linear precoders based on the knowledge of channel correlation matrix available as CSI at the transmitter. Using simulation experiments, it has been demonstrated that the proposed MSER precoder can significantly reduce the error probability of a MIMO system operating on a fading channel degraded by a keyhole effect.

An important extension to this work includes precoder design for more general multiple scattering fading channels [5]. In particular, an interesting case is the double scattering model [4] (of which the keyhole channel is a special case).

## Appendices

### 7. Proof of (24)

*confluent hypergeometric function*[26, (9.211-4)] :

where , , and is the Gamma function. Specifically, by letting , and , we obtain (24).

### 8. Solution of (28)

Clearly, all coefficients of (28) are real. A simplified graphical interpretation of the solution to this equation is shown in Figure 8. Note that is the updated solution for the precoder matrix, without considering the power constraint, that is, when . This solution satisfies the power constraint as an inequality if . In this case, the rescaling of to satisfy the power constraint will result in a solution with a lower SER. On the other hand, if , we have to choose so that satisfies the power constraint. Thus, we consider this case.

Otherwise the updated solution cannot satisfy the power constraint. This scenario is also evident from Figure 8. Thus, the step-size for gradient descent must be chosen sufficiently small, in order to ensure the convergence (however, an overly small step-size results in slow speed of convergence). In Figure 8, and are the two candidate solutions for the updated precoder , corresponding to the roots of the quadratic (28). The desired solution is (corresponding to the smaller ), which ensures the convergence of the gradient descent algorithm. This can be seen form the fact that , which corresponds to the larger , will be opposite to closer to the convergence point, where .

### 9. Proof of (34)

*differential*of can be expressed as:

*commutation matrix*of size [30]. For matrix , the

*commutation matrix*satisfies . Let . Then, using [28, (4)] , we obtain

where we use the fact that [30, Theorem 3.1]. From this, the result in (34) follows.

### 10. Proof of (38)

Under this condition, MSER precoder maximizes subject to , where . Now, from [31, Lemma 2.2], it directly follows that the MSER precoder is a scaled identity matrix and hence the result in (38).

### 11. Proof of (39)

and hence , that is, the objective function (25) is unchanged by a unitary transformation of .

Next, consider the singular value decomposition (SVD) of , where and are unitary matrices and is a diagonal matrix of singular values of . If has identical elements, then it has rank one and hence only a single nonzero singular value , that is, . Therefore, it can be seen that has the form , where is a vector and is the matrix of zero elements.

## Declarations

### Acknowledgments

The authors wish to thank the reviewers and the editor for their insightful comments and suggestions which help improve the paper. This work was supported by the Research Council of Norway projects 176773/S10 (OptiMO) and 183311/S10 (M2M), under the VERDIKT program.

## Authors’ Affiliations

## References

- Paulraj A, Nabar R, Gore D:
*Introduction to Space-Time Wireless Communications*. Cambridge University Press, New York, NY, USA; 2003.Google Scholar - Gesbert D, Shafi M, Shiu D-S, Smith PJ, Naguib A: From theory to practice: an overview of MIMO space-time coded wireless systems.
*IEEE Journal on Selected Areas in Communications*2003, 21(3):281-302. 10.1109/JSAC.2003.809458View ArticleGoogle Scholar - Chizhik D, Foschini GJ, Gans MJ, Valenzuela RA: Keyholes, correlations, and capacities of multielement transmit and receive antennas.
*IEEE Transactions on Wireless Communications*2002, 1(2):361-368. 10.1109/7693.994830View ArticleGoogle Scholar - Gesbert D, Bölcskei H, Gore DA, Paulraj AJ: Outdoor MIMO wireless channels: models and performance prediction.
*IEEE Transactions on Communications*2002, 50(12):1926-1934. 10.1109/TCOMM.2002.806555View ArticleGoogle Scholar - Salo J, El-Sallabi HM, Vainikainen P: Statistical analysis of the multiple scattering radio channel.
*IEEE Transactions on Antennas and Propagation*2006, 54(11):3114-3124.View ArticleMathSciNetGoogle Scholar - Almers P, Tufvesson F, Molisch AF: Keyhole effects in MIMO wireless channels: measurements and theory.
*Proceedings of IEEE Global Telecommunications Conference (GLOBECOM '03), December 2003, San Francisco, Calif, USA*4: 1781-1785.Google Scholar - Shin H, Lee JH: Performance analysis of space-time block codes over keyhole Nakagami-m fading channels.
*IEEE Transactions on Vehicular Technology*2004, 53(2):351-362. 10.1109/TVT.2004.823540View ArticleGoogle Scholar - Cui XW, Feng ZM: Lower capacity bound for MIMO correlated fading channels with keyhole.
*IEEE Communications Letters*2004, 8(8):500-502. 10.1109/LCOMM.2004.833785View ArticleGoogle Scholar - Loyka S, Kouki A: On MIMO channel capacity, correlations, and keyholes: analysis of degenerate channels.
*IEEE Transactions on Communications*2002, 50(12):1886-1888. 10.1109/TCOMM.2002.806543View ArticleGoogle Scholar - Shin H, Lee JH: Effect of keyholes on the symbol error rate of space-time block codes.
*IEEE Communications Letters*2003, 7(1):27-29. 10.1109/LCOMM.2002.807428View ArticleGoogle Scholar - Niyomsataya T, Miri A, Nevins M: Pairwise error probability of space-time codes for a keyhole channel.
*IET Communications*2007, 1(1):101-105. 10.1049/iet-com:20060201View ArticleGoogle Scholar - Sampath H, Paulraj A: Linear precoding for space-time coded systems with known fading correlations.
*IEEE Communications Letters*2002, 6(6):239-241. 10.1109/LCOMM.2002.1010867View ArticleGoogle Scholar - Jöngren G, Skoglund M, Ottersten B: Combining beamforming and orthogonal space-time block coding.
*IEEE Transactions on Information Theory*2002, 48(3):611-627. 10.1109/18.985950View ArticleGoogle Scholar - Hjørungnes A, Gesbert D: Precoding of orthogonal space-time block codes in arbitrarily correlated MIMO channels: iterative and closed-form solutions.
*IEEE Transactions on Wireless Communications*2007, 6(3):1072-1082.View ArticleGoogle Scholar - Hjørungnes A, Gesbert D: Precoded orthogonal space-time block codes over correlated ricean MIMO channels.
*IEEE Transactions on Signal Processing*2007, 55(2):779-783.View ArticleMathSciNetGoogle Scholar - Vu M, Paulraj A: Optimal linear precoders for MIMO wireless correlated channels with nonzero mean in space-time coded systems.
*IEEE Transactions on Signal Processing*2006, 54(6):2318-2332.View ArticleGoogle Scholar - Shiu D-S, Foschini GJ, Gans MJ, Kahn JM: Fading correlation and its effect on the capacity of multielement antenna systems.
*IEEE Transactions on Communications*2000, 48(3):502-513. 10.1109/26.837052View ArticleGoogle Scholar - Horn RA, Johnson CR:
*Matrix Analysis*. Cambridge University Press, Cambridge, UK; 1999.Google Scholar - Alamouti SM: A simple transmit diversity technique for wireless communications.
*IEEE Journal on Selected Areas in Communications*1998, 16(8):1451-1458. 10.1109/49.730453View ArticleGoogle Scholar - Tirkkonen O, Hottinen A: Square-matrix embeddable space-time block codes for complex signal constellations.
*IEEE Transactions on Information Theory*2002, 48(2):384-395. 10.1109/18.978740View ArticleMathSciNetMATHGoogle Scholar - Tarokh V, Jafarkhani H, Calderbank AR: Space-time block coding for wireless communications: performance results.
*IEEE Journal on Selected Areas in Communications*1999, 17(3):451-460. 10.1109/49.753730View ArticleGoogle Scholar - Lucent , Nokia , Seimens , Ericsson :
*A standardized set of MIMO radio propagation channels.*3GPP TSG RAN WG1, Cheju, Korea; November 2001.Google Scholar - Simon MK, Alouini M-S:
*Digital Communications over Fading Channels: A Unified Approach to Performance Analysis*. 2nd edition. John Wiley & Sons, New York, NY, USA; 2005.Google Scholar - Turin GL: Characteristic function of Hermitian quadratic forms in complex normal variables.
*Biometrika*1960, 47(1-2):199-201. 10.1093/biomet/47.1-2.199View ArticleMathSciNetMATHGoogle Scholar - Rohatgi VK:
*An Introduction to Probability Theory and Mathematical Statistics*. John Wiley & Sons, New York, NY, USA; 1976.MATHGoogle Scholar - Gradshteyn LS, Ryzhik IM:
*Tables of Integrals, Series, and Products*. 6th edition. Academic Press, Boston, Mass, USA; 2000.Google Scholar - Luenberger DG:
*Linear and Nonlinear Programming*. 2nd edition. Addison-Wesley, Reading, Mass, USA; 1989.Google Scholar - Hjørungnes A, Gesbert D: Complex-valued matrix differentiation: techniques and key results.
*IEEE Transactions on Signal Processing*2007, 55(6):2740-2746.View ArticleMathSciNetGoogle Scholar - Aalo VA: Performance of maximal-ratio diversity systems in a correlated Nakagami-fading environment.
*IEEE Transactions on Communications*1995, 43(8):2360-2369. 10.1109/26.403769View ArticleGoogle Scholar - Magnus JR, Neudecker H: The commutation matrix: some properties and applications.
*The Annals of Statistics*1979, 7(2):381-394. 10.1214/aos/1176344621View ArticleMathSciNetMATHGoogle Scholar - Popescu O, Rose C, Popescu DC: Maximizing the determinant for a special class of block-partitioned matrices.
*Mathematical Problems in Engineering*2004, 2004(1):49-61. 10.1155/S1024123X04307027View ArticleMathSciNetMATHGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.