MMSE-based joint source and relay optimization for interference MIMO relay systems

Abstract

In this paper, we investigate the transceiver design for amplify-and-forward (AF) interference multiple-input multiple-output (MIMO) relay communication systems when the direct links between the source and destination nodes are taken into consideration. The minimum mean-squared error (MMSE) of the signal waveform estimation at the destination nodes is chosen as the design criterion to optimize the source, relay, and receiver matrices for interference suppression. As the joint source, relay, and receiver optimization problem is nonconvex with matrix variables, a globally optimal solution is computationally intractable to obtain. We propose two iterative algorithms to provide computationally efficient solutions to the original problem through solving convex subproblems. These two algorithms provide efficient performance-complexity trade-off. Simulation results demonstrate that the proposed algorithms converge quickly after a few iterations and significantly outperform existing scheme in terms of the system bit error rate.

1 Introduction

Relay-aided multiple-input multiple-output (MIMO) communication technology has attracted great research interest recently [1,2]. By incorporating relay nodes in a MIMO system, the network coverage and reliability can be significantly improved. In a MIMO relay system, communication between source nodes and destination nodes can be assisted by single or multiple relays equipped with multiple antennas. The relays can either decode-and-forward (DF) or amplify-and-forward (AF) the relayed signals [3]. In the AF scheme, the received signals are simply amplified (including a possible linear transformation) through the relay precoding matrices before being forwarded to the destination nodes. Therefore, in general, the AF strategy has lower complexity and shorter processing delay than the DF strategy.

For single-user two-hop MIMO communication systems with a single relay node, the optimal source and relay precoding matrices have been developed in [4]. For a single-user two-hop MIMO relay system with multiple parallel relay nodes, the design of relay precoding matrices has been studied in [5]. Recent progress on the optimization of AF MIMO relay systems has been summarized in the tutorial of [2].

For MIMO interference channel, the idea of interference alignment (IA) [6] was developed for interference suppression by arranging the desired signal and interference into appropriated signal spaces. The idea of IA has been applied in interference MIMO relay systems in [7,8]. However, there is still no general solution for IA as a number of conditions must be met. One main reason is that the number of dimensions required for IA is very large and it depends on the number of independent fading channels. This leads to high computational complexity and infeasibility in practical systems. In [9], an iterative algorithm has been proposed to optimize the source beamforming vector and the relay precoding matrices to minimize the total source and relay transmit power such that a minimum signal-to-interference-plus-noise ratio (SINR) threshold is maintained at each receiver. Three iterative transceiver design algorithms to minimize either the matrix-weighted sum mean-squared error (SMSE) or the total leakage have been developed in [10]. However, the works in [7,10] did not take the direct source-destination links into consideration.

The direct links between the source and destination nodes provide valuable spatial diversity to the MIMO relay system and should not be ignored. In this paper, we investigate the transceiver design for AF interference MIMO relay communication systems where multiple source nodes transmit information simultaneously to the destination nodes with the aid of multiple relay nodes, and each node is equipped with multiple antennas. The direct source-destination links are taken into account for the design of the transceivers. We aim at optimizing the source, relay, and receiver matrices to suppress the interference and minimize the SMSE of the signal waveform estimation at the destination nodes, subjecting to transmission power constraints at the source and relay nodes. The SMSE criterion is chosen as it provides a good trade-off between performance and complexity. Since the joint source, relay, and receiver optimization problem is nonconvex with matrix variables, a globally optimal solution is computationally intractable to obtain. We propose two iterative algorithms to provide computationally efficient solutions to the original problem through solving convex subproblems. In each iteration of the first algorithm, we first optimize all receiver matrices based on the source and relay matrices from the previous iteration. Then, we optimize all relay matrices using the receiver matrices in this iteration and the source matrices from the previous iteration. Finally, the source matrices are updated.

In the second algorithm, the receiver matrices are optimized in the same way as the first algorithm. However, in contrast to the first algorithm, each source and relay matrix is optimized individually by fixing all other matrices. We show that both proposed algorithms converge. Comparing the two proposed algorithms, the first algorithm has a better MSE and bit error rate (BER) performance, while the second algorithm has a smaller per-iteration computational complexity. Such performance-complexity trade-off is very useful for practical MIMO relay communication systems. Simulation results demonstrate that the proposed algorithms outperform the existing technique in terms of the system MSE and BER.

We assume that similar to [10], the two proposed algorithms are carried out at a central controlling unit, which can be any node in the system. The controlling unit has knowledge of the global channel state information (CSI). After the convergence of the algorithms, the controlling unit sends the information on the optimal source, relay, and receiver matrices to corresponding nodes.

The rest of this paper is organized as follows. The system model and problem formulation are introduced in Section 2. Two iterative joint source, relay, and receiver matrices design algorithms are developed in Section 3. Simulation results are presented in Section 4 to demonstrate the performance of the proposed algorithms. Conclusions are drawn in Section 5.

Throughout this paper, scalars are denoted with lower- or uppercase normal letters, and vectors and matrices are denoted with bold-faced lower- and uppercase letters, respectively. Superscripts (·)^T,(·)^H, and (·)⁻¹ denote the matrix transpose, conjugate transpose, and inverse, respectively; t r() stands for the trace of a matrix; vec () stacks columns of a matrix on top of each other into a single vector; b d() denotes a block-diagonal matrix; ⊗ represents the Kronecker product, E[ ] denotes the statistical expectation; and I _n stands for the n×n identity matrix.

2 System model and problem formulation

We consider a two-hop interference MIMO relay communication system where K source-destination pairs communicate simultaneously with the aid of a network of L-distributed relay nodes as shown in Figure 1. The kth source node and the kth destination node are equipped with N _sk and N _dk antennas, respectively, k=1,⋯,K, and the number of antennas at the lth relay node is N _rl , l=1,⋯,L.

Figure 1

Block diagram of an interference MIMO relay system.

Using half duplex relay nodes, the communication between source and destination pairs is completed in two time slots. At the first time slot, the kth source node transmits an N _sk ×1 signal vector

$$ \textbf{x}_{sk}=\textbf{B}_{k}\textbf{s}_{k},\quad k=1,\cdots,K $$

((1))

to the relay nodes and the destination nodes, where s _k is the d×1 information-carrying symbol vector and B _k is the N _sk ×d source precoding matrix. The received signal vectors at the lth relay node and the kth destination node are given by

$$\begin{array}{@{}rcl@{}} \textbf{y}_{rl}&=&\sum\limits_{k=1}^{K}\textbf{H}_{lk}\textbf{x}_{sk}+\textbf{v}_{rl},\quad l=1,\cdots,L \end{array} $$

((2))

$$\begin{array}{@{}rcl@{}} \textbf{y}_{d1k}&=&\sum\limits_{m=1}^{K}\textbf{T}_{km}\textbf{x}_{sm}+\textbf{v}_{d1k},\quad k=1,\cdots,K \end{array} $$

((3))

where H _lk is the N _rl ×N _sk MIMO fading channel matrix between the kth source node and the lth relay node, T _km is the N _dk ×N _sm MIMO fading channel matrix between the mth source node and the kth destination node, v _rl is the N _rl ×1 additive white Gaussian noise (AWGN) vector at the lth relay node with zero mean and covariance matrix $E\left [\textbf {v}_{\textit {rl}}\textbf {v}_{\textit {rl}}^{H}\right ]=\sigma _{\textit {rl}}^{2}\textbf {I}_{N_{\textit {rl}}}$, l=1,⋯,L, and v _d1k is the N _dk ×1 AWGN vector at the kth destination node at the first time slot with zero mean and covariance matrix $E\left [\textbf {v}_{d1k}\textbf {v}_{d1k}^{H}\right ]=\sigma _{\textit {dk}}^{2}\textbf {I}_{N_{\textit {dk}}}$, k=1,⋯,K.

During the second time slot, the received signal vector at the lth relay node is amplified with the N _rl ×N _rl precoding matrix F _l as

$$ \textbf{x}_{rl}=\textbf{F}_{l} \textbf{y}_{rl},\quad l=1,\cdots,L. $$

((4))

The precoded signal vector x _rl is forwarded to the destination nodes. The received signal vector at the kth destination node is given by

$$ \textbf{y}_{d2k}= \sum\limits_{l=1}^{L}\textbf{G}_{kl}\textbf{x}_{rl}+\textbf{v}_{d2k},\quad k=1,\cdots,K $$

((5))

where G _kl is the N _dk ×N _rl MIMO channel matrix between the lth relay node and the kth destination node and v _d2k is the N _dk ×1 AWGN vector at the kth destination node at the second time slot with zero mean and covariance matrix $E\left [\textbf {v}_{d2k}\textbf {v}_{d2k}^{H}\right ]=\sigma _{\textit {dk}}^{2}\textbf {I}_{N_{\textit {dk}}}$, k=1,⋯,K.

From Equations 1 to 5, the signal vector received at the kth destination node over two consecutive time slots is

$$\begin{array}{@{}rcl@{}} \textbf{y}_{k} & = & \left[\begin{array}{c}\textbf{y}_{d2k}\\\textbf{y}_{d1k}\\\end{array}\right]\\ & = & \sum\limits_{m=1}^{K}\!\!\left[\begin{array}{c}\sum\limits_{l=1}^{L}\textbf{G}_{kl}\textbf{F}_{l}\textbf{H}_{lm}\\\textbf{T}_{km}\\\end{array}\right]\!\textbf{B}_{m}\textbf{s}_{m} \,+\, \left[\begin{array}{c}\bar{\textbf{v}}_{dk}\\\textbf{v}_{d1k}\\\end{array}\right] \end{array} $$

((6))

where $\bar {\textbf {v}}_{\textit {dk}}= \sum _{l=1}^{L}\textbf {G}_{\textit {kl}}\textbf {F}_{l}\textbf {v}_{\textit {rl}}+\textbf {v}_{d2k}$ is the total noise vector at the kth destination node at the second time slot.

Due to their simplicity, linear receivers are used at the destination nodes to retrieve the transmitted signals. Thus, the estimated signal vector at the kth destination node can be written as

$$ \hat{\textbf{s}}_{k}=\textbf{W}_{k}^{H} \textbf{y}_{k},\quad k=1,\cdots,K $$

((7))

where $\textbf {W}_{k}=\left [\textbf {W}_{k2}^{T},\textbf {W}_{k1}^{T}\right ]^{T}$ is the receiver weight matrix, and W _k1 and W _k2 are the N _dk ×d receiver weight matrices for the direct link and the relay link, respectively. In Equations 6 and 7, we have

$$\begin{array}{@{}rcl@{}} \hat{\textbf{s}}_{k} & = & \left[\textbf{W}_{k2}^{H} \;\;\textbf{W}_{k1}^{H}\right]\left[\begin{array}{c}\textbf{y}_{d2k}\\\textbf{y}_{d1k}\\\end{array}\right]\\ &=&\underbrace{\left(\textbf{W}_{k2}^{H}\sum\limits_{l=1}^{L}\textbf{G}_{kl}\textbf{F}_{l}\textbf{H}_{lk}+ \textbf{W}_{k1}^{H}\textbf{T}_{kk}\right)\textbf{B}_{k}\textbf{s}_{k}} \\ & &\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \text{desired\: signal} \\ & &+\underbrace{\!\!\sum_{m=1,m\neq k}^{K}\!\!\left(\textbf{W}_{k2}^{H}\sum_{l=1}^{L}\textbf{G}_{kl}\textbf{F}_{l}\textbf{H}_{lm}+ \textbf{W}_{k1}^{H}\textbf{T}_{km}\right)\textbf{B}_{m}\textbf{s}_{m}} \\ & &\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \text{interference} \\ & &+\underbrace{\textbf{W}_{k2}^{H}\bar{\mathbf{v}}_{dk}+\textbf{W}_{k1}^{H}\textbf{v}_{d1k}}.\\ & &\qquad\;\;\;\;\;\;\;\;\; \text{noise} \end{array} $$

((8))

In Equations 1 and 4, the transmission power constraints at the source and relay nodes can be written as

$$\begin{array}{@{}rcl@{}} & & tr\left(\textbf{B}_{k}\textbf{B}_{k}^{H}\right) \leq P_{sk}, \quad k=1,\cdots,K \end{array} $$

((9))

$$\begin{array}{@{}rcl@{}} & & tr\left(\textbf{F}_{l}E\!\left[\mathbf{y}_{rl}\mathbf{y}_{rl}^{H}\right]\textbf{F}_{l}^{H}\right) \leq P_{rl}, \quad l=1,\cdots,L \end{array} $$

((10))

where P _sk and P _rl denote the power budget at the kth source node and the lth relay node, respectively, and $E\!\left [\textbf {y}_{\textit {rl}}\textbf {y}_{\textit {rl}}^{H}\right ]=\sum _{m=1}^{K}\textbf {H}_{\textit {lm}}\textbf {B}_{m}\textbf {B}_{m}^{H}\textbf {H}_{\textit {lm}}^{H}+\sigma _{\textit {rl}}^{2}\textbf {I}_{N_{\textit {rl}}}$ is the covariance matrix of the received signal vector at the lth relay node.

In this paper, we aim at optimizing the source precoding matrices {B _k }={B _k ,k=1,⋯,K}, the relay precoding matrices {F _l }={F _l , l=1,⋯,L}, and the receiver weight matrices {W _k }={W _k , k=1,⋯,K}, to minimize the sum-MSE of the signal waveform estimation at the destination nodes under transmission power constraints at the source and relay nodes. We would like to mention that minimal MSE (MMSE) is a sensible design criterion based on the links of MSE to other performance measures in MIMO systems such as mutual information and SINR [4,11].

From Equation 8, the MSE of the kth source-destination pair can be calculated as

$$\begin{array}{@{}rcl@{}} {}\text{MSE}_{k} & = & tr\left(E\left[\left(\hat{\textbf{s}}_{k}-\textbf{s}_{k}\right)\left(\hat{\textbf{s}}_{k}-\textbf{s}_{k}\right)^{H}\right]\right) \\ {}&=& tr\left(\left(\textbf{W}_{k}^{H}\tilde{\mathbf{H}}_{kk}-\mathbf{I}_{d}\right)\left(\textbf{W}_{k}^{H}\tilde{\mathbf{H}}_{kk}-\mathbf{I}_{d}\right)^{H} \right.\\ {}&&\left.+\textbf{W}_{k}^{H}\textbf{C}_{k}\textbf{W}_{k}+\textbf{W}_{k}^{H}\boldsymbol{\Xi}_{k}\textbf{W}_{k}{\vphantom{\frac{0}{0}}}\right),\; k=1,\cdots,K \end{array} $$

((11))

where $\tilde {\mathbf {H}}_{\textit {km}}$ is the equivalent MIMO channel matrix from the mth source node to the kth destination node, $\textbf {C}_{k}= E\left \{\left [\bar {\textbf {v}}_{\textit {dk}}^{T}, \;\textbf {v}_{d1k}^{T}\right ]^{T}\left [\bar {\textbf {v}}_{\textit {dk}}^{H}, \;\textbf {v}_{d1k}^{H}\right ]\right \}$ and Ξ _k are the covariance matrices of the equivalent noise and the interference at the kth destination node, respectively. For k,m=1,⋯,K, they are given respectively as

$$\begin{array}{@{}rcl@{}} \tilde{\mathbf{H}}_{km} &=& \left[\begin{array}{c}\sum\limits_{l=1}^{L}\textbf{G}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lm}\\\textbf{T}_{km}\mathbf{B}_{m}\\\end{array}\right], \quad \boldsymbol{\Xi}_{k} = \sum_{m=1,m\neq k}^{K}{\tilde{\textbf{H}}}_{km} \tilde{\textbf{H}}_{km}^{H} \end{array} $$

$$\begin{array}{@{}rcl@{}} \textbf{C}_{k} &=&\left[\begin{array}{cc} \sum\limits_{l=1}^{L}\sigma_{rl}^{2} \textbf{G}_{kl}\textbf{F}_{l}\textbf{F}_{l}^{H}\textbf{G}_{kl}^{H}+\sigma_{dk}^{2}\textbf{I}_{N_{dk}} & \textbf{0}\\ \textbf{0} & \sigma_{dk}^{2}\textbf{I}_{N_{dk}}\\ \end{array}\right] \end{array} $$

where $\bar {\textbf {H}}_{\textit {lm}}=\textbf {H}_{\textit {lm}}\textbf {B}_{m}$ is the equivalent MIMO channel matrix between the mth source node and the lth relay node.

From Equations 9 to 11, the optimal source, relay, and receiver matrix design problem can be written as

$$\begin{array}{@{}rcl@{}} \!\!\!\!\!\!\!\!\min_{\{\mathbf{\textbf{W}}_{k}\},\{\mathbf{\textbf{B}}_{k}\},\{\mathbf{\textbf{F}}_{l}\}} & & \sum\limits_{k=1}^{K} \text{MSE}_{k} \end{array} $$

((12))

$$\begin{array}{@{}rcl@{}} \!\!\!\!\!\!\!\!s.t. & & tr\left(\textbf{B}_{k}\textbf{B}_{k}^{H}\right) \!\leq\! P_{sk},\; k\,=\,1,\cdots,K \end{array} $$

((13))

$$\begin{array}{@{}rcl@{}} \!\!\!\!\!\!\!\!& & tr\left(\textbf{F}_{l}E\left[\mathbf{y}_{rl}\mathbf{y}_{rl}^{H}\right]\textbf{F}_{l}^{H}\right) \!\leq\! P_{rl},\; l\,=\,1,\cdots,L. \end{array} $$

((14))

3 Proposed source, relay, and receiver matrix design algorithms

The problem (Equations 12 to 14) is highly nonconvex with matrix variables, and a globally optimal solution is intractable to obtain. In this section, we propose two iterative algorithms to solve the problem (Equations 12 to 14) by optimizing {W _k }, {B _k }, and {F _l } in an alternating way through solving convex subproblems.

3.1 3.1 Proposed Algorithm 1

In each iteration of this algorithm, we first optimize {W _k } based on {B _k } and {F _l } from the previous iteration. Then, we optimize all relay matrices based on {W _k } from the current iteration and {B _k } from the previous iteration. Finally, we optimize all source matrices using {W _k } and {F _l } from the current iteration.

It can be seen from Equation 11 that W _k only affects MSE _k . Thus, with given {F _l } and {B _k }, the optimal linear receiver matrix which minimizes MSE _k in Equation 11 is the solution to the following unconstrained optimization problem

$$ \min_{\textbf{W}_{k}} \quad \text{MSE}_{k}. $$

((15))

The solution to the problem (Equation 15) is the well-known MMSE receiver [12] given by

$$\begin{array}{@{}rcl@{}}{} \textbf{W}_{k} & = & \left(\tilde{\mathbf{H}}_{kk}\tilde{\mathbf{H}}_{kk}^{H}+\textbf{C}_{k}+\boldsymbol{\Xi}_{k}\right)^{-1}\tilde{\mathbf{H}}_{kk},\;\; k=1,\cdots,K. \end{array} $$

((16))

Let us introduce f _l =vec(F _l ), l=1,⋯,L. With given receiver matrices {W _k } and source precoding matrices {B _k }, the sum-MSE $\text {SMSE}=\sum _{k=1}^{K} \text {MSE}_{k}$ can be rewritten as a function of $\mathbf {f}=\left [\mathbf {f}_{1}^{T},\;\mathbf {f}_{2}^{T},\;\cdots,\mathbf {f}_{L}^{T}\right ]^{T}$ as

$$\begin{array}{@{}rcl@{}} {}\psi_{1}(\mathbf{f}) & = & \sum\limits_{k=1}^{K}\left[\left(\mathbf{O}_{kk}\mathbf{f}-\textbf{o}_{k}\right)^{H}\vphantom{\sum\limits_{k=1}^{K} sum_{m\neq k}^{K}(\mathbf{O}_{km}\mathbf{f}-\textbf{q}_{km})^{H}(\mathbf{O}_{km}\mathbf{f}-\textbf{q}_{km})}(\mathbf{O}_{kk}\mathbf{f}-\textbf{o}_{k}) +\mathbf{f}^{H}\mathbf{{Q}}_{k}\mathbf{f}\right.\\ & &+\left.\!\sum_{m\neq k}^{K}\left(\mathbf{O}_{km}\mathbf{f}-\textbf{q}_{km}\right)^{H}\left(\mathbf{O}_{km}\mathbf{f}-\textbf{q}_{km}\right)\right]+t_{1} \end{array} $$

((17))

where $t_{1}=\sum _{k=1}^{K}\sigma _{\textit {dk}}^{2}tr\left (\textbf {W}_{k}^{H}\textbf {W}_{k}\right)$ is independent of f and for k,m=1,⋯,K, l=1,⋯,L

$$\begin{array}{@{}rcl@{}} \mathbf{{O}}_{km} & = & \left[\mathbf{O}_{k,1,m},\:\mathbf{O}_{k,2,m},\cdots,\:\mathbf{O}_{k,L,m}\right] \end{array} $$

((18))

$$\begin{array}{@{}rcl@{}} \mathbf{{Q}}_{k} & = & bd\left(\mathbf{Q}_{k1},\:\mathbf{Q}_{k2},\:\cdots,\:\mathbf{Q}_{kL}\right) \end{array} $$

((19))

$$\begin{array}{@{}rcl@{}} \textbf{o}_{k} &=& \text{vec}\left(\mathbf{I}_{d}-\bar{\textbf{T}}_{kk}\right), \quad \textbf{q}_{km} = - \text{vec}\left(\bar{\textbf{T}}_{km}\right) \end{array} $$

((20))

$$\begin{array}{@{}rcl@{}} \mathbf{O}_{k,l,m} &=& \bar{\textbf{H}}_{lm}^{T}\otimes\bar{\mathbf{G}}_{kl}, \quad \mathbf{Q}_{kl} =\sigma_{rl}^{2}\mathbf{I}_{N_{rl}}\!\otimes \left(\bar{\mathbf{G}}_{kl}^{H}\bar{\mathbf{G}}_{kl}\right). \end{array} $$

((21))

Here, $\bar {\mathbf {G}}_{\textit {kl}}= \textbf {W}_{k2}^{H}\textbf {G}_{\textit {kl}}$ is the equivalent MIMO channel matrix between the lth relay node and the kth destination node and $\bar {\textbf {T}}_{\textit {km}}=\textbf {W}_{k1}^{H}\textbf {T}_{\textit {km}}\mathbf {B}_{m}$ is the equivalent direct link MIMO channel matrix between the mth source node and the kth destination node. The detailed proof of Equation 17 is given in Appendix 6.1.

By introducing

$$ \mathbf{D}_{ll}\,=\,\left(\sum_{m=1}^{K}\!\textbf{H}_{lm}\textbf{B}_{m}\textbf{B}_{m}^{H}\textbf{H}_{lm}^{H}\,+\, \sigma_{rl}^{2}\textbf{I}_{N_{r}}\!\!\right)^{\!\!T} \!\!\otimes \mathbf{I}_{N_{rl}}, \; l=1,\cdots,L $$

((22))

and $\bar {\textbf {D}}_{l}=bd\left (\mathbf {D}_{l1},\;\mathbf {D}_{l2},\;\cdots,\mathbf {D}_{\textit {lL}}\right)$, where D _lj =0, l≠j, the relay transmit power constraints in Equation 10 can be rewritten as

$$ \textbf{f}^{H}\bar{\textbf{D}}_{l}\mathbf{f} \leq P_{rl}, \quad l=1,\cdots,L. $$

((23))

In Equations 17 and 23, the relay matrix optimization problem can be written as

$$\begin{array}{@{}rcl@{}} \min_{{\mathbf{f}}} & & \psi_{1}(\mathbf{f}) \end{array} $$

((24))

$$\begin{array}{@{}rcl@{}} s.t. & &\textbf{f}^{H}\bar{\textbf{D}}_{l}\mathbf{f}\leq P_{rl},\;\; l=1,\cdots,L. \end{array} $$

((25))

The optimization problem (Equations 24 to 25) is a quadratically constrained quadratic programming (QCQP) problem [13]. From Equation 21, we can see that Q _kl , k=1,⋯,K, l=1,⋯,L are positive semidefinite (PSD) matrices, and thus from Equation 19, Q _k , k=1,⋯,K are PSD matrices. Moreover, it can be seen from Equation 22 that D _ll , l=1,⋯,L are PSD matrices, and thus, $\bar {\textbf {D}}_{l}$, l=1,⋯,L are PSD matrices. Therefore, the QCQP problem (Equations 24 to 25) is convex and can be efficiently solved by the interior-point method [13]. In particular, the problem (Equations 24 to 25) can be solved by the CVX MATLAB toolbox for disciplined convex programming [14].

Let us introduce b _k =vec(B _k ), k=1,⋯,K. With given receiver matrices {W _k } and relay matrices {F _l }, the sum-MSE can be rewritten as a function of $\mathbf {b} =\left [\mathbf {b}_{1}^{T},\; \mathbf {b}_{2}^{T},\;\cdots, \mathbf {b}_{K}^{T}\right ]^{T}$ as

$$ {}\Phi_{1}(\mathbf{b})=\!\sum_{k=1}^{K}\left(\bar{\textbf{S}}_{k} \mathbf{b}-\text{vec}(\mathbf{I}_{d})\right)^{\!H}\!\left(\bar{\textbf{S}}_{k} \mathbf{b}-\text{vec}(\mathbf{I}_{d})\right)\,+\,\mathbf{b}^{H}\textbf{U}\mathbf{b}+t_{2}. $$

((26))

where $t_{2} =\sum _{k=1}^{K} tr(\textbf {W}_{k}^{H}\textbf {C}_{k}\textbf {W}_{k})$ can be ignored in the optimization process as it is independent of b and

$$\begin{array}{@{}rcl@{}} \textbf{U} &=& bd(\mathbf{U}_{1}, \mathbf{U}_{2}, \cdots,\mathbf{U}_{K}) \end{array} $$

((27))

$$\begin{array}{@{}rcl@{}} \bar{\textbf{S}}_{k} &=& \left[\mathbf{S}_{k1},\mathbf{S}_{k2},\cdots,\mathbf{S}_{kK}\right] \end{array} $$

((28))

$$\begin{array}{@{}rcl@{}} \mathbf{S}_{kk} &=& \mathbf{I}_{d}\otimes \bar{\mathbf{P}}_{kk}, \quad \mathbf{S}_{ki}=\textbf{0}, i\ne k \end{array} $$

((29))

$$\begin{array}{@{}rcl@{}} \mathbf{U}_{k} &=& \mathbf{I}_{d}\otimes \left(\sum_{m=1,m\neq k}^{K}\bar{\mathbf{P}}_{mk}^{H}\bar{\mathbf{P}}_{mk}\right). \end{array} $$

((30))

Here, $\bar {\mathbf {P}}_{\textit {mk}} = \textbf {W}_{m2}^{H}\sum \limits _{l=1}^{L}\bar {\mathbf {G}}_{\textit {ml}}\textbf {F}_{l}\textbf {H}_{\textit {lk}} + \textbf {W}_{m1}^{H}\textbf {T}_{\textit {mk}}$. The detailed proof of Equation 26 is given in Appendix 7.1.

Let us introduce $\textbf {E}_{\textit {ij}}=\textbf {I}_{d}\otimes \left (\textbf {H}_{\textit {ij}}^{H}\textbf {F}_{i}^{H}\textbf {F}_{i}\textbf {H}_{\textit {ij}}\right)$, E _l =b d(E _l1,E _l2,⋯,E _lK ), $\bar {\textbf {E}}_{i}=bd\left (\bar {\textbf {E}}_{i1},\bar {\textbf {E}}_{i2},\cdots,\bar {\textbf {E}}_{\textit {iK}}\right)$, where $\bar {\textbf {E}}_{\textit {ii}}=\textbf {I}_{{dN}_{s}}$ and $\bar {\textbf {E}}_{\textit {ij}}=\textbf {0},i\neq j$. The optimal b can be obtained by solving the following problem

$$\begin{array}{@{}rcl@{}}{} \min_{{\textbf{b}}} & & \Phi_{1}(\mathbf{b}) \end{array} $$

((31))

$$\begin{array}{@{}rcl@{}} s.t. & & \textbf{b}^{H}\bar{\textbf{E}}_{m}\textbf{b}\leq P_{sm},\;\; m=1,\cdots,K \end{array} $$

((32))

$$\begin{array}{@{}rcl@{}} & & \textbf{b}^{H}{\textbf{E}}_{l}\textbf{b}\leq P_{rl}-\sigma_{rl}^{2}tr\left(\textbf{F}_{l}\textbf{F}_{l}^{H}\right),\;\; l=1,\cdots,L. \end{array} $$

((33))

From Equation 30, we can see that U _k , k=1,⋯,K are PSD matrices, and thus from Equation 27, U is PSD. Moreover, it can be seen that $\bar {\textbf {E}}_{m}$, m=1,⋯,K and E _l , l=1,⋯,L are PSD matrices. Therefore, the problem (Equations 31 to 33) is a convex QCQP problem and can be solved by the CVX MATLAB toolbox [14] for disciplined convex programming.

The steps of applying the proposed Algorithm 1 to optimize {B _k }, {F _l }, and {W _k } are summarized in Table 1, where the superscript (n) denotes the variable at the nth iteration, and ε is a small positive number up to which convergence is acceptable. Since all subproblems (Equation 15, 24 to 25, and 31 to 33) are convex, the solution to each subproblem is optimal. Thus, the value of the objective function (Equation 12) monotonically decreases after each iteration. Moreover, the value of Equation 12 is lower bounded by at least zero. Therefore, the proposed Algorithm 1 is guaranteed to converge.

Table 1 Procedure of solving the problem (Equations 12 to 14 ) by the proposed Algorithm 1

3.2 3.2 Proposed Algorithm 2

In the proposed Algorithm 1, all source precoding matrices are optimized together through b, while all relay precoding matrices are updated together through f. Since the dimensions of b and f are $\sum _{k=1}^{K} \!N_{\textit {sk}} d$ and $\sum _{l=1}^{L} \!N_{\textit {rl}}^{2}$, respectively, the computational complexity of solving the QCQP problems (Equations 24 to 25 and 31 to 33) using the interior point method [15] is ${\mathcal O}\left (\left (\sum _{k=1}^{K} \!N_{\textit {sk}} d\right)^{3}\right)$ and ${\mathcal O}\left (\left (\sum _{l=1}^{L} \!N_{\textit {rl}}^{2}\right)^{3}\right)$, respectively. Therefore, the computational complexity at each iteration of the proposed Algorithm 1 is ${\mathcal O}\left (\left (\sum _{k=1}^{K} \!N_{\textit {sk}} d\right)^{3}+\left (\sum _{l=1}^{L} \!N_{\textit {rl}}^{2}\right)^{3}\right)$, which can be very high for interference MIMO relay systems with a large K and L. To reduce the per-iteration complexity, in this subsection, we develop an iterative algorithm where each source and relay matrix is optimized individually by fixing all other matrices.

Adopting notations from proposed Algorithm 1, with given receiver matrices {W _k }, source precoding matrices {B _k }, and relay precoding matrices F _j , j=1,⋯,L, j≠l, the sum-MSE can be rewritten as a function of F _l as

$$\begin{array}{@{}rcl@{}} \text{SMSE} &=& \sum_{k=1}^{K}tr \left[\left(\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lk}- \mathbf{A}_{kl}\right)\left(\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lk}-\mathbf{A}_{kl}\right)^{H}\right.\\ & &+\sum_{l=1}^{L} \sigma_{rl}^{2}\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\textbf{F}_{l}^{H}\bar{\mathbf{G}}_{kl}^{H}+\!\!\!\!\!\!\!\sum_{m=1,m\neq k}^{K}\!\!\!\!\!\! \left(\bar{\mathbf{G}}_{kl} \textbf{F}_{l} \bar{\textbf{H}}_{lm}\,-\,\mathbf{D}_{k,l,m}\right) \\ &&\left.\times\left(\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lm}\,-\,\mathbf{D}_{k,l,m}\right)^{H}\!\right]+t_{1} \end{array} $$

((34))

where for k,m=1,⋯,K, l=1,⋯,L

$$\begin{array}{@{}rcl@{}} \mathbf{A}_{kl} = \mathbf{I}_{d}-\!\!\!\!\sum_{j=1,j\neq l}^{L}\!\!\!\!\bar{\mathbf{G}}_{kj}\mathbf{F}_{j}\bar{\textbf{H}}_{jk}-{\mathbf{T}}_{kk}\\ \mathbf{D}_{k,l,m} = -\!\!\!\!\sum_{j=1,j\neq l}^{L}\!\!\!\! \bar{\mathbf{G}}_{kj}\textbf{F}_{j}\bar{\textbf{H}}_{jm}-{\mathbf{T}}_{km}. \end{array} $$

((35))

Using the identities in Equations 43 to 45, the SMSE in Equation 34 can be written as

$$\begin{array}{@{}rcl@{}} & & \psi_{2}(\mathbf{f}_{l})=\\ & & \sum_{k=1}^{K}\left[{\vphantom{\sum_{m=1,m\neq k}^{K}}}(\mathbf{O}_{k,l,k}\mathbf{f}_{l}-\mathbf{a}_{kl})^{H}(\mathbf{O}_{k,l,k}\mathbf{f}_{l}-\mathbf{a}_{kl})+\mathbf{f}_{l}^{H}\mathbf{Q}_{kl}\mathbf{f}_{l}+r_{kl}\right.\\ & & \left.+ \!\!\!\sum_{m=1,m\neq k}^{K} \!\!\!\left(\mathbf{{O}}_{k,l,m}\mathbf{f}_{l}- \mathbf{d}_{k,l,m}\right)^{H}\!\!\left(\mathbf{{O}}_{k,l,m}\mathbf{f}_{l}-\mathbf{d}_{k,l,m}\right)\right] \end{array} $$

((36))

where for k,m=1,⋯,K, l=1,⋯,L

$$\begin{array}{@{}rcl@{}} \mathbf{a}_{kl} &=& \text{vec}(\mathbf{A}_{kl}) \\ r_{kl} &=& tr\left(\sum_{j=1,j\neq l}^{L}\!\!\!\sigma_{rj}^{2}\bar{\mathbf{G}}_{kj}\textbf{F}_{j}\textbf{F}_{j}^{H}\bar{\mathbf{G}}_{kj}^{H}+\sigma_{dk}^{2}\textbf{W}_{k}^{H}\textbf{W}_{k}\right) \\ \mathbf{d}_{k,l,m} &=& \text{vec}\left(\mathbf{D}_{k,l,m}\right). \end{array} $$

Note that since the terms r _kl in Equation 36 are independent of f _l , they can be ignored when optimizing f _l . The relay transmit power constraint in Equation 10 can be rewritten as

$$ \textbf{f}_{l}^{H}\textbf{D}_{ll}\textbf{f}_{l}\leq P_{rl}. $$

((37))

Based on Equations 36 and 37, the optimal f _l can be obtained by solving the following problem for each l=1,⋯,L

$$ \min_{\textbf{f}_{l}} \psi_{2} (\mathbf{f}_{l}) \qquad s.t. \;\; \textbf{f}_{l}^{H}\textbf{D}_{l}\textbf{f}_{l}\leq P_{rl}. $$

((38))

The problem (Equation 38) is a QCQP problem and can be solved effectively using the CVX toolbox.

With given receiver matrices {W _k }, relay precoding matrices {F _l }, and source precoding matrices B _j , j=1,⋯,K, j≠k, the SMSE can be rewritten as a function of b _k as

$$\begin{array}{@{}rcl@{}} \Phi_{2}(\mathbf{b}_{k}) &=& (\mathbf{S}_{kk} \mathbf{b}_{k}-\text{vec}(\mathbf{I}_{d}))^{H}(\mathbf{S}_{kk} \mathbf{b}_{k}-\text{vec}(\mathbf{I}_{d}))\\ & & +\mathbf{b}_{k}^{H}\mathbf{U}_{k}\mathbf{b}_{k} + z_{k} \end{array} $$

where

$$\begin{array}{@{}rcl@{}} z_{k} &=& \sum_{m=1,m\neq k}^{K}\left[(\mathbf{S}_{mm} \mathbf{b}_{m}-\text{vec}(\mathbf{I}_{d}))^{H}(\mathbf{S}_{mm} \mathbf{b}_{m}-\text{vec}(\mathbf{I}_{d}))\right.\\ & & \qquad \qquad \left.+\mathbf{b}_{m}^{H}\mathbf{U}_{m} \mathbf{b}_{m}\right]+t_{2}, \quad k=1,\cdots,K. \end{array} $$

By introducing ${c}_{\textit {lk}}=\sigma _{\textit {rl}}^{2}tr\left (\textbf {F}_{l}\textbf {F}_{l}^{H}\right) +\sum _{j=1,j\neq k}^{K}\textbf {b}_{j}^{H}\textbf {E}_{\textit {lj}}\textbf {b}_{j}$, k=1,⋯,K, l=1,⋯,L, the optimal b _k can be obtained by solving the following problem for each k=1,⋯,K

$$\begin{array}{@{}rcl@{}} \min_{{\textbf{b}_{k}}} & & \Phi_{2}(\mathbf{b}_{k}) \end{array} $$

((39))

$$\begin{array}{@{}rcl@{}} s.t. & & \textbf{b}_{k}^{H}\textbf{b}_{k}\leq P_{sk} \end{array} $$

((40))

$$\begin{array}{@{}rcl@{}} & & \textbf{b}_{k}^{H}\textbf{E}_{lk}\textbf{b}_{k}\leq P_{rl}-{c}_{lk},\;\; l=1,\cdots,L. \end{array} $$

((41))

The problem (Equations 39 to 41) is a QCQP problem and can be solved by the CVX MATLAB toolbox [14] for disciplined convex programming. The steps of using the proposed Algorithm 2 to optimize {B _k }, {F _l }, and {W _k } are summarized in Table 2. Similar to the analysis used to the proposed Algorithm 1, since all subproblems (Equations 15, 38, and 39 to 41) are convex, the solution to each subproblem is optimal. Thus, the value of the objective function (Equation 12) monotonically decreases after each iteration. Moreover, the value of Equation 12 is lower bounded by at least zero. Therefore, the convergence of the proposed Algorithm 2 follows directly from this observation.

Table 2 Procedure of solving the problem (Equations 12 to 14 ) by the proposed Algorithm 2

Since the dimensions of b _k and f _l are N _sk d and $N_{\textit {rl}}^{2}$, respectively, the computational complexity of solving the QCQP problems (Equations 38 and 39 to 41) is ${\mathcal O}\left ((N_{\textit {sk}}d)^{3}\right)$ and ${\mathcal O}\left (N_{\textit {rl}}^{6}\right)$, respectively. Thus, the computational complexity at each iteration of the proposed Algorithm 2 is ${\mathcal O}\left (\sum _{k=1}^{K}\!(N_{\textit {sk}}d)^{3}+ \sum _{l=1}^{L}\! N_{\textit {rl}}^{6}\right)$, which is lower than the per-iteration computational complexity of the proposed Algorithm 1. However, we will see through numerical simulations that the proposed Algorithm 1 has a better MSE and BER performance than that of the proposed Algorithm 2. Such performance-complexity trade-off is very useful for practical interference MIMO relay communication systems.

4 Numerical examples

In this section, we illustrate the performance of the proposed algorithms through numerical simulations. All channel matrices have independent and identically distributed (i.i.d.) complex Gaussian entries with zero mean and unit variance. The noises are i.i.d. Gaussian with zero mean and unit variance. Unless explicitly mentioned, the QPSK constellations are used to modulate the source symbols. For the sake of simplicity, we set d=2 and assume that all nodes have three antennas, i.e., N _sk =N _dk =N _rl =3, k=1,⋯,K, l=1,⋯,L, all source nodes have the same power budget as P _sk =15dB, k=1,⋯,K, and all relay nodes have the same power budget as P _rl =P, l=1,⋯,L.

For all simulation examples, the simulation results are averaged over 10⁵ independent channel realizations. Unless explicitly mentioned, we assume that there are K=4 source-destination pairs and L=5 relay nodes in the interference MIMO relay system. The proposed algorithms are initialized at $\textbf {F}_{l}^{(0)}=\vspace *{-1pt}{\sqrt {P_{\textit {rl}} /tr\left (\sum _{k=1}^{K} \bar {\textbf {H}}_{\textit {lk}}\bar {\textbf {H}}_{\textit {lk}}^{H}+\textbf {I}_{N_{\textit {rl}}}\right)} \textbf {I}_{N_{\textit {rl}}}}$, l=1,⋯,L, and $\textbf {B}_{k}^{(0)}={\sqrt {P_{\textit {sk}}/N_{\textit {sk}}}\textbf {I}_{N_{\textit {sk}}}}$, k=1,⋯,K. We would like to mention that when the matrix weight is identity matrix, the performance of the matrix-weighted sum-MSE minimization (WMSE) algorithm without power control in [10] is similar to the proposed Algorithm 2 without considering the direct links.

In the first example, we study the performance of the proposed algorithms at different number of iterations. We also compare the performance of the algorithms when the direct links are ignored. Moreover, the performance of the total leakage minimization (TLM) algorithm in [10] is included as a benchmark. Figure 2 shows the MSE performance of the proposed algorithms versus P at different number of iterations for the first source-destination pair (k=1). It can be seen from Figure 2 that both proposed algorithms perform better than the TLM algorithm when the direct links are ignored. The performance of both proposed algorithm is significantly improved when the direct links are taken into account. For both proposed algorithms, the MSE reduces with increasing number of iterations. Moreover, it can be observed that after ten iterations, the decreasing of the MSE is small. Thus, we suggest that only ten iterations need to be carried out in practice to achieve a good performance-complexity trade-off. It can also be seen from Figure 2 that both proposed algorithms have almost the same MSE performance at convergence.

Figure 2

Example 1: MSE versus P at different number of iterations.

For this example, the average BER of all source-destination pairs yielded by both proposed algorithms versus P at different number of iterations is shown in Figure 3. It can be clearly seen that the proposed algorithms with direct links yield much smaller BER than the case when the direct links are ignored, especially at high P level. We can also observe from Figure 3 that the proposed Algorithm 1 has a slightly better BER performance than the proposed Algorithm 2. It can also be seen from Figure 3 that when the direct links are ignored, the proposed algorithms perform better than the TLM algorithm.

Figure 3

Example 1: BER versus P at different number of iterations.

In the second example, we study the performance of the proposed algorithms with different number of relay nodes. Figure 4 shows the MSE performance of the proposed Algorithm 1 versus P with L = 5 and L = 10. It can be seen that by doubling the number of relay nodes, a power gain of 10 dB is obtained at the MSE of 0.2.

Figure 4

Example 2: MSE versus P for different L .

For this example, the BER performance of the proposed Algorithm 1 with L = 5 and L = 10 is illustrated in Figure 5. It can be seen that by increasing the number of relay nodes, the system spatial diversity is increased, and thus, a better BER performance is achieved. In particular, we observe that an 8 dB gain is obtained at the BER of 10⁻³ by increasing L from 5 to 10.

Figure 5

Example 2: BER versus P for different L .

In the next example, we study the performance of the proposed algorithms with different number of source-destination pairs. Figure 6 shows the BER performance of both proposed algorithms versus P. Moreover, the BER of both algorithms using the 16QAM modulation scheme is also illustrated in Figure 6. As expected, the system BER is increased when higher order constellations are used. We can also obverse from Figure 6 that with a smaller number of source-destination pairs, the number of interference channels decreases which yields a better BER performance. Interestingly, the BER difference between the two proposed algorithm becomes bigger when K=3.

Figure 6

Example 3: BER versus P for different K .

In the last example, we study the performance of the proposed algorithms on the achievable end-to-end sum-rates of all source-destination pairs. It can be seen from Figure 7 that, as expected, with the direct links taken into account, both proposed algorithms achieve a higher sum-rate. Figure 7 shows that the proposed Algorithm 1 yields slightly better rate than the proposed Algorithm 2.

Figure 7

Example 4: achievable end-to-end sum-rates.

5 Conclusions

We have investigated the transceiver design for interference MIMO relay systems with direct source-destination links based on the MMSE criterion. Two iterative algorithms have been developed to jointly optimize the source, relay, and receiver matrices under power constrains at each source node and relay node. Numerical simulation results show that the proposed algorithms converge quickly after a few iterations. The system MSE and BER performance can be significantly improved compared with the algorithms without considering the direct links. The proposed Algorithm 1 has a better MSE and BER performance than the proposed Algorithm 2 at a higher per-iteration computational complexity.

6 Appendix A

6.1 6.1 Proof of Equation 17

From Equation 11, we have

$${} \begin{aligned} & \text{SMSE} \\ & =\sum_{k=1}^{K}\!tr\left[\!\left(\sum_{l=1}^{L}\!\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lk}\!+ \bar{\textbf{T}}_{kk}-\mathbf{I}_{d}\right)\right.\\ & \left.\times\left(\sum_{l=1}^{L}\bar {\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lk}+\bar{\textbf{T}}_{kk}\,-\,\mathbf{I}_{d}\right)^{\!H}+\sum\limits_{l=1}^{L}\sigma_{rl}^{2}\bar {\mathbf{G}}_{kl}\textbf{F}_{l}\textbf{F}_{l}^{H}\bar{\mathbf{G}}_{kl}^{H}\right.\\ & \left.+\sigma_{dk}^{2}\textbf{W}_{k2}^{H}\textbf{W}_{k2}+\sigma_{dk}^{2}\textbf{W}_{k1}^{H}\textbf{W}_{k1}\right.\\ & \left.+\!\!\sum_{m\neq k}^{K}\!\left(\sum_{l=1}^{L}\!\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lm}+\bar{\textbf{T}}_{km}\right)\!\!\left(\sum_{l=1}^{L}\!\bar{\mathbf{G}}_{kl}\textbf{F}_{l}\bar{\textbf{H}}_{lm}\,+\,\bar{\textbf{T}}_{km}\right)^{\!\!H}\!\!\right]. \end{aligned} $$

((42))

Using the identities of [16]

$$\begin{array}{@{}rcl@{}} tr\left(\mathbf{A}^{T}\mathbf{B}\right)&=&\left(\text{vec}\left(\mathbf{A}\right)\right)^{T}\text{vec}(\mathbf{B}) \end{array} $$

((43))

$$\begin{array}{@{}rcl@{}} tr\left(\mathbf{A}^{H}\mathbf{BAC}\right)&=&\left(\text{vec}(\mathbf{A})\right)^{H}\left(\mathbf{C}^{T}\!\otimes \mathbf{B}\right)\text{vec}(\mathbf{A}) \end{array} $$

((44))

$$\begin{array}{@{}rcl@{}} \text{vec}(\mathbf{ABC})&=&\left(\mathbf{C}^{T}\!\otimes\mathbf{A}\right)\text{vec}(\mathbf{B}) \end{array} $$

((45))

the SMSE (Equation 42) can be represented as a function of f _l , l=1,⋯,L, as

$$\begin{array}{@{}rcl@{}} \text{SMSE} &=& \sum\limits_{k=1}^{K}\left[\left(\sum_{l=1}^{L} \textbf{O}_{k,l,k} \textbf{f}_{l} - \textbf{o}_{k}\right)^{\!H}\left(\sum_{l=1}^{L}\textbf{O}_{k,l,k} \textbf{f}_{l}-\textbf{o}_{k}\right)\right.\\ & & \left. +\!\sum_{m\neq k}^{K}\!\left(\sum_{l=1}^{L}\!\textbf{O}_{k,l,m} \textbf{f}_{l}\!-\textbf{q}_{km}\right)^{H}\!\!\left(\sum_{l=1}^{L}\!\textbf{O}_{k,l,m} \textbf{f}_{l}\!-\textbf{q}_{km}\!\right)\right.\\ & & \left. +\sum_{l=1}^{L}\textbf{f}_{l}^{H}\textbf{Q}_{kl}\textbf{f}_{l} \!\right]\,+\,t_{1}\\ &=& \psi_{1}(\mathbf{f}). \end{array} $$

7 Appendix B

7.1 7.1 Proof of Equation 26

From Equation 11, we have

$$\begin{array}{@{}rcl@{}} \text{SMSE} &=&\sum_{k=1}^{K} \!tr\left[\left(\bar{\mathbf{P}}_{kk}\textbf{B}_{k}\,-\,\mathbf{I}_{d}\right) \left(\bar{\mathbf{P}}_{kk}\textbf{B}_{k}\,-\,\mathbf{I}_{d}\right)^{\!H}\right.\\ & & +\left.\sum_{m=1,m\neq k}^{K}\bar{\mathbf{P}}_{km}\textbf{B}_{m} \textbf{B}_{m}^{H}\bar{\mathbf{P}}_{km}^{H}\right]+t_{2}. \end{array} $$

((46))

Using the identities in Equations 43 to 45, the SMSE function in Equation 46 can be written as

$$\begin{array}{@{}rcl@{}} & & \text{SMSE} \\ & & = \sum_{k=1}^{K} \left[{\vphantom{\sum_{m=1,m\neq k}^{K}}}\left(\mathbf{S}_{kk} \mathbf{b}_{k}-\text{vec}(\mathbf{I}_{d})\right)^{H}\left(\mathbf{S}_{kk} \mathbf{b}_{k}-\text{vec}(\mathbf{I}_{d})\right)\right.\\ & & \left.+\sum_{m=1,m\neq k}^{K}\textbf{b}_{m}^{H}\left(\mathbf{I}_{d}\otimes\bar{\mathbf{P}}_{km}^{H}\bar{\mathbf{P}}_{km}\right) \textbf{b}_{m}\right]\,+\,t_{2} \\ & & = \!\sum_{k=1}^{K} \!\left[\!(\mathbf{S}_{kk} \mathbf{b}_{k}\,-\,\text{vec}(\mathbf{I}_{d}))^{H}\!(\mathbf{S}_{kk} \mathbf{b}_{k}\,-\,\text{vec}(\mathbf{I}_{d}))\,+\,\mathbf{b}_{k}^{H}\mathbf{U}_{k} \mathbf{b}_{k} \!\right]\!\! +\!t_{2} \\ & & = \Phi_{1}(\textbf{b}). \end{array} $$