Low-complexity QL-QR decomposition- based beamforming design for two-way MIMO relay networks

Abstract

In this paper, we investigate the optimization problem of joint source and relay beamforming matrices for a two-way amplify-and-forward (AF) multi-input multi-output (MIMO) relay system. The system, consisting of two source nodes and two relay nodes, is considered, and the linear minimum mean-square-error (MMSE) is employed at both receivers. We assume individual relay power constraints and study an important design problem, a so-called determinant maximization (DM) problem. Since this DM problem is nonconvex, we consider an efficient iterative algorithm by using an MSE balancing result to obtain at least a locally optimal solution. The proposed algorithm is developed based on QL, QR, and Choleskey decompositions which differ in complexity and performance. Analytical and simulation results show that the proposed algorithm can significantly reduce computational complexity compared with their existing two-way relay systems and have equivalent bit-error-rate (BER) performance to the singular value decomposition (SVD) based on a regular block diagonal (RBD) scheme.

1 Introduction

Recently, wireless relay networks have been the focus of a lot of research because the relaying transmission is a promising technique which can be applied to extend the coverage or increase the system capacity. Various cooperative relaying schemes have been proposed, such as amplify-and-forward (AF) [1, 2], decode-and-forward (DF) [3], denoise-and-forward (DNF) [4], and compress-and-forward (CF) [5] cooperative relaying protocols. Among these approaches, an AF scheme is most widely used because it does not need to detect the transmitted signal. In addition, it requires less processing power at the relays compared to other schemes.

In a one-way relaying (OWR) approach, to completely exchange information between two base stations, four time slots are required in uplink (UL) and downlink (DL) communications, which leads to a loss of one-half spectral resources [6]. In order to solve this problem, a two-way relaying approach has been considered in [7–9]. In a typical two-way relaying scheme, the communication is completed in two steps. First, the transmitters send their symbols to two relays, simultaneously. Upon receiving the signals, each relay processes them based on an efficient relaying scheme to produce new signals. After these processes, the processed signals are broadcasted to both receiver nodes.

Multi-input multi-output (MIMO) relay systems have been investigated in [10–13]. It is shown that, by employing multiple antennas at the transmitter and/or the receiver, one can significantly improve the transmission reliability by leveraging spatial diversity. Relay precoder design methods have been investigated in [14–16]. A problem in designing optimal beamforming vectors for multi-casting is challenging due to its nonconvex nature. In [14], the authors propose a transceiver precoding scheme at the relay node by using zero-forcing (ZF) and MMSE criteria with certain antenna configurations. The information theoretic capacity of the multi-antenna multicasting is studied in [15], along with the achievable rates using lower complexity transmission schemes, as the number of antennas or users goes to infinity. In [16], the authors propose an alternative method to characterize the capacity region of a two-way relay channel (TWRC) by applying the idea of rate profile.

Joint optimizations of the relay and source nodes for the MIMO TWRC have been studied in [9, 17]. In [9], the authors develop a unified framework for optimizing two-way linear non-regenerative MIMO relay systems and show that the optimal relay and source matrices have a general beamforming structure. The joint source node and relay precoding design for minimizing the mean squared error in a MIMO two-way relay (TWR) system is studied in [17].

Since singular value decomposition (SVD) and generalized SVD (GSVD) are widely used to find the orthogonal complement to solve an optimization problem [2, 9, 16, 33], but their computational complexity is extremely high. In order to reduce the complexity, the SVD can be replaced with a less complex QR decomposition [18] in this work. However, this approach leads to degrading the bit-error-rate (BER) performance. In addition, it is difficult to realize in the TWRC. In this paper, we investigate the joint source and relay precoding matrix optimization for a two-way relay amplify-and-forward relaying system where two source nodes and two relay nodes are equipped with multiple antennas. Also, in order to apply the QL-QR decomposition to the TWRC, we design a three-part relay filter. Compared with existing works such as [9–14], the contributions of this paper can be summarized as follows. Firstly, we investigate a two-way MIMO relay system using the criteria which minimize an MSE of the signal waveform estimation for both two source nodes. We prove an optimal sum-MSE solution can be obtained as the Wiener filter while the signal-to-noise ratios (SNR) at both source nodes are equivalent [20], which leads to an MSE balancing result. Secondly, we propose a new cooperative scenario, i.e., the QL-QR compared with the Choleskey decomposition which significantly reduces the computational complexity of the optimal design. In this proposed design, the channels of its left side are decomposed by the QL decomposition while those of its right side factorized by the QR decomposition. And the equivalent noise covariance is decomposed by the Choleskey decomposition. We also design the three-part relay filter, which is comprised of a left filter, a middle filter, and a right filter, to efficiently combine two source nodes and the relay nodes. By these approaches, the received signals at both two source nodes are able to be redeemed as either lower or upper triangular matrices. Stemming from one of the properties of triangular matrices such that their determinant is identical to the multiplication of their eigenvalues, we are able to straightforwardly solve the optimization problem as a determinant maximization problem. Also, we can obtain the BER performance equivalent to that of the singular value decomposition-regular block diagonal (SVD-RBD) scheme.

The rest of this paper is organized as follows. Section 2 describes a system model of the TWRC and raises a sum-MSE problem. In Section 3, we propose an iterative QL-QR algorithm and a joint optimal beamforming design. In Section 4, we discuss the computational complexity of an efficient channel model. The simulation results are presented to show the excellent performance of our proposed algorithm for the TWRC in Section 5. Section 6 concludes this paper.

Notations: A ^∗, A ^T, A ^H, E(A), tr(A), ℜ(A), and det(A) denote the conjugate, transpose, Hermitian transpose, statistical expectation, trace, real part, and determinate of a matrix A, respectively. An N×N identity matrix is denoted by I _N .

2 System model and sum-MSE

We consider a TWRC consisting of two source nodes, S ₁ and S ₂, and two relay nodes, R ₁ and R ₂, as shown in Fig. 1. The source and relay nodes are equipped with M and N antennas, respectively. We adopt the relay protocol with two time slots introduced in [14]. In the first time slot, the information vector $\mathbf {x}_{i}\in {\mathbb {C}^{G\times {1}}}$, where G≤M, is linearly processed by a precoding matrix, $\mathbf {V}_{i}\in {\mathbb {C}^{M\times {M}}}$, and then transmitted to the relay nodes. In this paper, we assume that each transmit antenna satisfies the unity transmission power constraint, which is $\text {tr}\left \{\mathbf {x}_{i}\mathbf {x}^{\mathrm {H}}_{i}\right \}=\mathbf {I}_{M}$. The received signals at R _i , i∈{1,2} can be expressed as

$$\begin{array}{@{}rcl@{}} &&\mathbf{y}_{R_{1}}=\mathbf{H}_{1,1}\mathbf{s}_{1}+\mathbf{H}_{1,2}s_{2}+\mathbf{n}_{R_{1}}\notag\\ &&\mathbf{y}_{R_{2}}=\mathbf{H}_{2,1}\mathbf{s}_{1}+\mathbf{H}_{2,2}s_{2}+\mathbf{n}_{R_{2}}, \end{array} $$

((1))

Fig. 1

Proposed QL-QR amplify-and-forward MIMO TWR system

where $\mathbf {y}_{R_{i}}\in {\mathbb {C}^{N\times 1}}, i\in \{1,2\}$, indicates the received signal vector, $\mathbf {H}_{i,j}\in {\mathbb {C}^{N\times {M}}}, i,j\in \{1,2\}$, represents the channel matrix from source j to relay i, as shown in Fig. 1, $\mathbf {s}_{i}=\mathbf {V}_{i}\mathbf {x}_{i}\in {\mathbb {C}^{M\times 1}}$ is the transmitted symbol vector from S _i with a power constraint $\text {tr}\left \{E\left (\mathbf {s}_{i}\mathbf {s}^{\mathrm {H}}_{i}\right)\right \}\leq {P_{i}}$, and $\mathbf {n}_{R_{i}}\sim {CN\left (0,\sigma ^{2}_{R_{i}}\mathbf {I}_{N}\right)}$ represents the additive white Gaussian noise (AWGN) vector with zero mean and variance $\sigma ^{2}_{R_{i}}$ at relay node i.

In the second time slot, the relay node R _i linearly amplifies $\mathbf {y}_{R_{i}}$ with an N×N matrix F _i and then broadcasts the amplified signal vector $\mathbf {x}_{R_{i}}$ to source nodes 1 and 2. The signal transmitted from relay node i can be expressed as

$$\begin{array}{@{}rcl@{}} \mathbf{x}_{R_{i}}=\mathbf{F}_{i}\mathbf{y}_{R_{i}}. \end{array} $$

((2))

Using (1) and (2), the received signal vectors at S ₁ and S ₂ can be, respectively, written as

$${} \begin{aligned} \mathbf{y}_{1}&=\mathbf{H}^{\mathrm{T}}_{1,1}\mathbf{F}_{1}\mathbf{H}_{1,1}\mathbf{s}_{1}+\mathbf{H}^{\mathrm{T}}_{1,1}\mathbf{F}_{1}\mathbf{H}_{1,2}\mathbf{s}_{2} +\mathbf{H}^{\mathrm{T}}_{2,1}\mathbf{F}_{2}\mathbf{H}_{2,1}\mathbf{s}_{1}\\ &\quad+\,\mathbf{H}^{\mathrm{T}}_{2,1}\mathbf{F}_{2}\mathbf{H}_{2,2}\mathbf{s}_{2}+\mathbf{H}^{\mathrm{T}}_{1,1}\mathbf{F}_{1}\mathbf{n}_{R_{1}}, +\,\mathbf{H}^{\mathrm{T}}_{2,1}\mathbf{F}_{2}\mathbf{n}_{R_{2}}+\mathbf{n}_{1}\\ \mathbf{y}_{2}&=\mathbf{H}^{\mathrm{T}}_{1,2}\mathbf{F}_{1}\mathbf{H}_{1,1}\mathbf{s}_{1}+\mathbf{H}^{\mathrm{T}}_{1,2}\mathbf{F}_{1}\mathbf{H}_{1,1}\mathbf{s}_{2} +\mathbf{H}^{\mathrm{T}}_{2,2}\mathbf{F}_{2}\mathbf{H}_{2,1}\mathbf{s}_{1}\\ &\quad+\,\mathbf{H}^{\mathrm{T}}_{2,2}\mathbf{F}_{2}\mathbf{H}_{2,1}\mathbf{s}_{2}+\mathbf{H}^{\mathrm{T}}_{1,2}\mathbf{F}_{1}\mathbf{n}_{R_{1}}+\mathbf{H}^{\mathrm{T}}_{2,2}\mathbf{F}_{2}\mathbf{n}_{R_{2}}+\mathbf{n}_{2}. \end{aligned} $$

((3))

where ${\mathbf {H}}^{\mathrm {T}}_{i,j}$, i,j∈{1,2}, indicates the M×N channel matrix from the relay node i to the source node j and n _i , i∈{1,2}, is an M×1 noise vector at S _i .

We assume that the relay nodes perfectly know the channel state information (CSI) of H _i,j . The relay node R _i performs the optimizations of F _i and V _i and then transmits the information to the source nodes 1 and 2. Since source node i knows its own transmitted signal vector s _i and full CSI, the self-interference components in (3) can be efficiently canceled. The effective received signal vectors are given by

$$\begin{array}{@{}rcl@{}} \widetilde{\mathbf{y}}_{1}&=&\mathbf{H}^{\mathrm{T}}_{1,1}\mathbf{F}_{1}\mathbf{H}_{1,2}\mathbf{s}_{2}+\mathbf{H}^{\mathrm{T}}_{2,1}\mathbf{F}_{2}\mathbf{H}_{2,2}\mathbf{s}_{2}+\mathbf{H}^{\mathrm{T}}_{1,1}\mathbf{F}_{1}\mathbf{n}_{R_{1}}\notag\\ &&+\ \mathbf{H}^{\mathrm{T}}_{2,1}\mathbf{F}_{2}\mathbf{n}_{R_{2}}+\mathbf{n}_{1}\notag\\ &=&\mathbf{\widetilde{H}}_{1}\mathbf{s}_{2}+\widetilde{\mathbf{n}}_{1}, \end{array} $$

((4))

$$\begin{array}{@{}rcl@{}} \widetilde{\mathbf{y}}_{2}&=&\mathbf{H}^{\mathrm{T}}_{1,2}\mathbf{F}_{1}\mathbf{H}_{1,1}\mathbf{s}_{1}+\mathbf{H}^{\mathrm{T}}_{2,2}\mathbf{F}_{2}\mathbf{H}_{2,1}\mathbf{s}_{1}+\mathbf{H}^{\mathrm{T}}_{1,2}\mathbf{F}_{1}\mathbf{n}_{R_{1}}\notag\\ &&+\ \mathbf{H}^{\mathrm{T}}_{2,2}\mathbf{F}_{2}\mathbf{n}_{R_{2}}+\mathbf{n}_{2}\notag\\ &=&\mathbf{\widetilde{H}}_{2}\mathbf{s}_{1}+\widetilde{\mathbf{n}}_{2}, \end{array} $$

((5))

where $\widetilde {\mathbf {H}}_{1}=\mathbf {H}^{\mathrm {T}}_{1,1}\mathbf {F}_{1}\mathbf {H}_{1,2}+ \mathbf {H}^{\mathrm {T}}_{2,1}\mathbf {F}_{2}\mathbf {H}_{2,2}$ and $\widetilde {\mathbf {H}}_{2}=\mathbf {H}^{\mathrm {T}}_{1,2}$ $\mathbf {F}_{1}\mathbf {H}_{1,1}+\mathbf {H}^{\mathrm {T}}_{2,2}\mathbf {F}_{2}\mathbf {H}_{2,1}$ are the equivalent MIMO channels seen at source nodes S ₁ and S ₂, respectively. The vectors $\widetilde {\mathbf {n}}_{1}=\mathbf {H}^{\mathrm {T}}_{1,1}\mathbf {F}_{1}\mathbf {n}_{R_{1}}+\mathbf {H}^{\mathrm {T}}_{2,1}\mathbf {F}_{2}\mathbf {n}_{R_{2}}+\mathbf {n}_{1}$ and $\widetilde {\mathbf {n}}_{2}=\mathbf {H}^{\mathrm {T}}_{1,2}\mathbf {F}_{1}\mathbf {n}_{R_{1}}+ \mathbf {H}^{\mathrm {T}}_{2,2}\mathbf {F}_{2}\mathbf {n}_{R_{2}}+\mathbf {n}_{2}$ are the equivalent noises at source nodes S ₁ and S ₂, respectively.

Due to the lower computational complexity, linear receivers are applied at source node i to retrieve the transmitted signals sent from the other nodes. The estimated signal waveform vector is given as $\widehat {\mathbf {s}}_{\overline {i}}=\mathbf {W}^{\mathrm {H}}_{i}\widetilde {\mathbf {y}}_{i}$, where W _i is an M×M weight matrix, with $\overline {i}=2$ for i=1 and $\overline {i}=1$ for i=2. From (4), the MSE matrix of the signal waveform estimation is denoted by $\mathbf {MSE}_{\overline {i}}=\mathrm {E}\left [\left (\widehat {\mathbf {s}}_{i}-{\mathbf {s}}_{i}\right) \left (\widehat {\mathbf {s}}_{i}-{\mathbf {s}}_{i}\right)^{\mathrm {H}}\right ]$, which can be further written as

$$\begin{array}{@{}rcl@{}} \mathbf{MSE}_{i}&=&\left(\mathbf{W}^{\mathrm{H}}_{i}\widetilde{\mathbf{H}}_{i}-\mathrm{I}_{M}\right) \left(\mathbf{W}^{\mathrm{H}}_{i}\widetilde{\mathbf{H}}_{i}-\mathrm{I}_{M}\right)^{\mathrm{H}}\notag\\ &&+\mathbf{W}^{\mathrm{H}}_{i}\mathbf{C}_{n_{i}}\mathbf{W}_{i} \end{array} $$

((6))

where $\mathbf {C}_{n_{i}}=\mathbf {H}^{\mathrm {T}}_{i,i}\mathbf {F}_{i}\mathbf {F}^{\mathrm {H}}_{i} \mathbf {H}^{*}_{i,i}+\mathbf {H}^{\mathrm {T}}_{\overline {i},i}\mathbf {F}_{\overline {i}}\mathbf {F}^{\mathrm {H}}_{\overline {i}} \mathbf {H}^{*}_{\overline {i},i}+\mathbf {I}_{M}$ is the equivalent noise covariance. The sum-MSE of the two source nodes in the proposed system model can be written as

$$\begin{array}{@{}rcl@{}} \mathbf{MSE}_{\text{sum}}=\mathbf{MSE}_{1}+\mathbf{MSE}_{2}. \end{array} $$

((7))

Note that the sum-MSE minimization criterion measures the overall transmission performance of both the DL and the UL. Since the two data streams are transmitted at different directions during the two time slots, M S E _sum is considered in the TWR network.

3 Joint source and relay beamforming design

In this section, we develop an iterative QL-QR algorithm by using the MSE balancing result. The QL-QR algorithm involves two steps, i.e., the linear receiver matrix optimization and the joint source and relay beamformer design.

3.1 Proposed optimal detector and optimization problem

We would like to find the jointly optimal beamforming vectors W _i , V _i , and F _i such that the following sum-MSE is minimized:

$$\begin{array}{@{}rcl@{}} \min\limits_{\mathbf{W}_{i},\mathbf{F}_{i}, \mathbf{V}_{i}}~~\mathbf{MSE}_{\text{sum}}. \end{array} $$

((8))

According to (2), we consider the following individual transmission power constraint at relay node:

$$\begin{array}{@{}rcl@{}} \text{tr}{\left(\mathbf{F}_{i}\mathbf{D}_{i}\mathbf{F}^{\mathrm{H}}_{i}\right)}\leq P_{R_{i}}, \end{array} $$

((9))

where $\mathbf {D}_{i}=\mathbf {H}_{i,i}\mathbf {V}_{i}\mathbf {V}^{\mathrm {H}}_{i}\mathbf {H}^{\mathrm {H}}_{i,i}+ \mathbf {H}_{i,\overline {i}}\mathbf {V}_{\overline {i}}\mathbf {V}^{\mathrm {H}}_{\overline {i}}\mathbf {H}^{\mathrm {H}}_{i,\overline {i}}+\mathbf {I}_{N}$ and the $P_{R_{i}}$ denotes the individual power constraint at the relay node R _i . The transmission power constraint at two source nodes can be written as

$$\begin{array}{@{}rcl@{}} \text{tr}{\left(\mathbf{V}_{i}\mathbf{V}^{\mathrm{H}}_{i}\right)}\leq{P_{i}},~~~~~ i=1,2 \end{array} $$

((10))

where P _i is the available power at the ith source node. According to (8), (9), and (10), the joint optimization problem of the sum-MSE can be formulated as follows:

$$\begin{array}{@{}rcl@{}} &\min\limits_{\mathbf{W}_{i},\mathbf{F}_{i},\mathbf{V}_{i}}&~~~~~~~~~~~~~~~~~~~~~\mathbf{MSE}_{\text{sum}}\notag\\ &\mathrm{s.t.}&~\text{tr}{\left(\mathbf{F}_{i}\mathbf{D}_{i}\mathbf{F}^{\mathrm{H}}_{i}\right)}\leq P_{R_{i}},~\text{tr}{\left(\mathbf{V}_{i}\mathbf{V}^{\mathrm{H}}_{i}\right)}\leq{P_{i}}. \end{array} $$

((11))

It is shown in [20] that at the optimum, SNR₁=SNR₂ holds true, thus leading to an SNR balancing result. Otherwise, if SNR₁>SNR₂, then P ₂ can be reduced to retain SNR₁=SNR₂, and this reduction of P ₂ will not violate the power constraint, i.e.,

$$\begin{array}{@{}rcl@{}} P_{1}\cdot \text{SNR}_{1}=P_{2}\cdot \text{SNR}_{2}. \end{array} $$

((12))

In Fig. 2, we show two examples of the SNR regions with α ₁=0.5 and α ₂=0.3, where ω _i ∈[ 0,1] is a Lagrange multiplier weight value and α _i ∈[ 0,1] is an SNR weight value. We have assumed that the sum of SNR is a constant value. It is clear that the SNR region of α ₁ is larger than that of α ₂. For further details, see [20]. As discussed in [21], the optimization problems have the performance matrix that are functions of SNR, namely the MSE at the output of a linear-MMSE (LMMSE) filter of each user:

$$\begin{array}{@{}rcl@{}} \mathbf{MSE}=\frac{1}{1+\text{SNR}}. \end{array} $$

((13))

Fig. 2

Examples of the SNR regions achieved in a TWRC with two relays

By these two approaches, the max-min optimization problem in (11) can be efficiently written as

$$\begin{array}{@{}rcl@{}} &\min\limits_{\mathbf{W}_{1},\mathbf{F}_{i},\mathbf{V}_{2}}&~~~~~~~~~~~~~~~~~~~~~~~~~\mathbf{MSE}_{1} \end{array} $$

((14))

$$\begin{array}{@{}rcl@{}} &\mathrm{s.t.}&~\text{tr}{\left(\mathbf{F}_{i}\mathbf{D}_{i}\mathbf{F}^{\mathrm{H}}_{i}\right)}\leq P_{R_{i}},~\mathbf{MSE}_{1}=\mathbf{MSE}_{2}, \forall_{i}=1,2.\notag\\ \end{array} $$

((15))

Since the optimization problem (14)–(15) is nonconvex, it is difficult to obtain the globally optimal solution. In this paper, we present a locally optimal solution of the joint optimization problem over W _i , V _i , and F _i , where i=1,2, which can be solved by three stages, i.e., (1) the linear receiver weighted matrices are optimized with the fixed source precoding matrix V _i and relay amplifying matrices F _i (W _i is not in constraint (15)). (2) With given W _i and fixed F _i , update V _i . (3) With given W _i and V _i , obtain suboptimal F _i to solve (14).

Lemma 1.

For any fixed V _i and F _i , the minimization problems in (14) are convex quadratic problems and the optimal W _i can be obtained as the Wiener filter [22] which is used to decode s _i shown as follows:

$$\begin{array}{@{}rcl@{}} \mathbf{W}^{o}_{i}=\left(\mathbf{\widetilde{H}}_{i}\mathbf{\widetilde{H}}^{\mathrm{H}}_{i}+\mathbf{C}_{n_{i}}\right)^{-1}\mathbf{\widetilde{H}}_{i}, \end{array} $$

((16))

Proof.

For source node i, the MSE can be further expressed as

$$\begin{array}{@{}rcl@{}} \mathbf{MSE}_{i}&=&\mathbf{W}^{\mathrm{H}}_{i}\widetilde{\mathbf{H}}_{i}\widetilde{\mathbf{H}}^{\mathrm{H}}_{i}\mathbf{W}_{i}-\mathbf{W}^{\mathrm{H}}_{i}\widetilde{\mathbf{H}}_{i}-\widetilde{\mathbf{H}}^{\mathrm{H}}_{i}\mathbf{W}_{i}\notag\\ &&+\,\mathbf{I}_{M}+\mathbf{W}^{\mathrm{H}}_{i}\mathbf{C}_{n_{i}}\mathbf{W}_{i} \end{array} $$

((17))

Based on (17), the derivation of an optimal MSE detection matrix $\mathbf {W}^{\text {opt}}_{i}$ is equivalent to solving the following equation [23]:

$$\begin{array}{@{}rcl@{}} \frac{\partial\mathbf{MSE}_{i}}{\partial\mathbf{W}^{\mathrm{H}}_{i}}=2\widetilde{\mathbf{H}}_{i}\widetilde{\mathbf{H}}^{\mathrm{H}}_{i}\mathbf{W}_{i}-2\widetilde{\mathbf{H}}_{i}+2\mathbf{C}_{n_{i}}\mathbf{W}_{i}=0. \end{array} $$

((18))

Then, we may obtain the closed-form solution of W _i , which is

$$\begin{array}{@{}rcl@{}} \mathbf{W}^{o}_{i}=\left(\mathbf{\widetilde{H}}_{i}\mathbf{\widetilde{H}}^{\mathrm{H}}_{i}+\mathbf{C}_{n_{i}}\right)^{-1}\mathbf{\widetilde{H}}_{i}. \end{array} $$

((19))

This completes the proof.

With the optimal $\mathbf {W}^{o}_{1}$ fixed, the outer minimization problem in (14) can be rewritten as

$$\begin{array}{@{}rcl@{}} &\min\limits_{\mathbf{F}_{1},\mathbf{F}_{2},\mathbf{V}_{1},\mathbf{V}_{2}}&~~~~~~~~~~~~~~~~~~~~~~\mathbf{MSE}^{o}_{1}\notag\\ &\mathrm{s.t.}&~\text{tr}{\left(\mathbf{F}_{i}\mathbf{D}_{i}\mathbf{F}^{\mathrm{H}}_{i}\right)}\leq P_{R_{i}},~\mathbf{MSE}^{o}_{1}=\mathbf{MSE}^{o}_{2}, \end{array} $$

((20))

where $\mathbf {MSE}^{o}_{i}$ is the MSE matrix using $\mathbf {W}^{o}_{i}$. By substituting (16) into (6), we have

$$\begin{array}{@{}rcl@{}} \mathbf{MSE}^{o}_{1}=\left[\mathbf{I}_{M}+\mathbf{\widetilde{H}}^{\mathrm{H}}_{1}\mathbf{C}^{-1}_{n_{1}}\mathbf{\widetilde{H}}_{1}\right]^{-1},~~~ i=1,2. \end{array} $$

((21))

Note that the matrix inversion lemma is used to obtain (21).

3.2 Joint optimal source and relay beamforming matrices design and iterative algorithm

In this section, we focus on the source and relay beamforming matrices design and develop an iterative algorithm which is suboptimal for the general case but has a much lower computational complexity. For the fixed F _i , the source precoding matrix V _i is optimized by solving the following problem:

$$\begin{array}{@{}rcl@{}} &\min\limits_{\mathbf{V}_{i}}&~~~~~~~~~~~~~\text{tr}\left[\mathbf{I}_{M}+\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}\right]^{-1}\notag\\ &\mathrm{s.t.}&~~~\text{tr}\left\{\textbf{V}^{\mathrm{H}}_{i}\mathbf{\Psi}_{i}\textbf{V}_{i}\right\}\leq{P_{R_{i}}},~\text{tr}\left\{\mathbf{V}^{\mathrm{H}}_{i}\mathbf{V}_{i}\right\}\leq P_{i}, \end{array} $$

((22))

where $\mathbf {\Phi }_{i}=\mathbf {\widetilde {H}}^{\mathrm {H}}_{i} \mathbf {\widehat {C}}^{-1}_{n_{i}} \mathbf {\widetilde {H}}_{i}$, $\mathbf {\widehat {C}}^{-1}_{n_{i}}=\left (\mathbf {\widetilde {n}}_{i}\right)^{\mathrm {H}} \mathbf {\widetilde {n}}_{i}$, $\mathbf {\Psi }_{i}=\mathbf {H}^{\mathrm {H}}_{i,i}\mathbf {F}^{\mathrm {H}}_{i}$ $\mathbf {F}_{i} \mathbf {H}_{i,i}+\mathbf {H}^{\mathrm {H}}_{\overline {i},i}\mathbf {F}^{\mathrm {H}}_{\overline {i}}\mathbf {F}_{\overline {i}}\mathbf {H}_{\overline {i},i}$.

The Lagrangian function associated with the problem (22) is given by

$${} \begin{aligned} L_{\mathbf{V}}&=\text{tr}\left[\mathbf{I}_{M}+\mathbf{V}^{\mathrm{H}}_{\overline{i}} \mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}\right]^{-1}+\mu_{1}\left(\text{tr}\left\{\mathbf{V}^{\mathrm{H}}_{i}\mathbf{V}_{i}\right\}-P_{i}\right) \\&\quad+\mu_{2}\left(\text{tr}\left\{\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{V}_{\overline{i}}\right\}-P_{\overline{i}}\right) +\,\mu_{3}\left\{\text{tr}\left\{\textbf{V}^{\mathrm{H}}_{i}\mathbf{\Psi}_{i}\textbf{V}_{i}\right\}-P_{R_{i}}\right\} \\&\quad +\mu_{4}\left\{\text{tr}\left\{\textbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Psi}_{\overline{i}}\textbf{V}_{\overline{i}}\right\}-P_{R_{\overline{i}}}\right\}, \end{aligned} $$

((23))

where μ _i ≥0 is the Lagrange multiplier.

Case 1: When μ _i =0, making the derivative of L _V with respect to $\mathbf {V}_{\overline {i}}$ be zero, we obtain

$$\begin{array}{@{}rcl@{}} \frac{\partial L_{\mathbf{V}}}{\partial \mathbf{V}_{\overline{i}}}=-\left[\mathbf{I}_{M}+ \mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}\right]^{-2} \mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Phi}_{i}=0. \end{array} $$

((24))

Since $\mathbf {V}_{\overline {i}}$ and Φ _i are nonsingular matrices, (24) can be represented as

$$\begin{array}{@{}rcl@{}} \mathbf{I}_{M}+\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}=0. \end{array} $$

((25))

Simplifying (25), I _M >0 and $\mathbf {V}^{\mathrm {H}}_{\overline {i}}\mathbf {\Phi }_{i}\mathbf {V}_{\overline {i}}\geq 0$. Consequently, in Case 1, the optimal solution is not existent.

Case 2: When μ _i >0, we rewrite the Lagrangian function as

$${} \begin{aligned} L_{\mathbf{V}}&=\left[\mathbf{I}_{M}+\mathbf{V}^{\mathrm{H}}_{\overline{i}} \mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}\right]^{-1}-\mu_{1}P_{i}-\mu_{2} P_{\overline{i}}-\mu_{3}P_{R_{i}}-\mu_{4}P_{R_{\overline{i}}}\\ &\quad+ \mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Upsilon}_{\overline{i}} \mathbf{\Upsilon}_{\overline{i}}^{\mathrm{H}}\mathbf{V}_{\overline{i}}+\mu_{3} \mathbf{V}_{i}^{\mathrm{H}}\mathbf{\Psi}_{i}\mathbf{V}_{i}+\mu_{1}\mathbf{V}_{i}^{\mathrm{H}}\mathbf{V}_{i}, \end{aligned} $$

((26))

where $\mathbf {\Upsilon }_{\overline {i}}\mathbf {\Upsilon }_{\overline {i}}^{\mathrm {H}}=\mu _{2} \mathbf {I}_{M}+\mu _{4}\mathbf {\Psi }_{\overline {i}}$. We obtain the derivative of L _V as

$$ \frac{\partial L_{\mathbf{V}}}{\partial \mathbf{V}_{\overline{i}}}=-\left[\mathbf{I}_{M}+ \mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}\right]^{-2} \mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Phi}_{i}+\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Upsilon}_{\overline{i}}\mathbf{\Upsilon}_{\overline{i}}^{\mathrm{H}}=0. $$

((27))

Since $\mathbf {V}^{\mathrm {H}}_{\overline {i}}$ and Φ _i are nonsingular matrices, multiply both sides by $\left (\mathbf {V}^{\mathrm {H}}_{\overline {i}}\right)^{-1}$ and $\mathbf {\Phi }_{i}^{-1}$; we have

$$\begin{array}{@{}rcl@{}} \left(\mathbf{V}^{\mathrm{H}}_{\overline{i}}\right)^{-1}\left[\mathbf{I}_{M}+\mathbf{V}^{\mathrm{H}}_{\overline{i}} \mathbf{\Phi}_{i}\mathbf{V}_{\overline{i}}\right]^{-2}\mathbf{V}^{\mathrm{H}}_{\overline{i}}= \mathbf{\Upsilon}_{\overline{i}}\mathbf{\Upsilon}_{\overline{i}}^{\mathrm{H}}\mathbf{\Phi}_{i}^{-1}. \end{array} $$

((28))

Since Φ _i is Hermitian and positive definite, we apply the Choleskey decomposition of $\mathbf {\Phi }_{i}=\mathbf {\Omega }_{i}^{\mathrm {H}}\mathbf {\Omega }_{i}$, where Ω _i is a lower triangular matrix. Consequently, we represent (28) as

$$\begin{array}{@{}rcl@{}} &&\left(\mathbf{\Omega}_{i}^{\mathrm{H}}\right)^{-1}\left(\mathbf{V}^{\mathrm{H}}_{\overline{i}}\right)^{-1}\left[\mathbf{I}_{M}+\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Omega}_{i}^{\mathrm{H}}\mathbf{\Omega}_{i}\mathbf{V}_{\overline{i}}\right]^{-2}\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Omega}_{i}^{\mathrm{H}}\notag\\ &=&\left(\mathbf{\Omega}_{i}^{\mathrm{H}}\right)^{-1}\mathbf{\Upsilon}_{\overline{i}}\mathbf{\Upsilon}_{\overline{i}}^{\mathrm{H}}\left(\mathbf{\Omega}_{i}^{\mathrm{H}}\mathbf{\Omega}_{i}\right)^{-1}\mathbf{\Omega}_{i}^{\mathrm{H}}. \end{array} $$

((29))

By the definition of the matrix identity as

$$\begin{array}{@{}rcl@{}} \left[\mathbf{I}_{M}+\mathbf{X}\mathbf{X}^{\mathrm{H}}\right]^{-1}\mathbf{X}=\mathbf{X}\left[\mathbf{I}_{N}+\mathbf{X}^{\mathrm{H}}\mathbf{X}\right]^{-1}, \end{array} $$

((30))

for any M×N matrix X, we can rewrite (29) as

$$\begin{array}{@{}rcl@{}} \left[\mathbf{I}_{M}+\mathbf{\Omega}_{i}\mathbf{V}_{\overline{i}}\mathbf{V}^{\mathrm{H}}_{\overline{i}}\mathbf{\Omega}_{i}^{\mathrm{H}}\right]^{-2}=\left(\mathbf{\Omega}_{i}^{\mathrm{H}}\right)^{-1}\mathbf{\Upsilon}_{\overline{i}}\mathbf{\Upsilon}_{\overline{i}}^{\mathrm{H}}\mathbf{\Omega}_{i}^{-1}. \end{array} $$

((31))

Solving (31) for $\mathbf {V}_{\overline {i}}$, we have

$$\begin{array}{@{}rcl@{}} \mathbf{V}_{\overline{i}}=\left(\nabla_{i}{\nabla_{i}^{\mathrm{H}}}-\mathbf{\Phi}_{i}^{-1}\right)^{\frac{1}{2}}, \end{array} $$

((32))

where $\nabla _{i}=\left (\mathbf {\Upsilon }_{\overline {i}}^{\mathrm {H}}\mathbf {\Omega }_{i}\right)^{-\frac {1}{2}}$.

Figure 3 shows our proposed relay filter design, which forwards the received signal (input) from S ₁ amplified by a left filter (LF) matrix F _L,i and the signal from S ₂ amplified by a right filter (RF) matrix F _R,i to a center filter (CF) F _D,i . F _D,i amplifies the outputs from the LF matrix F _L,i and the RF matrix F _R,i (i∈{1,2} denotes relay node i) and forwards them to S ₁ and S ₂ (output).¹

Fig. 3

The relay filter design of the proposed QL-QR technique

Lemma 2.

The optimal relay filter constructive of F _L , F _R , and F _D matrices, i.e., for R ₁ and R ₂, can be designed as

$$\begin{array}{@{}rcl@{}} &&\mathbf{F}_{L,1}=\mathbf{Q}^{*}_{L,1},~\mathbf{F}_{L,2}=\mathbf{Q}^{*}_{L,2},\notag\\ &&\mathbf{F}_{R,1}=\mathbf{Q}^{\mathrm{H}}_{R,1},~\mathbf{F}_{R,2}=\mathbf{Q}^{\mathrm{H}}_{R,2},\notag\\ &&\mathbf{F}_{D,1}~\text{and}~ \mathbf{F}_{D,2}~\mathrm{are~ diagonal~ matrices},\notag \end{array} $$

where Q _L,i and Q _R,i for i=1,2 are the unitary matrices which relate to QL and QR decompositions for the dependent channel coefficients.

Proof.

For F _L,i and F _R,i , the proof is similar to Theorem 3.1 in [24]. For F _D,i , using Theorem 2 in [9], the structure of F _D,i is optimal for the two cases of

$$\begin{array}{@{}rcl@{}} &&\mathrm{(a)}:~~~~~R(\mathbf{H}_{1}\mathbf{V}_{1})\bot R(\mathbf{H}_{2}\mathbf{V}_{2}); (\mathbf{H}^{*}_{1})\bot R(\mathbf{H}^{*}_{2})\notag\\ && ~~\text{and}~\, ~~~R(\mathbf{H}_{3}\mathbf{V}_{1})\bot R(\mathbf{H}_{4}\mathbf{V}_{2}); (\mathbf{H}^{*}_{3})\bot R(\mathbf{H}^{*}_{4})\notag\\ &&\mathrm{(b)}:~~~~~R(\mathbf{H}_{1}\mathbf{V}_{1})\| R(\mathbf{H}_{2}\mathbf{V}_{2}); (\mathbf{H}^{*}_{1})\bot R(\mathbf{H}^{*}_{2})\notag\\ && ~~\text{and}~\, ~~~R(\mathbf{H}_{1}\mathbf{V}_{1})\| R(\mathbf{H}_{2}\mathbf{V}_{2}); (\mathbf{H}^{*}_{1})\bot R(\mathbf{H}^{*}_{2}). \end{array} $$

((33))

Case a: If N=2, the optimal F _D,i is a diagonal matrix given as

$$\begin{array}{@{}rcl@{}} \mathbf{F}_{D,i} &\triangleq& { \left[\begin{array}{cc} \mathbf{f}_{d,i,1} & \mathbf{0} \\ \mathbf{0} & \mathbf{f}_{d,i,2} \\ \end{array}\right]}. \end{array} $$

((34))

If N=2a, a=2,3,…, the optimal F _D,i is a 2×2 block diagonal matrix given as

$$\begin{array}{@{}rcl@{}} \mathbf{F}_{D,i} &\triangleq & {\left[ \begin{array}{cc} \mathbf{F}_{D,i,1} & \mathbf{0} \\ \mathbf{0} & \mathbf{F}_{D,i,2} \\ \end{array}\right]}, \end{array} $$

((35))

where F _D,i,1 and F _D,i,2 are $\frac {N}{2}\times \frac {N}{2}$ matrices.

Case b: The optimal F _D,i is defined as

$$\begin{array}{@{}rcl@{}} \mathbf{F}^{\star}_{D,i} &\triangleq& {\left[ \begin{array}{cc} \mathbf{f}_{d,i,1} \\ \mathbf{f}_{d,i,2} \\ \end{array}\right]}. \end{array} $$

((36))

Discussion 1: In Case b, since $\mathbf {F}^{\star }_{D,i}$ is optimal, but the computational complexity will be considerably increased compared with Case a, so we exclude it.

For Case a, Before we develop a numerical method to solve vector F _D,i , let us have some insights into the structure of this suboptimal relay beamforming matrix. To simplify relay beamforming matrix F _D,i , we introduce the following properties:

Property 1: The statistical behavior of a unitary matrix U remains unchanged when multiplied by any unitary matrix T independent of U. In other words, T U has the same distribution as U, i.e., in (33),

$$\begin{array}{@{}rcl@{}} &&|\mathbf{F}_{1}|=|\mathbf{F}_{L,1}\mathbf{F}_{D,1}\mathbf{F}_{R,1}|=|\mathbf{F}_{D,1}|\notag\\ &&|\mathbf{F}_{2}|=|\mathbf{F}_{L,2}\mathbf{F}_{D,2}\mathbf{F}_{R,2}|=|\mathbf{F}_{D,2}|. \end{array} $$

((37))

Now, let us introduce the following Q L decomposition:

$$\begin{array}{@{}rcl@{}} \left[\mathbf{H}_{1,1}\mathbf{V}_{1}, \mathbf{H}_{2,1}\mathbf{V}_{1}\right]&=&\left[\mathbf{Q}_{L,1}\mathbf{L}_{1}, \mathbf{Q}_{L,2}\mathbf{L}_{2} \right], \end{array} $$

((38))

where Q _L,i for i=1,2 is a unitary matrix with a dimension ${\mathbb {C}^{N\times {N}}}$ and $\left \{\mathbf {L}_{1}, \mathbf {L}_{2}\right \}\in {\mathbb {C}^{N\times {M}}}$ are lower triangular matrices.

Similarly, let us introduce another decomposition, namely Q R decomposition, as

$$\begin{array}{@{}rcl@{}} \left[\mathbf{H}_{1,2}\mathbf{V}_{2}, \mathbf{H}_{2,2}\mathbf{V}_{2}\right]&= &\left[\mathbf{Q}_{R,1}\mathbf{R}_{1},\mathbf{Q}_{R,2}\mathbf{R}_{2} \right], \end{array} $$

((39))

where $\mathbf {Q}_{R,i}\in {\mathbb {C}^{N\times {M}}}$ for i=1,2 is a unitary matrix and $\left \{\mathbf {R}_{1}, \mathbf {R}_{2}\right \} \in {\mathbb {C}^{M\times {M}}}$ are upper triangular matrices. Substituting (38) and (39) back into (4), we can get equivalent received signals shown as

$$\begin{array}{@{}rcl@{}} \widehat{\mathbf{y}}_{i}&=&\left(\mathbf{L}^{\mathrm{T}}_{i}\mathbf{F}_{D,i}\mathbf{R}_{i}+\mathbf{L}^{\mathrm{T}}_{\overline{i}}\mathbf{F}_{D,\overline{i}}\mathbf{R}_{\overline{i}}\right)\mathbf{x}_{\overline{i}}\notag\\ &&+\,\mathbf{L}^{\mathrm{T}}_{i}\mathbf{F}_{D,i}\mathbf{n}_{R_{i}}+\mathbf{L}^{\mathrm{T}}_{\overline{i}}\mathbf{F}_{D,\overline{i}}\mathbf{n}_{R_{\overline{i}}}+\mathbf{n}_{\overline{i}}\notag\\ &=&\mathbf{\widehat{H}}_{i,\overline{i}}\mathbf{x}_{\overline{i}}+\widehat{\mathbf{n}}_{i}, \end{array} $$

((40))

where $\mathbf {\widehat {H}}_{i}=\mathbf {L}^{\mathrm {T}}_{i}\mathbf {F}_{D,i} \mathbf {R}_{i}+\mathbf {L}^{\mathrm {T}}_{\overline {i}}\mathbf {F}_{D,\overline {i}}\mathbf {R}_{\overline {i}}$ and $\widehat {\mathbf {n}}_{i}=\mathbf {L}^{\mathrm {T}}_{i}\mathbf {F}_{D,i}\mathbf {n}_{R_{i}}+ \mathbf {L}^{\mathrm {T}}_{\overline {i}}\mathbf {F}_{D,\overline {i}}\mathbf {n}_{R_{\overline {i}}}+\mathbf {n}_{\overline {i}}$ are efficient channel and noise coefficients; obtained from the covariance of $\mathbf {\widehat {C}}_{i}$, we have

$$\begin{array}{@{}rcl@{}} \mathbf{\widehat{C}}_{i}&=&\widehat{\mathbf{n}}_{i}\widehat{\mathbf{n}}^{\mathrm{H}}_{i}\notag\\ &=&\mathbf{L}^{\mathrm{T}}_{i}\mathbf{F}_{D,i}\mathbf{F}^{\mathrm{H}}_{D,i}\mathbf{L}^{*}_{i}+\mathbf{L}^{\mathrm{T}}_{\overline{i}}\mathbf{F}_{D,\overline{i}}\mathbf{F}^{\mathrm{H}}_{D,\overline{i}}\mathbf{L}^{*}_{\overline{i}}+\mathbf{I}_{N}. \end{array} $$

((41))

For fixed V _i , using (40) and Property 1, the optimal problem (20) becomes

$$\begin{array}{@{}rcl@{}} &\max\limits_{\mathbf{F}_{D,1}, \mathbf{F}_{D,2}}&~~~~~~~~~~~~\text{tr}\left(\mathbf{I}_{N}+\mathbf{\widehat{H}}^{\mathrm{H}}_{i} \mathbf{\widehat{C}}_{i}^{-1}\mathbf{\widehat{H}}_{i}\right) \end{array} $$

((42))

$$\begin{array}{@{}rcl@{}} &\mathrm{s.t.}&~~\text{tr}{\left(\textbf{F}^{\mathrm{H}}_{D,i}\mathbf{D}_{i}\textbf{F}_{D,i}\right)}\leq{P_{R_{i}}}, \mathbf{MSE}_{1}=\mathbf{MSE}_{2}, \end{array} $$

((43))

where we have employed the principle min(a)=max (a ⁻¹), for a≠0. By using the lemma tr(A + B)=tr(A)+tr(B), (42) can be represented as

$$\begin{array}{@{}rcl@{}} \text{tr}\left(\mathbf{\widehat{H}}^{\mathrm{H}}_{i}\mathbf{\widehat{C}}_{i}^{-1}\mathbf{\widehat{H}}_{i}\right)+n. \end{array} $$

((44))

Since the matrix $\mathbf {\widehat {C}}_{i}$ is Hermitian and positive definite, we can decompose this matrix using the Cholesky factorization as

$$\begin{array}{@{}rcl@{}} \mathbf{\widehat{C}}_{i}&=&\mathbf{\Xi}^{\mathrm{H}}_{i}\mathbf{\Xi}_{i} \end{array} $$

((45))

where Ξ _i denotes a lower triangular matrix. By substituting (45) back into (44), we can simply rewrite the optimal problem as

$$\begin{array}{@{}rcl@{}} &&\text{max}~\left(\mathbf{MSE}_{i}\right)^{-1}\notag\\ &=&\text{max}~\text{tr}\left(\mathbf{\widehat{H}}^{\mathrm{H}}_{i}\left(\mathbf{\Xi}^{\mathrm{H}}_{i}\mathbf{\Xi}_{i}\right)^{-1}\mathbf{\widehat{H}}_{i}\right)+n\notag\\ &\overset{a}{=}&\text{max}~\text{tr}\left(\left(\mathbf{\widehat{H}}^{\mathrm{H}}_{i}\mathbf{\Xi}^{-1}_{i}\right)\left(\mathbf{\widehat{H}}^{\mathrm{H}}_{i}\mathbf{\Xi}^{-1}_{i}\right)^{\mathrm{H}}\right)\notag\\ &=&\text{max}~\text{tr}\left(\mathbf{B}_{i}\mathbf{B}^{\mathrm{H}}_{i}\right), \end{array} $$

((46))

where $\overset {a}{=}$ denotes n has nothing to do with the maximum solution and $\mathbf {B}_{i}=\mathbf {\widehat {H}}^{\mathrm {H}}_{i}\mathbf {\Xi }^{-1}_{i}$. Therefore, the optimal problem can be represented as the determinant maximization of $\mathbf {B}_{i}^{2}$ [29].

In Case a, since F _D,i is the block diagonal matrix, its determinant can be written as

$$\begin{array}{@{}rcl@{}} \text{det}{\mathbf{F}_{D,i}}=\text{det}{\mathbf{F}_{D,i,1}}\cdot\text{det}{\mathbf{F}_{D,i,2}}. \end{array} $$

((47))

Let A, B, C, and D be an $\frac {N}{4}\times \frac {N}{4}$ matrix. We can define detF _D,i,i , for i∈1,2, as

$$\begin{array}{@{}rcl@{}} \text{det}{\mathbf{F}_{D,i,i}} &=& {\left| \begin{array}{cc} \mathbf{A} & \mathbf{D} \\ \mathbf{B} & \mathbf{C} \\ \end{array}\right|}\notag\\ &=& {\left|\begin{array}{cc} \mathbf{A} & \mathbf{0} \\ \mathbf{B} & \mathbf{I} \\ \end{array}\right|} {\left|\begin{array}{cc} \mathbf{I} & \mathbf{A}^{-1}\mathbf{D} \\ \mathbf{0} & \mathbf{C}-\mathbf{BA^{-1}D} \\ \end{array}\right|}\notag\\ &=& |\mathbf{A}|\left|\mathbf{C}-\mathbf{BA^{-1}D}\right|, \end{array} $$

((48))

where I stands for an $\frac {N}{4}\times \frac {N}{4}$ identity matrix. In (48), to obtain maximum detF _D,i,i , we should minimize BA ⁻¹ D. Let us introduce the SVD of B, A, and D as

$$\begin{array}{@{}rcl@{}} \mathbf{B}=\mathbf{U}_{B}\mathbf{\Sigma}_{B}\mathbf{\Lambda}^{\mathrm{H}}_{B},~ \mathbf{A}=\mathbf{U}_{A}\mathbf{\Sigma}_{A}\mathbf{\Lambda}^{\mathrm{H}}_{A},~ \mathbf{D}=\mathbf{U}_{D}\mathbf{\Sigma}_{D}\mathbf{\Lambda}^{\mathrm{H}}_{D}, \end{array} $$

((49))

where U _i , Λ _i , i∈{A,B,D}, are the unitary matrices and Σ _i is an $\frac {N}{4}\times \frac {N}{4}$ diagonal matrix. Substituting (49) back into BA ⁻¹ D, we have

$$\begin{array}{@{}rcl@{}} &&\min~\text{tr}\left(\mathbf{U}_{B}\mathbf{\Sigma}_{B}\mathbf{\Lambda}^{\mathrm{H}}_{B}\left(\mathbf{U}_{A} \mathbf{\Sigma}_{A}\mathbf{\Lambda}^{\mathrm{H}}_{A}\right)^{-1}\mathbf{U}_{D}\mathbf{\Sigma}_{D}\mathbf{\Lambda}^{\mathrm{H}}_{D}\right)\notag\\ &\triangleq&\min~\text{tr}\left(\mathbf{\Sigma}_{B}\left(\mathbf{\Sigma}_{A}\right)^{-1}\mathbf{\Sigma}_{D}\right)\notag\\ &\triangleq&\min_{b_{i},d_{i}}~\sum^{n/4}_{i}\frac{b_{i}d_{i}}{a_{i}}, \end{array} $$

((50))

where b _i , d _i , and a _i are the diagonal elements of Σ _B , Σ _D , and Σ _A , respectively. To simplify our discussion, we assume that F _D,i,i is a semi-positive matrix; thus, we have the minimum solution as b _i d _i =0. Interestingly, if both b _i and d _i are 0, F _D,i,i is a diagonal matrix. Otherwise, it is a lower/upper triangular matrix. In addition, for S ₁, the equivalent channel $\widehat {\mathbf {H}}_{1}$, since the terms $\mathbf {L}^{\mathrm {T}}_{1}$, $\mathbf {L}^{\mathrm {T}}_{2}$, R ₁, and R ₂ are upper triangular matrices, the optimal F _D,i should be an upper triangular matrix. Since the equivalent channel $\widehat {\mathbf {H}}_{2}$, L ₁, L ₂, $\mathbf {R}^{\mathrm {T}}_{1}$, and $\mathbf {R}^{\mathrm {T}}_{2}$ are lower triangular matrices for S ₂, the optimal F _D,i is a lower triangular matrix. Therefore, if and only if F _D,i is a diagonal matrix, the sum-MSE is optimal in our proposed method.

This completes the proof for Lemma 2.

Property 2: For any M×N rectangular matrices G and J, matrices A and B are lower/upper triangular matrices based on QR or QL decomposition of G and J. If a _i,i +b _i,i ≠0, where a _i,i and b _i,i are diagonal elements of matrices A and B, respectively, we can easily obtain

$$\begin{array}{@{}rcl@{}} \text{det}\left(\mathbf{A}+\mathbf{B}\right)=\prod^{m}_{i=1}(a_{i,i}+b_{i,i}) \geq\text{det}\mathbf{A}+\text{det}\mathbf{B}. \end{array} $$

((51))

Consequently, we have

$$\begin{array}{@{}rcl@{}} \text{det}\mathbf{\widehat{H}}^{\mathrm{H}}_{i}&=&\prod^{M}_{m=1} \left(l_{i,m,m}f_{D,i,m}r_{i,m,m}+l_{\overline{i},m,m}f_{D,\overline{i},m}r_{\overline{i},m,m}\right)\notag\\ &=&\prod^{M}_{m=1}(\varsigma_{i}+\varsigma_{\overline{i}}), \end{array} $$

((52))

where ς ₁=l _i,m,m f _D,i,m r _i,m,m , $\varsigma _{\overline {i}}=l_{\overline {i},m,m}f_{D,\overline {i},m}r_{\overline {i},m,m}$, l _i,m,m , f _D,i,m , r _i,m,m , $l_{\overline {i},m,m}$, $f_{D,\overline {i},m}$, and $r_{\overline {i},m,m}$ are diagonal elements of L _i , F _D,i , R _i , $\mathbf {L}_{\overline {i}}$, $\mathbf {F}_{D,\overline {i}}$, and $\mathbf {R}_{\overline {i}}$, respectively. Since Ξ _i and $\mathbf {\Xi }_{i}^{-1}$ are also lower triangular matrices, we have

$$\begin{array}{@{}rcl@{}} \text{det}\mathbf{B}_{i}&=&\text{det}\mathbf{\widehat{H}}^{\mathrm{H}}_{i}\text{det}\mathbf{\Xi}_{i}^{-1}\notag\\ &=&\prod^{M}_{m=1}(\varsigma_{i}+\varsigma_{\overline{i}})\xi_{m}, \end{array} $$

((53))

where ξ _m is the diagonal element of $\mathbf {\Xi }^{-1}_{i}$.

After introducing slack the variable τ _i , the objective problem can be equivalently converted into the optimization one with respect to an individual relay power constraint, shown as follows:

$$\begin{array}{@{}rcl@{}} &\max\limits_{\mathbf{F}_{D,i}}&~~~~~~~~~~~~~~~~~~~~~~~~~~~~\tau_{i} \end{array} $$

((54))

$$\begin{array}{@{}rcl@{}} &\mathrm{s.t.}&{\left(\begin{array}{cc} \tau_{i} & \text{det}\left(\mathbf{\Xi}^{-1}_{i}\widehat{\mathbf{H}}^{\mathrm{H}}_{i}\right) \\ \text{det}\left(\widehat{\mathbf{H}}_{i}\left(\mathbf{\Xi}^{-1}_{i}\right)^{\mathrm{H}}\right) & 1 \\ \end{array}\right)}\preceq0 \end{array} $$

((55))

$$\begin{array}{@{}rcl@{}} &&\text{tr}\left(\textbf{F}_{D,i}\mathbf{D}_{i}\textbf{F}^{\mathrm{H}}_{D,i}\right)\leq P_{R_{i}},\text{det}\mathbf{B}^{2}_{i}\geq \tau_{i}, \tau_{i}\geq0, \forall_{i}=1,2.\\ \end{array} $$

((56))

From (54) to (56), it is easy to see that τ _i is only dependent on the beamformer F _D,t and with respect to (55) and (56). Thus, by using (53), we have

$${} {\fontsize{9.2pt}{9.6pt}\selectfont{\begin{aligned} \tau_{i}\leq\prod^{M}_{m=1}\Re\left(l_{i,m,m}\,f_{D,i,m}r_{i,m,m}+l_{\overline{i},m,m}\,f_{D,\overline{i},m}r_{\overline{i},m,m}\right)^{2}\times\Re\left(\xi_{m}\right)^{2}. \end{aligned}}} $$

((57))

From (57), it is easy to follow that with fixed F _D,i /$\mathbf {F}_{D,\overline {i}}$, τ _i is convex with regard to $\mathbf {F}_{D,\overline {i}}$/ F _D,i . In summary, we outline the iterative beamforming design algorithm as follows (Q L−Q R A l g o r i t h m):

Clearly, Algorithm 1 will converge to a sub-optimal solution as $\tau ^{(n)}_{\text {max}}-\tau ^{(n-1)}_{\text {max}}\leq \epsilon$. Therefore, ε is initialized to be a small value and N _max is set to limit the number of iterations.

Discussion 2: Figure 4 displays two extended system models. One is the multi-pair scenario with two relay nodes and K pairs of source nodes. The other is a Z relay node scenario with two source nodes. In Fig. 4 a, each pair of sources and two relay nodes can be seen as a group. Since each pair of source nodes are independent of one another, we can design each relay node comprised of K RFs, LFs, and one CF. Therefore, the extended system model (a) can be classified as another version of our proposed system model with K parallel nodes. The objective problem can be expressed as

$$\begin{array}{@{}rcl@{}} &\max\limits_{\mathbf{W}_{t},\mathbf{F}_{t},\mathbf{V}_{t}}&~~~~~~~~~~~~~~~~~~\mathbf{MSE}^{\mathrm{(a)}}_{\text{sum}}\notag\\ &\mathrm{s.t.}&~\text{tr}{\left(\textbf{F}_{t}\textbf{D}_{t}\textbf{F}^{\mathrm{H}}_{t}\right)}\leq{P_{R_{t}}}, ~\text{tr}{\left(\mathbf{V}_{i}\mathbf{V}^{\mathrm{H}}_{i}\right)}\leq{P_{i}}, \end{array} $$

((58))

Fig. 4

The extended system model. a The K pairs of the source node scenario. b The T relay node scenario

where $\mathbf {MSE}^{(a)}_{\text {sum}}=\sum _{i=1}^{K}\text {det}\left (\mathbf {B}_{i}\right)^{2}+ \sum _{j=K+1}^{2K}\text {det}\left (\mathbf {B}_{j}\right)^{2}$ is the sum-MSE of the multi-pair scenario. It is clear that (58) is a bi-convex problem which is similar to (54)–(56) with different effective channel coefficients and can be solved by our proposed algorithm.

In Fig. 1 b, a TWRC consisting of two source nodes and L relay nodes is considered. Obviously, by employing the MSE balancing result, the objective of the extended system model (b) is to minimize the sum-MSE, which is subject to the individual relay transmit power constraint, shown as

$${} {\fontsize{9.4pt}{9.6pt}\selectfont{\begin{aligned} &\min\limits_{\mathbf{W}_{t},\mathbf{F}_{i},\mathbf{V}_{t}}~~~~~~~~~~~~~~~~~~~~~~~~~~\mathbf{MSE}_{t}^{\mathrm{(b)}}\\ &\mathrm{s.t.}~~~~~\text{tr}{\left(\mathbf{F}_{i}\textbf{D}_{i}\textbf{F}^{\mathrm{H}}_{i}\right)}\leq{P_{R_{i}}}, ~\text{tr}{\left(\mathbf{V}_{t}\mathbf{V}^{\mathrm{H}}_{t}\right)}\leq{P_{t}},~ \mathbf{MSE}_{1}^{\mathrm{(b)}}=\mathbf{MSE}_{2}^{\mathrm{(b)}}. \end{aligned}}} $$

((59))

where $\mathbf {MSE}_{t}=\left (\mathbf {W}^{\mathrm {H}}_{i}\widetilde {\mathbf {H}}_{t}-\mathrm {I}_{M}\right) \left (\mathbf {W}^{\mathrm {H}}_{i}\widetilde {\mathbf {H}}_{t}-\mathrm {I}_{M}\right)^{\mathrm {H}}+\mathbf {W}^{\mathrm {H}}_{i}\mathbf {C}_{n_{t}}$ W _i , for t=1,2, $\widetilde {\mathbf {H}}_{t}=\sum _{i=1}^{L}\mathbf {H}^{\mathrm {T}}_{t,i} \mathbf {F}_{i}\mathbf {H}_{\overline {t},i}$, and $\mathbf {C}_{n_{t}}=\sum _{i=1}^{L} \mathbf {H}^{\mathrm {T}}_{t,i}\mathbf {F}_{i}\mathbf {F}^{\mathrm {H}}_{i}\mathbf {H}^{*}_{t,i}+\mathbf {I}_{M}$, is the equivalent noise covariance, for i=1,…,L. Due to the semi-infinite constraints at the relay node, the objective problem (59) is non-convex. In this scenario, we consider a two-stage solution where in the first step, the semi-infinite constraints are converted to linear matrix inequalities (LMI) and in the second step, we use our proposed iterative algorithm to solve it. By using the S-Lemma [28], the relay power constraint can be converted into the LMI version, we have

$${} \begin{aligned} {\left[\begin{array}{cccc} \sum_{i=1}^{l}P_{R_{i}}-\sum_{i=1}^{l}\phi_{i} & -\text{vec}\left(\mathbf{F}_{1}\right)^{\mathrm{H}} & \cdots & -\text{vec}\left(\mathbf{F}_{L}\right)^{\mathrm{H}}\\ -\text{vec}\left(\mathbf{F}_{1}\right) & \phi_{1}\mathbf{I}_{N^{2}} & \cdots & \mathbf{0}\\ \vdots & \vdots & \ddots & \vdots\\ -\text{vec}\left(\mathbf{F}_{L}\right) & \mathbf{0} & \cdots & \phi_{L}\mathbf{I}_{N^{2}}\\ \end{array}\right]} \succeq0, \end{aligned} $$

((60))

where vec(·) denotes to stack the columns of a matrix into a single vector. Now, the objective problem (59) becomes

$$\begin{array}{@{}rcl@{}} &\min\limits_{\mathbf{W}_{t},\mathbf{F}_{i},\mathbf{V}_{t}}&~~~~~~~~~~~~~~\mathbf{MSE}_{t}^{\mathrm{(b)}}\notag\\ &\mathrm{s.t.}&~(60), ~\text{tr}{\left(\mathbf{V}_{t}\mathbf{V}^{\mathrm{H}}_{t}\right)}\leq{P_{t}},~\phi_{i}\geq0. \end{array} $$

((61))

In step 2, we use an iterative algorithm based on alternating convex search (ACS) to solve the resulting convex problem. The algorithm is almost the same as our proposed one which only converts [Algorithm 1, Steps 4–11] into “for fixed $\mathbf {W}^{(n)}_{i}$, $\mathbf {V}^{(n)}_{i}$ update $\mathbf {F}_{i}^{(n)}$ via solving (61)”.

4 Computational complexity analysis

In this section, we measure the performance of the proposed QL-QR scheme in terms of the computational complexity compared with existing algorithms by using the total number of floating point operations (FLOPs). A flop is defined as a real floating operation, i.e., a real addition, multiplication, and division. In [30], the authors show the computational complexity of the real Choleskey decomposition. For complex numbers, a multiplication followed by an addition needs eight FLOPs, which leads to four times its real computation. According to [31], the required number of FLOPs of each matrix is described as follows:

• 1. Multiplication of m×n and n×p complex matrices: 8mnp−2mp;

• 2. Multiplication of m×n and n×m complex matrices: 4nm×(m+1);

• 3. SVD of an m×n(m≤n) complex matrix where only Σ is obtained: 32(mn ²−n ³/3);

• 4. SVD of an m×n(m≤n) complex matrix where only Σ and Λ are obtained: 32(nm ²+2m ³);

• 5. SVD of an m×n(m≤n) complex matrix where U, Σ, and Λ are obtained: 8(4n ² m+8nm ²+9m ³);

• 6. Inversion of an m×m real matrix using Gauss-Jordan elimination: 2m ³−2m ²+m;

• 7. Cholesky factorization of an m×m complex matrix: 8m ³/3;

• 8. QR or QL decomposition of an m×n conplex matrix $16\left (n^{2}m-nm^{2}+\frac {1}{3}m^{3}\right)$.

For the RBD method [32], the authors consider a linear MU-MIMO precoding scheme for DL MIMO systems. For the non-regenerative MIMO relay systems [33], the authors investigate a precoding design for a three-node MIMO relay network. In [2], a relay-aided system based on a quasi-EVD channel is proposed. We compare the required number of FOLPs of our proposed method with conventional precoding algorithms, such as the RBD, the non-regenerative MIMO relay system, and the CD-BD algorithm as shown in Tables 1, 2, 3, and 4, respectively, under the assumption that N _T =N _R and $\overline {N}_{i}=N_{T}-N_{i}$.

Table 1 Computational complexity of the proposed QL-QR algorithm

Table 2 Computational complexity of the non-regenerative MIMO relay system [33]

Table 3 Computational complexity of the conventional RBD [32]

Table 4 Computational complexity of the CD-BD algorithm [2]

For instance, the (2,2,2)×6 case denotes a system with three users (K=3), where each user is equipped with two antennas (N _i =2) and the total number of transmit antennas is six (N _T =2×3=6). The required number of FLOPs of the QL-QR algorithm, the RBD, the non-regenerative MIMO relay system, and the CD-BD algorithm are counted as 33,530, 40,824, 45,306, and 34,638, respectively. From these results, we can see that the reduction in the number of FLOPs of our proposed precoding method is 17.87, 25.99, and 3.20 % on an individual basis compared to the RBD, the non-regenerative MIMO relay systems, and the CD-BD algorithm. Thus, our proposed QL-QR algorithm exhibits lower complexity than conventional algorithms. In addition, the complexity reduces as N _i and N _T increase with fixed K.

We summarize our calculation results of the required number of FLOPs of the alternative methods in Tables 1, 2, 3, and 4 and show them in Figs. 5 and 6. Figure 5 shows the computational complexity where N _i =2 and a value of K varies. And Fig. 6 shows the computational complexity where K=4 and a value of N _i varies. For the RBD method, the orthogonal complementary vector V _k,0 requires K times SVD operations. If only V _k,0 is obtained, it is not computationally efficient. In step 5, the efficient channel $\mathbf {H}_{\text {eff}}=\mathbf {H}_{i}\mathbf {P}^{a}_{i}$ is decomposed by the SVD with a dimension R _eff×N _T , where R _eff is the rank of H _eff. In the non-regenerative MIMO relay method and the CD-BD algorithm, two SVD operations are performed for the channels from the source to the relay and from the relay to the destination, and then the efficient channel covariance matrix is measured. In the non-regenerative MIMO relay method, the authors compute A using the EVD, and then they diagonalize G. In the CD-BD algorithm, the authors calculate $\mathbf {V}^{a}_{i}$ by the SVD of $\mathbf {H}^{\dag }_{\text {mse}}$, and then they structure $\mathbf {V}^{b}_{i}$ by using the Choleskey decomposition.

Fig. 5

The complexity comparisons for required FLOPs versus the number of the users K

Fig. 6

The complexity comparisons for required FLOPs versus the number of the receive antennas N _i for each user

In our proposed QL-QR algorithm, we take advantage of QL and QR decompositions instead of the SVD operation, and then we compute an efficient channel as well as decompose a noise covariance matrix by the Choleskey decomposition. Finally, we calculate the determinant of $\mathbf {B}^{2}_{i}$ to solve an optimization problem. Obviously, our proposed QL-QR algorithm outperforms conventional algorithms in the light of the computational complexity.

5 Simulation results

In this section, we study the performance of the proposed QL-QR algorithm for two-way MIMO relay networks. All the simulations are performed on the assumption that all the channel estimates are the Rayleigh fading channels, and they are independent and identically distributed (i.i.d.) complex Gaussian random variables. The noise variances ${\sigma ^{2}_{i}}$ are equally given as σ ². All the simulation results are averaged over 1000 channel trials.

In Fig. 7, we compare the sum mutual information (SMI) of various MU-MIMO schemes where full CSI is known at each node. We set P ₁=P ₂=10 dB and M=1 with an equal power budget for the two relays. The negative SMI is adopted in [16] which can be defined as

$$\begin{array}{@{}rcl@{}} \mathbf{MI}_{\text{sum}}=\text{log}_{2}\left|\mathbf{MSE}_{1}\right|+\text{log}_{2}\left|\mathbf{MSE}_{2}\right|. \end{array} $$

((62))

Fig. 7

The achieved SMI for N=4,2

In our proposed method, the SMI shown in the simulation results is calculated as $-2~\text {log}_{2}|\mathbf {B}^{2}_{i}|$ by using (45), (52), and (53). It can be observed that the proposed QL-QR algorithm has the same SMI performance as an optimal solution in [16].

Figure 8 shows the performance of our proposed SMI performance versus the number of the relays, T, which is even. We consider a practical scenario with different relay power constraints and set P _R =30 dB. It is clear that, for different values of P ₁ and P ₂, a solution of our proposed QL-QR algorithm shows better performance than a max-power solution.

Fig. 8

The SMI versus the number of the relays T

Figure 9 a, b exhibits the BER performance of the BD water filling, the RBD, the SVD-RBD, and our proposed QL-QR method, where the quadrature phase shift keying (QPSK) and 16 quadrature amplitude modulation (16-QAM) are made use of. As pointed out in [35], the BER performance for a MIMO precoding system is actually determined by the energy of the transmitted signal. To simplify our discussion, we assume a=0. In the RBD, $\text {det}\left (\overline {\mathbf {H}} \overline {\mathbf {H}}^{\mathrm {H}}\right)= \prod ^{m}_{i=1}{\lambda ^{2}_{i}}$, where $\overline {\mathbf {H}}\in {\mathbb {C}^{N\times M}}$, for M<N, is an equivalent channel matrix with its eigenvalues λ _i . In our proposed QL-QR method, for source node S ₁, we have $\text {det}\left (\mathbf {\widehat {H}}_{1}\mathbf {\widehat {H}}^{\mathrm {H}}_{1}\right)=\prod ^{m}_{i=1}{\varsigma ^{2}_{1}}$. Under the stipulation that detF _D,i =1, we are able to easily obtain λ _i =ς _i . Therefore, our proposed QL-QR method has the same BER performance as that of the SVD-RBD method.

Fig. 9

a BER performance on the Rayleigh fading channel with QPSK. b BER performance on the Rayleigh fading channel with 16-QAM

6 Conclusions

This paper studies a joint optimization problem of an AF based on the MIMO TWRC, where two source nodes exchange their messages with two relay nodes. A relay filter is designed, which is able to efficiently join the source and the relay nodes. Our main contribution is that the optimal beamforming vectors can efficiently be computed using determinant maximization techniques through an iterative QL-QR algorithm based on a MSE balancing method. Our proposed QL-QR algorithm can significantly reduce the computational complexity and has an equivalent BER performance to that of the SVD-BD algorithm.

7 Endnote

¹For example: For S ₁, the equivalent channel can be written as $\widetilde {\mathbf {H}}_{1}=\mathbf {H}^{\mathrm {T}}_{1,1}\mathbf {F}_{1} \mathbf {H}_{1,2}+\mathbf {H}^{\mathrm {T}}_{2,1}\mathbf {F}_{2}\mathbf {H}_{2,2}=\mathbf {H}^{\mathrm {T}}_{1,1}\mathbf {F}_{L,1} \mathbf {F}_{D,1}\mathbf {F}_{R,1}\mathbf {H}_{1,2}+\mathbf {H}^{\mathrm {T}}_{2,1}\mathbf {F}_{L,2}\mathbf {F}_{D,2}\mathbf {F}_{R,2}\mathbf {H}_{2,2}$. For S ₂, the equivalent channel can be written as $\widetilde {\mathbf {H}}_{2}= \mathbf {H}^{\mathrm {T}}_{1,2}\mathbf {F}_{1}\mathbf {H}_{1,1}+\mathbf {H}^{\mathrm {T}}_{2,2}\mathbf {F}_{2} \mathbf {H}_{2,1}=\mathbf {H}^{\mathrm {T}}_{1,2}\mathbf {F}_{R,1}\mathbf {F}_{D,1}\mathbf {F}_{L,1} \mathbf {H}_{1,1}+\mathbf {H}^{\mathrm {T}}_{2,1}\mathbf {F}_{R,2}\mathbf {F}_{D,2}\mathbf {F}_{L,2}\mathbf {H}_{2,2}$.