Performance of MIMO systems in measured indoor channels with transmitter noise

Abstract

This study analyzes the impact of transmitter noise on the performance of multiple-input multiple-output (MIMO) systems with linear and nonlinear receivers and precoders. We show that the performance of MIMO linear and decision-feedback receivers is not significantly influenced by the presence of transmitter noise, which does not hold true in the case of MIMO systems with precoding. Nevertheless, we also show that this degradation can be greatly alleviated when the transmitter noise is considered in the MIMO precoder design. A MIMO testbed developed at the University of A Coruña has been employed for experimentally evaluating how much the transmitter noise impacts the system performance. Both the transmitter noise and the receiver noise covariance matrices have been estimated from a set of 260 indoor MIMO channel realizations. The impact of transmitter noise has been assessed in this realistic scenario.

1 Introduction

Practical implementations of wireless transmitters suffer from a large number of impairments such as quantization noise, sampling offset, phase noise, I/Q imbalance ... which can be classified into systematic and non-systematic effects. These impairments are normally ignored when multiple-input multiple-output (MIMO) signaling methods are designed. However, noise generated at the transmitter can significantly affect predicted performance in practical scenarios. For instance, the performance of a linear zero-forcing (ZF) MIMO detector affected by the transmitter impairments of a MIMO orthogonal frequency-division multiplexing (OFDM) hardware demonstrator is tested in [1], where it is shown how the performance achieved by such noisy systems suffers from a loss greater than 4dB for a bit error rate (BER) of 10^-2. More specifically, the impact of residual transmitter radio-frequency (RF) impairments on both MIMO channel capacity and receiver performance is analyzed in [2]. It has been demonstrated that maximum-likelihood (ML) and Max-log a posteriori probability (APP) MIMO detection suffer from a substantial performance loss under the presence of weak transmitter noise, whereas linear ZF receivers are much less affected.

In this study, the impact of residual non-deterministic, non-systematic transmit impairments on MIMO signaling methods not considered in [1, 2] is studied. First of all, we focus on MIMO systems with either linear receivers [3–5] or linear precoders [4–8]. We will also focus on MIMO systems with nonlinear decision feedback (DF) receivers [9, 10] and nonlinear Tomlinson-Harashima (TH) precoders [5, 11, 12]. Both DF receivers and TH precoders are widely used because of their good trade-off between performance and complexity. The modulo operator at the receiver of a MIMO system with TH precoding has also motivated the proposal of a more general MIMO precoding technique where the data signal superimposed with a perturbation signal inputs a linear filter at the transmitter. This scheme is referred to as vector precoding (VP) in the literature [5, 13]. The optimum perturbation signal is found with a closest point search in a lattice. Despite its larger complexity, VP outperforms TH precoding.

Although the impact of transmitter noise has been evaluated over spatially-white Rayleigh channels in [14, 15] and some preliminary results obtained from testbed measurements in an indoor scenario have been presented in [15, 16], only our study evaluates the performance of all the aforementioned schemes affected by transmitter noise over measured indoor channels by means of the testbed developed by the University of A Coruña [17]. Contrary to previous studies, not only the channel coefficients are measured using such a testbed, but also a estimate of all the noise covariance matrices existing in practical MIMO systems and of the signal-to-transmitter noise ratio (STxNR) are obtained and plugged into the robust filter expressions to guarantee a complete reproducibility of a real indoor environment and the evaluation in terms of BER performance of the proposed schemes under such realistic conditions.

In this study, we show that noise generated at the transmitter significantly affects the performance of MIMO systems with precoding. On the other hand, the performance of both linear and DF MIMO receivers is more robust against the presence of transmitter noise (cf. findings in the FP6-IST project MASCOT on multiuser MIMO communication systems [18]). At a first glance, it seems natural that MIMO systems with precoding be more sensitive to transmitter noise since channel equalization is carried out by processing signals prior transmission. However, it will be also shown that the performance of precoded MIMO systems can be substantially improved if the transmitter noise is considered inside the precoder design.

This study is organized as follows. Section 2 introduces the MIMO signal model which takes into account the aforementioned transmit RF impairments. The design of MIMO linear receivers and precoders considering the presence of transmitter noise is addressed in Section 3, whereas Section 4 focuses on MIMO nonlinear receivers (decision feedback) and precoders (Tomlinson-Harashima and vector precoding). Section 5 briefly introduces the MIMO testbed used to obtain measurements of the noise and of the channel parameters necessary to reproduce a realistic, indoor scenario. Next, the performance of the MIMO systems analyzed in the previous sections is evaluated for this indoor scenario. Some concluding remarks are presented in Section 6. Finally, Appendix 1 details all derivations corresponding to the expressions of all linear and nonlinear filters derived including transmitter noise in the optimizations.

Vectors and matrices are denoted by lower-case bold, and capital-bold letters, respectively. We use E[•], tr(•), (•)^T, (•)^H, and ∥ • ∥₂ for expectation, trace of a matrix, transposition, conjugate transposition, and Euclidean norm, respectively. The i th element of a vector v is denoted by v _i .

2 Signal model with transmitter noise

Let us consider a narrowband MIMO communication system with N_t transmit and N_r receive antennas. When considering only the receiver noise, this system can be represented by the following discrete-time model

$$\mathbf{y}\left[n\right]=\mathbf{Hx}\left[n\right]+{\mathit{\eta }}_r\left[n\right]\in {ℂ}^{N_{\text{r}}},$$

(1)

where $\mathbf{x}\left[n\right]\in {ℂ}^{N_{\text{t}}}$ represents the transmitted signals, ${\mathit{\eta }}_r\left[n\right]\in {ℂ}^{N_{\text{r}}}$ is the noise vector introduced by the receivers (which will be referred to as the Rx-noise), $\mathbf{H}\in {ℂ}^{N_{\text{r}}\times N_{\text{t}}}$ is the MIMO channel matrix, and $\mathbf{y}\left[n\right]\in {ℂ}^{N_{\text{r}}}$ is the received signal vector. Note that we assume a block-fading channel, where H remains constant during the transmission of a data frame. Rx-noise is complex-valued, Gaussian-distributed with zero mean and covariance matrix $C_{{\mathit{\eta }}_r}=E\left[{\mathit{\eta }}_{\text{r}}\left[n\right]{\mathit{\eta }}_{\text{r}}^H\left[n\right]\right]$, i.e., ${\mathit{\eta }}_r\left[n\right]~{\mathcal{N}}_{ℂ}\left(0,C_{{\mathit{\eta }}_r}\right)$. Transmit energy is constrained to a value

$$E_{\mathrm{tx}}=\text{tr}\left({\mathit{C}}_{\mathit{x}}\right),$$

(2)

where C _x is the covariance matrix corresponding to the input symbols x[n]. Accordingly, we define the signal-to-receiver-noise ratio (SRxNR) as

$$\text{SRxNR}=\frac{E_{\mathrm{tx}}{N}_{\text{r}}}{\text{tr}\left(C_{{\mathit{\eta }}_{\text{r}}}\right)},$$

(3)

where we have assumed that the channel is normalized to have a mean Frobenius norm equal to N_tN_r.

When the residual impairments at the transmitter are taken into account, a more accurate model for the transmitted signal is

$${\mathit{x}}_t\left[n\right]=\mathit{x}\left[n\right]+{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{t}}},$$

(4)

where ${\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{t}}}$ will be denoted as the Tx-noise. The subscript _t will be used hereafter to denote signals affected by Tx-noise. This noise encompasses different practical effects, for example phase-noise. As explained in [2], Tx-noise is adequately modeled as an additive Gaussian noise since it results from the sum of a large number of residual transmit impairments. Tx-noise is assumed to be zero-mean with covariance matrix ${\mathit{C}}_{{\mathit{\eta }}_t}$ and the STxNR is defined as

$$\text{STxNR}=\frac{E_{\mathrm{tx}}}{\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_t}\right)},$$

(5)

where E _tx is fixed and given by Equation (2). As an example, practical implementations of the IEEE 802.11 (WiFi) standard achieve STxNR values ranging from 22 dB to 32 dB (see [2] and references therein). The Tx-noise is also assumed to be statistically independent from the Rx-noise. Note that we use the STxNR instead of the transmitter error vector magnitude (EVM), which is another measure of the signal modulation quality (see [19]) that additionally considers systematic errors.

Equations (1) and (4) are also adequate to represent a multi-antenna, multiuser wireless system where a base station equipped with N antennas communicates with K single-antenna users. When considering the uplink, N_t = K and N_r = N, whereas N_t = N and N_r = K for the downlink. Although the channel model is the same, multiuser systems impose certain constraints on the signal processing that can be carried out to recover the transmitted information [20]. Indeed, for the uplink, the base station collects the signals from all the users, so that they can be separated using conventional MIMO receiver methods. These methods, however, cannot be used for the downlink since users do not normally cooperate among them and, consequently, each receiver does not know the signals from the other receivers. Therefore, in such a case signal separation can only be carried out by resorting to precoding or to any other form of transmit processing.

3 MIMO linear receivers and precoders

Figure 1 shows the block diagram of a general MIMO system with linear transmit and receive processing. Notice that transmitter noise has been also included. We assume the transmitter sends a set of N_r data streams encoded into the vector of transmitted symbols $\mathit{u}\left[n\right]\in A^{N_{\text{r}}}$ where $A$ represents the utilized modulation alphabet. For linear transmit and receive processing, one transmit filter, $\mathit{F}\in {ℂ}^{N_{\text{t}}\times N_{\text{r}}}$, so-called precoding filter, and one receive filter, $\mathit{G}\in {ℂ}^{N_{\text{r}}\times N_{\text{r}}}$, are used to recover the transmitted data. Thus, after linear precoding, the transmitted signal is

Figure 1

MIMO linear transmit processing and linear receive processing with Tx-noise.

$${\mathit{x}}_t\left[n\right]=\mathit{Fu}\left[n\right]+{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_t},$$

(6)

and at the other end, the signal at the output of the receive filter G is given by

$${\widehat{\mathit{u}}}_t\left[n\right]=\mathit{GH}{\mathit{x}}_t\left[n\right]+\mathit{G}{\mathit{\eta }}_r\left[n\right]\in {ℂ}^{N_{\text{r}}}.$$

(7)

This signal is the input to a symbol detector (represented by Q(•) in Figure 1) that maps û _t [n] onto the modulation alphabet $A$ and produces the estimates of the transmitted symbols, ũ[n].

3.1 MIMO linear receiver design with transmitter noise

Let us consider a simplified MIMO linear transmission system where the transmit filter is a weighted identity matrix, i.e., F= p I. Thus, all the effort to compensate for the channel spatial inter-symbol interference (ISI) is carried out by the receiver filter G(see [3, 5, 21]). Elaborating on the signal model, the symbols at the output of the receiver filter can be written as

$${\widehat{\mathit{u}}}_t\left[n\right]=\widehat{\mathit{u}}\left[n\right]+\mathit{GH}{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{t}}},$$

(8)

where $\widehat{\mathit{u}}\left[n\right]=p\mathit{GHu}\left[n\right]+\mathit{G}{\mathit{\eta }}_r\left[n\right]$ would be the received signal if there were no Tx-noise.

Next, let us define the error vector ${\epsilon }_t\left[n\right]=\mathit{u}\left[n\right]-{\widehat{\mathit{u}}}_t\left[n\right]$. The optimum receive filter G and transmit weight p will be those that minimize the mean square error (MSE) $\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]$ which, under the presence of Tx-noise, can be expressed as follows [3]

$$\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]=\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]+\text{tr}\left(\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right).$$

(9)

Notice that $\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]$ is the cost function to be minimized in conventional minimum mean square error (MMSE) linear receiver design where the transmitter noise is not accounted for.

Similarly to the study in [5], it can be demonstrated that by restricting p ∈ ℝ⁺ and having in mind the transmit-energy constraint E _tx = tr(C _x ), the minimization of the cost function given by Equation (9) produces the MMSE solution for the linear receiver given by

$$\begin{array}{ll}\hfill {\mathit{G}}_{\text{MMSE}}& ={\left(\frac{{\mathit{C}}_{\mathit{u}}^{-1}}{p_{\text{MMSE}}}+p_{\text{MMSE}}{\mathit{H}}^{\text{H}}{\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)}^{-1}\mathit{H}\right)}^{-1}{\mathit{H}}^{\text{H}}{\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)}^{-1},\\ \hfill p_{\text{MMSE}}& =\sqrt{\frac{E_{\text{tx}}}{\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)}},\end{array}$$

(10)

where C _u is the covariance matrix of the transmitted symbols u[n].

From the MMSE design of the MIMO linear receiver, it is straightforward to obtain the expressions for the ZF MIMO linear receiver [5]: it is the limiting case when $\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^H\right)/E_{\text{tx}}\to 0$, i.e.,

$$\begin{array}{ll}\hfill {\mathit{G}}_{\text{ZF}}& ={\left(p_{\text{ZF}}{\mathit{H}}^H{\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_r}{\mathit{H}}^{\text{H}}\right)}^{-1}\mathit{H}\right)}^{-1}{\mathit{H}}^H{\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)}^{-1},\\ \hfill p_{\text{ZF}}& =\sqrt{\frac{E_{\text{tx}}}{\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)}}.\end{array}$$

(11)

Note that a necessary condition for the existence of the filter G_ZF is that N_t ≤ N_r since it exists only if ${\mathit{H}}^H{\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)}^{-1}\mathit{H}$ is invertible.

3.2 MIMO linear precoder design with transmitter noise

Let us consider the MIMO transmission scheme dual to the previous one: the transmitted symbols $\mathit{u}\left[n\right]\in {ℂ}^{N_{\text{r}}}$ are linearly precoded with the transmit filter $\mathit{F}\in {ℂ}^{N_{\text{t}}\times N_{\text{r}}}$ and the receive filter is a weighted identity matrix G= g I, where g ∈ ℝ⁺. Thus, the transmitted signal is

$${\mathit{x}}_t\left[n\right]=\mathit{Fu}\left[n\right]+{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{t}},}$$

(12)

and the signal at the receiver input is given by

$${\mathit{y}}_t\left[n\right]=\mathit{y}\left[n\right]+\mathit{H}{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{r}}},$$

(13)

where $\mathit{y}\left[n\right]=\mathit{HFu}\left[n\right]+{\mathit{\eta }}_r\left[n\right]$ would be the received signal if there were no Tx-noise. After multiplication by the receive gain, g, we get the estimated symbols

$${\widehat{\mathit{u}}}_t\left[n\right]=\widehat{\mathit{u}}\left[n\right]+g\mathit{H}{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{r}}},$$

(14)

where û[n] = g y [n]. We restrict the scalar value g to being common to all the receivers to simplify the optimization procedure.

The optimum transmit and receive filters are those that minimize the MSE between u[n] and û _t [n] subject to the transmit-energy constraint E _tx = tr(C _x ) (see [3, 5–7]). Again, let us define the error vector ϵ _t [n] = u[n] - û _t [n] and express the MSE cost function as

$$\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]=\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]+\text{tr}\left({\left|g\right|}^2\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right),$$

(15)

where $\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]$ is the MSE when there is no Tx-noise [4, 5, 8].

Minimizing this cost function produces the MMSE solution for the MIMO linear precoding filter given by

$$\begin{array}{ll}\hfill {\mathit{F}}_{\text{MMSE}}& =g_{\text{MMSE}}^{-1}{\left({\mathit{H}}^{\text{H}}\mathit{H}+{\xi }_t\mathbf{I}\right)}^{-1}{\mathit{H}}^{\text{H}},\\ \hfill g_{\text{MMSE}}& =\sqrt{\frac{\text{tr}\left({\left({\mathit{H}}^H\mathit{H}+{\xi }_t\mathbf{I}\right)}^{-2}{\mathit{H}}^H{\mathit{C}}_{\mathit{u}}\mathit{H}\right)}{E_{\text{tx}}}},\end{array}$$

(16)

where ${\xi }_t=\xi +\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)/E_{\text{tx}}$, with ξ being the inverse of the SRxNR defined in Equation (5).

By applying the matrix inversion lemma to the MMSE solution of Equation (16) and considering afterwards the limiting case when ξ _t → 0, the expressions for the ZF linear precoder can be readily obtained and expressed as follows

$$\begin{array}{ll}\hfill {\mathit{F}}_{\text{ZF}}& =g_{\text{ZF}}^{-1}{\mathit{H}}^H{\left(\mathit{H}{\mathit{H}}^H\right)}^{-1},\\ \hfill g_{\text{ZF}}& =\sqrt{\frac{\text{tr}\left({\left({\mathit{H}}^{\text{H}}\mathit{H}\right)}^{-1}{\mathit{C}}_{\mathit{u}}\right)}{E_{\text{tx}}}}.\end{array}$$

(17)

4 MIMO nonlinear receivers and precoders

In this section, we focus on MIMO systems that use either nonlinear transmitters or nonlinear receivers to recover the transmitted data. It is well known that maximum likelihood detection (MLD) is the optimum detection scheme in the sense of minimizing the probability of a symbol being erroneously detected. The computational complexity of MLD grows exponentially with the number of transmit antennas and the modulation alphabet size and, for this reason, suboptimum nonlinear detection schemes such as the decision feedback one are preferred in a large number of practical situations.

However, decision-feedback receivers suffer from the major drawback of error propagation caused by feeding back erroneous decisions. One way to avoid this harmful effect is to perform a nonlinear filtering similar to that in DF but at the transmitter side. This idea leads to the concept of Tomlinson-Harashima precoding (THP). Finally, VP is another form of nonlinear MIMO transmit processing which will be considered in this section. Similarly to MLD, VP consists in a lattice search carried out at the transmitter instead of at the receiver side.

The impact of transmitter noise on the performance of MLD has already been analyzed in [2]. In the following sections, we will derive the expressions of the filters for the remaining nonlinear transceivers when the Tx-noise is accounted for.

4.1 MIMO decision feedback receiver design with transmitter noise

Figure 2 plots the block diagram of a MIMO system with a DF receiver. Information symbols will be represented by $\mathit{u}\left[n\right]\in A^{N_t}$, where $A$ denotes the modulation alphabet, which are directly sent by the transmit antennas. It is assumed that u[n] is zero mean with covariance matrix denoted by C _u .

Figure 2

MIMO linear transmit processing and nonlinear receive processing (DF) with Tx-noise.

It is apparent from Equations (1) and (4) that the input signal at the receiver can be written as

$${\mathit{y}}_t\left[n\right]=\mathit{y}\left[n\right]+\mathit{H}{\mathit{\eta }}_t\left[n\right]\in {ℂ}^{N_{\text{r}}},$$

(18)

where $\mathit{y}\left[n\right]=\mathit{Hu}\left[n\right]+{\mathit{\eta }}_r\left[n\right]$ is the received signal if there is no Tx-noise [5].

In DF reception, the signals at the channel output are passed through the feedforward filter $\mathit{G}\in {ℂ}^{N_{\text{t}}\times N_{\text{r}}}$, which forces the ISI to be spatially causal and the error to be spatially white (i.e., minimum variance). By means of the feedback filter $\mathit{I}-\mathit{B}\in {ℂ}^{N_{\text{t}}\times N_{\text{t}}}$ and of the feedback loop shown in Figure 2, ISI can be recursively canceled without changing the statistical properties of the noise provided that the noise variance is sufficiently small so that the symbol detector (represented by Q(●) in Figure 2) produces correctly detected symbols. By elaborating on the signal model according to Figure 2, the estimated signal û _t [n] can be written as

$${\widehat{\mathit{u}}}_t\left[n\right]=\mathit{G}{\mathit{y}}_t\left[n\right]+\left(\mathit{I}-\mathit{B}\right)\tilde{\mathit{u}}\left[n\right],$$

(19)

where y _t [n] is defined as in Equation (18) and $\tilde{\mathit{u}}\left[n\right]\in A^{N_{\text{t}}}$ denotes the detected symbols after the threshold quantizer.

The order in which symbols are detected has a significant influence on the performance of DF MIMO receivers. In the system model shown in Figure 2, the ordering is obtained with the multiplication of the detected symbols, ũ[n], by the permutation matrix P^T. This multiplication produces ũ _p [n], which constitutes the vector of detected symbols conveniently sorted. Having in mind that PP^T = I, we have that ũ[n] = Pũ _p [n] and hence, û _t [n] can be rewritten as

$${\widehat{\mathit{u}}}_t\left[n\right]=\mathit{G}{\mathit{y}}_t\left[n\right]+\left(\mathit{I}-\mathit{B}\right)\mathit{P}{\tilde{\mathit{u}}}_p\left[n\right].$$

The MMSE design of the DF MIMO receiver searches for the filtering and permutation matrices that minimize the variance of the error vector

$${\epsilon }_{t,p}\left[n\right]=\mathit{Pu}\left[n\right]-{\widehat{\mathit{u}}}_t\left[n\right].$$

Assuming correct decisions (i.e., ũ _p [n] = u[n]) and according to Equation (18), this error vector can be rewritten as

$${\epsilon }_{t,p}\left[n\right]=\mathit{BPu}\left[n\right]-\mathit{G}{\mathit{y}}_t\left[n\right]={\epsilon }_p\left[n\right]-\mathit{GH}{\mathit{\eta }}_t\left[n\right],$$

where ε _p [n] = BPu[n] - Gy[n] is the error vector when there is no Tx-noise. Since the Tx-noise is independent from the Rx-noise and the transmitted signals, the MSE cost function to be minimized can be written as

$$\text{E}\left[{∥{\epsilon }_{t,p}\left[n\right]∥}_2^2\right]=\text{E}\left[{∥{\epsilon }_p\left[n\right]∥}_2^2\right]+\text{tr}\left(\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right),$$

(20)

where $\text{E}\left[{∥{\epsilon }_p\left[n\right]∥}_2^2\right]$ is the MSE with no Tx-noise. Notice that $\text{E}\left[{∥{\epsilon }_p\left[n\right]∥}_2^2\right]$ is the cost function that is minimized in the conventional MMSE design, whereas the additional term $\text{tr}\left(\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)$ is the MSE improvement caused by the inclusion of the Tx-noise.

An MMSE design of the MIMO link that accounts for the Tx-noise should minimize the MSE given by Equation (20). Similarly to the scenario without Tx-noise [22], minimization of Equation (20) is readily accomplished from the Cholesky factorization with symmetric permutation of

$${\mathbf{\Phi }}_t={\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right)}^{-1}.$$

This factorization is given by PΦ _t P^T= LDL^H, where L is a unit lower triangular matrix and D is a diagonal matrix. After this decomposition, it can be demonstrated that the filters G and B for the MMSE DF nonlinear MIMO receiver solution are

$$\begin{array}{ll}\hfill {\mathit{G}}_{\text{MMSE}}^{\text{DF}}& =\mathit{D}{\mathit{L}}^{\text{H}}\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1},\\ \hfill {\mathit{B}}_{\text{MMSE}}^{\text{DF}}& ={\mathit{L}}^{-1}.\end{array}$$

(21)

The minimum value of the MSE cost function is obtained plugging ${\mathit{G}}_{\text{MMSE}}^{\text{DF}}$ and ${\mathit{B}}_{\text{MMSE}}^{\text{DF}}$ into Equation (20). Hence, the MMSE value is

$$\text{MMS}{\text{E}}_{t,\text{DF}}=\text{tr}\left(\mathit{D}\right),$$

(22)

where D is the diagonal matrix obtained from the Cholesky factorization with symmetric permutation of Φ _t .

Notice that the MMSE expression given by Equation (22) depends on the permutation matrix P. Brute force optimization of P can be carried out by computing the MMSE for all the N_t! possible permutation matrices and choosing the one that provides the minimum value of Equation (22). Alternatively, more efficient ordering algorithms (such as the one described in [22]) can be used.

From the MMSE design of the DF receiver, it is straightforward to obtain the expressions for the ZF DF receiver: it is the limiting case when $\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)/E_{\text{tx}}\to 0$. The final expressions for the ZF DF filters are exactly the same as before although L and D should be obtained from the Cholesky decomposition of

$${\mathbf{\Phi }}_t={\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}\right)}^{-1}.$$

4.2 MIMO Tomlinson-Harashima precoder design with transmitter noise

Figure 3 shows the block diagram of a MIMO system employing THP. THP is a nonlinear precoding technique made up of a feedforward filter $\mathit{F}\in {ℂ}^{N_{\text{t}}\times N_{\text{r}}}$, a feedbackward filter $\mathit{I}-\mathit{B}\in {ℂ}^{N_{\text{r}}\times N_{\text{r}}}$, and a modulo operator, represented in Figure 3 by M(●). The modulo operator is introduced to avoid the increase in transmit power due to the feedback loop [11]. Data symbols sent from the transmitter will be represented by $\mathit{u}\left[n\right]\in A^{N_{\text{r}}}$, where $A$ denotes the modulation alphabet. The ordering considerably affects the performance of THP and, for this reason, transmit symbols are passed through a permutation filter P. Minimization is carried out under the restriction of B being a spatially causal filter and E_tx being the transmitted energy, i.e., $\text{E}\left[{∥\mathit{x}\left[n\right]∥}_2^2\right]=E_{\text{tx}}$, where $\mathit{x}\left[n\right]=\mathit{Fv}\left[n\right]\in {ℂ}^{N_{\text{t}}}$ is the transmitted signal, with v[n] representing the output of the modulo operator. At reception, we assume that all the receive antennas apply the same positive real value denoted by g. These assumptions are necessary in order to arrive at closed-form, unique solutions for the MMSE THP design.

Figure 3

Nonlinear MIMO system with THP and Tx-noise.

In order to carry out the THP optimization, and taking into account the linear representation of THP [5, 11], the desired signal is denoted by d[n] and it is expressed as

$$\mathit{d}\left[n\right]={\mathit{P}}^{\text{T}}\mathit{Bv}\left[n\right].$$

(23)

The received signal under the presence of Tx-noise is rewritten as

$${\widehat{\mathit{d}}}_{\text{t}}\left[n\right]=\widehat{\mathit{d}}\left[n\right]+g\mathit{H}{\mathit{\eta }}_{\text{t}}\left[n\right]\in {ℂ}^{N_{\text{r}}},$$

(24)

where $\widehat{\mathit{d}}\left[n\right]=g\mathit{HFv}\left[n\right]+g{\mathit{\eta }}_{\text{r}}\left[n\right]$ is the received signal when there is no Tx-noise. At the receivers, the modulo operator is applied again to invert the effect of this operator at the transmitter and the resulting signal is passed through a symbol detector (represented by Q(●) in Figure 3) to produce the detected symbols $\tilde{\mathit{u}}\left[n\right]\in A^{N_{\text{r}}}$.

As explained in [5], the MMSE THP design searches for the filtering and permutation matrices that minimize the variance of the error vector

$${\epsilon }_{\text{t}}\left[n\right]={\mathit{P}}^T\mathit{Bv}\left[n\right]-{\widehat{\mathit{d}}}_{\text{t}}\left[n\right]=\epsilon \left[n\right]-g\mathit{H}{\mathit{\eta }}_t\left[n\right],$$

where ε[n] = P^TB v[n] - gy[n] is the error vector when there is no Tx-noise.

Since the Tx-noise is independent from the transmitted signal and the Rx-noise, the MSE can be decomposed as

$$\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]=\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]+{\left|g\right|}^2\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right),$$

(25)

where $\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]$ is the MSE when there is no Tx-noise, which constitutes the cost function that is minimized in the conventional MMSE design of THP.

Following similar derivations as in [12], the minimization of the MSE cost function in Equation (25), subject to the mentioned constraints, can be carried out from the factorization of

$${\mathbf{\Phi }}_t={\left(\mathit{H}{\mathit{H}}^{\text{H}}+{\xi }_t\mathit{I}\right)}^{-1},$$

where

$${\xi }_{\text{t}}=\xi +\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)/E_{\text{tx}},$$

(26)

with $\xi =\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_{\text{r}}}\right)/E_{\text{tx}}$. The symmetrically permuted Cholesky decomposition of this matrix is

$$\mathit{P}{\mathbf{\Phi }}_{\text{t}}{\mathit{P}}^{\text{T}}={\mathit{L}}^{\text{H}}\mathit{DL},$$

(27)

where L and D are, respectively, unit lower triangular and diagonal matrices. Finally, the MMSE solution for the THP filters that account for the Tx-noise is given by

$$\begin{array}{ll}\hfill {\mathit{F}}_{\text{MMSE}}^{\text{THP}}& =g_{\text{MMSE}}^{\text{THP},-1}{\mathit{H}}^{\text{H}}{\mathit{P}}^{\text{T}}{\mathit{L}}^{\text{H}}\mathit{D},\\ \hfill {\mathit{B}}_{\text{MMSE}}^{\text{THP}}& ={\mathit{L}}^{-1}\end{array}$$

(28)

The receive scalar weight $g_{\text{MMSE}}^{\text{THP}}$ is directly obtained from the transmit-energy constraint. Assuming that it is real and positive, it is obtained that

$$g_{\text{MMSE}}^{\text{THP}}=\sqrt{\frac{\text{tr}\left({\mathit{H}}^{\text{H}}{\mathit{P}}^{\text{T}}{\mathit{L}}^{\text{H}}{\mathit{D}}^2{\mathit{C}}_v\mathit{LPH}\right)}{E_{\text{tx}}}},$$

(29)

where C _v is the covariance matrix of v[n], which is diagonal with entries depending on the modulation alphabet [11].

The minimum value for the MSE cost function given by Equation (25) can be obtained by substituting the expressions obtained for the optimum filters ${\mathit{F}}_{\text{MMSE}}^{\text{THP}}$ and ${\mathit{B}}_{\text{MMSE}}^{\text{THP}}$, and for the gain factor $g_{\text{MMSE}}^{\text{THP}}$. It is easy to show that the final MMSE under the presence of Tx-noise is

$$\text{MMS}{\text{E}}_{t,\text{THP}}={\xi }_{\text{t}}\text{tr}\left({\mathit{C}}_v\mathit{D}\right),$$

(30)

where ξ_t is given by Equation (26) and D is the diagonal matrix that results from the permuted Cholesky factorization of Equation (27).

As it is done in [12], instead of testing all the possible permutation matrices to find the one that minimizes the cost function of Equation (30), the ordering optimization can be included in the computation of the Cholesky decomposition of Equation (27).

Again, it is straightforward to obtain the expressions for the ZF THP design as the limiting case when ξ_t → 0. The expressions for the filters ${\mathit{F}}_{\text{ZF}}^{\text{THP}}$ and ${\mathit{B}}_{\text{ZF}}^{\text{THP}}$ are equal to those obtained for ${\mathit{F}}_{\text{MMSE}}^{\text{THP}}$ and ${\mathit{B}}_{\text{MMSE}}^{\text{THP}}$, respectively, although the matrices P, L, and D should be obtained from the symmetrically permuted Cholesky factorization of

$${\mathbf{\Phi }}_t={\left(\mathit{H}{\mathit{H}}^{\text{H}}\right)}^{-1}.$$

4.3 MIMO vector precoder design with transmitter noise

Figure 4 shows the block diagram of a MIMO system with VP. The transmitter has the freedom to add an arbitrary perturbation signal $\mathit{a}\left[n\right]\in \tau {\mathbb{Z}}^{N_{\text{r}}}+\text{j}\tau {\mathbb{Z}}^{N_{\text{r}}}$ to the data signal prior to the linear transformation with the filter $\mathit{F}\in {ℂ}^{N_{\text{t}}\times N_{\text{r}}}$. This perturbation will be later on removed by the modulo operator M(●) at the receiver. Here, τ denotes a constant that depends on the modulation alphabet.^a This constant is associated with the nonlinear modulo operator M(●), defined as

Figure 4

Nonlinear MIMO system with VP and Tx-noise.

$$\text{M}\left(x\right)=x-\left(⌊\frac{\Re \left(x\right)}{\tau }+\frac{1}{2}⌋\tau +\text{j}⌊\frac{\Im \left(x\right)}{\tau }+\frac{1}{2}⌋\tau \right)\in V,$$

(31)

where ⌊●⌋ denotes the floor operator which gives the largest integer smaller than or equal to the argument. The corresponding fundamental Voronoi region is

$$V=\left\{x\in ℂ|-\frac{\tau }{2}\le \Re \left(x\right)<\frac{\tau }{2},-\frac{\tau }{2}\le \Im \left(x\right)<\frac{\tau }{2}\right\},$$

which means that the modulo operator constrains the real and imaginary part of x to the interval [-τ/2, τ/2] by adding integer multiples of τ and j τ to the real and imaginary part, respectively.

As it can be seen from Figure 4, the data vector $\mathit{u}\left[n\right]\in {ℂ}^{N_{\text{r}}}$ is first superimposed with the perturbation vector a[n], and the resulting vector is then processed by the linear filter F to form the transmitted signal $\mathit{x}\left[n\right]=\mathit{Fd}\left[n\right]\in {ℂ}^{N_t}$, n = 1,..., N_B, where d[n] is the desired signal given by u[n] + a[n] and n is the symbol index in a block size of N_B data symbols. The transmit-energy constraint is expressed as ${\sum }_{n=1}^{N_{\text{B}}}{∥\mathit{x}\left[n\right]∥}_2^2/N_{\text{B}}\le E_{\text{tx}}$ since transmit-symbols statistics are unknown.

The weight g in Figure 4 is assumed to be constant throughout the block of N_B symbols. Again, note that a common weight for all the receivers is used. Thus, the weighted estimated signal is given by

$${\widehat{\mathit{d}}}_{\text{t}}\left[n\right]=\widehat{\mathit{d}}\left[n\right]+g\mathit{H}{\mathit{\eta }}_{\text{t}}\left[n\right],$$

with $\widehat{\mathit{d}}\left[n\right]=g\mathit{HFd}\left[n\right]+g{\mathit{\eta }}_{\text{r}}\left[n\right]$. The modulo operator at the receiver compensates the effect of adding the perturbation a[n] at the transmitter.

Since a[n] is discrete, their optimum values cannot be obtained after derivation. The optimization procedure is as follows. We start by fixing a[n], after which x[n] and g are optimized taking into account the transmit power constraint. For these optimum x[n] and g we choose the best a[n] in order to minimize the following MSE criterion [13]

$$\text{MS}{\text{E}}_{t,\text{VP}}=\frac{1}{N_{\text{B}}}\sum _{n=1}^{N_{\text{B}}}\text{E}\left[{∥\mathit{d}\left[n\right]-{\widehat{\mathit{d}}}_{\text{t}}\left[n\right]∥}_2^2|\mathit{u}\left[n\right]\right].$$

(32)

With Tx-noise being present, the previous MSE cost function can be expressed as

$$\text{MS}{\text{E}}_{t,\text{VP}}=\text{MS}{\text{E}}_{\text{VP}}+\frac{1}{N_{\text{B}}}\sum _{n=1}^{N_{\text{B}}}{∥g\mathit{H}{\mathit{\eta }}_{\text{t}}\left[n\right]∥}_2^2,$$

(33)

where $\text{MS}{\text{E}}_{\text{VP}}={\sum }_{n=1}^{N_{\text{B}}}\text{E}\left[{∥\mathit{d}\left[n\right]-\widehat{\mathit{d}}\left[n\right]∥}_2^2|\mathit{u}\left[n\right]\right]/N_{\text{B}}$ is the MSE cost function when the Tx-noise is away. Following an optimization procedure similar to that described in [13], we arrive at the MMSE VP solution given by

$$\begin{array}{ll}\hfill {\mathit{x}}_{\text{MMSE}}^{\text{VP}}\left[n\right]& =\frac{1}{g_{\text{MMSE}}^{\text{VP}}}{\left({\mathit{H}}^{\text{H}}\mathit{H}+{\xi }_{\text{t}}\mathit{I}\right)}^{-1}{\mathit{H}}^{\text{H}}\mathit{d}\left[n\right],\\ \hfill g_{\text{MMSE}}^{\text{VP}}& =\sqrt{\frac{{\sum }_{n=1}^{N_{\text{B}}}{\mathit{d}}^{\text{H}}\left[n\right]\mathit{H}{\left({\mathit{H}}^{\text{H}}\mathit{H}+{\xi }_t\mathit{I}\right)}^{-2}{\mathit{H}}^{\text{H}}\mathit{d}\left[n\right]}{E_{\text{tx}}{N}_{\text{B}}}},\end{array}$$

(34)

where $g_{\text{MMSE}}^{\text{VP}}$ is directly obtained from the transmit-energy constraint.

By defining the matrix Φ_t = (HH^H + ξ_tI)^-1 and applying the matrix inversion lemma to Equation (34), the MSE cost function given by Equation (33) is reduced to [13]

$$\text{MMS}{\text{E}}_{t,\text{VP}}=\frac{{\xi }_t}{N_{\text{B}}}\sum _{n=1}^{N_{\text{B}}}{\mathit{d}}^{\text{H}}\left[n\right]{\mathbf{\Phi }}_{\text{t}}\mathit{d}\left[n\right].$$

Since Φ_t is positive definite, we can use the Cholesky factorization to obtain a lower triangular matrix L and a diagonal matrix D with the following relationship,

$${\mathbf{\Phi }}_{\text{t}}={\left(\mathit{H}{\mathit{H}}^{\text{H}}+{\xi }_{\text{t}}\mathbf{I}\right)}^{-1}={\mathit{L}}^{\text{H}}\mathit{DL}.$$

Thus, the perturbation signal can be found by means of the following search [13]

$$\begin{array}{ll}\hfill {\mathit{a}}_{\text{MMSE}}^{\text{VP}}\left[n\right]& =\underset{a\left[n\right]\in \tau {\mathbb{Z}}^{N_{\text{r}}}+\text{j}\tau {\mathbb{Z}}^{N_{\text{r}}}}{\text{argmin}}{\left(\mathit{u}\left[n\right]+\mathit{a}\left[n\right]\right)}^{\text{H}}{\mathbf{\Phi }}_{\text{t}}\left(\mathit{u}\left[n\right]+a\left[n\right]\right)\\ =\underset{a\left[n\right]\in \tau {\mathbb{Z}}^{N_{\text{r}}}+\text{j}\tau {\mathbb{Z}}^{N_{\text{r}}}}{\text{argmin}}{∥{\mathit{D}}^{1/2}\mathit{L}\left(\mathit{u}\left[n\right]+\mathit{a}\left[n\right]\right)∥}_2^2.\end{array}$$

(35)

This search can be solved using the Schnorr-Euchner sphere-decoding algorithm [23]. It is interesting to note that THP can be interpreted as a suboptimum approach to VP where a[n] is successively computed.

The ZF constraint $\text{E}\left[\widehat{\mathit{d}}\left[n\right]|\mathit{d}\left[n\right]\right]=g\mathit{HFd}\left[n\right]$, for n = 1,..., N_B, leads to similar expressions for ${\mathit{x}}_{\text{ZF}}^{\text{VP}}\left[n\right]$ and $g_{\text{ZF}}^{\text{VP}}$ as those obtained for the MMSE VP design when ξ _t → 0. Following similar steps as before, the cost function for ZF VP has exactly the same form as that of MMSE VP but considering Φ_t = (HH^H)^-1. Finally, the optimum perturbation vectors are found by the following closest point search in a lattice

$${\mathit{a}}_{\text{ZF}}^{\text{VP}}\left[n\right]=\underset{a\left[n\right]\in \tau {\mathbb{Z}}^{N_{\text{r}}}+\mathrm{j\tau }{\mathbb{Z}}^{N_{\text{r}}}}{\text{argmin}}{∥{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{H}}^{\text{H}}\right)}^{-1}\mathit{d}\left[n\right]∥}_2^2.$$

5 Experimental results

In this section, the results of several computer simulations are presented to illustrate the impact of Tx-noise on the performance of all the MIMO transmission systems already described in the previous sections. Figure 5 shows a picture of the transmitter of the wireless testbed developed at the University of A Coruña [17], which has been used to obtain realistic 4 × 4 MIMO channel realizations from an indoor scenario. We also measured the covariance matrices of both the Tx-noise and the Rx-noise corresponding to this specific testbed. The interaction with the testbed is solved by means of a custom-designed, distributed, multilayer software architecture [17].

Figure 5

Picture of testbed transmitter developed at the University of A Coruña.

5.1 Testbed details

As explained in [24, 25], both transmit and receive testbed nodes are equipped with a Quad Dual-Band front-end from Lyrtech, Inc [26]. This RF front-end is equipped with up to eight antennas,^b which are connected to four direct-conversion transceivers by means of an antenna switch. The front-end is based on Maxim [27] MAX2829 chip (also found in front-ends like Ettus [28] XCVR2450 or Sundance [29] SMT911). It supports both up and down conversion operations from either a 2.4 to 2.5 GHz band or a 4.9 to 5.875 GHz band. The front-end also incorporates a programmable variable attenuator to control the transmit power. The attenuation ranges from 0 to 31dB in 1 dB steps, while the maximum transmit power declared by Lyrtech is 25dBm per transceiver.

The baseband hardware is based on commercial off the shelf (COTS) components from Sundance multiprocessor [29]. More specifically, the transmit node contains four DACs, which generate intermediate-frequency (IF) signals to feed the RF front-end only through the I branch (while the Q branch is not used). Given that an IF signal is provided to a direct-conversion front-end, the desired signal plus an undesired replica are recovered at its output. Nevertheless, this replica is suppressed at the receiver by shifting the RF carrier frequency and by an adequate filtering in the digital domain.

Both transmit and receive nodes make use of real-time buffers which are used to store the signals to be sent to the DACs as well as the signals acquired by the analog-to-digital converters (ADCs). The utilization of such buffers allows for transmission and acquisition of signals in real-time, while signal generation and processing is carried out off-line. Additionally, both baseband hardware and RF front-ends of all the nodes are synchronized in both time and frequency by means of two mechanisms:

• First, a hardware trigger is attached to the real-time buffers and to the DACs at the transmitter. When the trigger is switched on (this action can also be started by the user directly from MATLAB), all the buffers and DACs receive the trigger signal and they start to transmit simultaneously.

• Second, the same common external 40 MHz reference oscillator is utilized by both DACs and RF front-ends of all the nodes, thus guaranteeing frequency synchronization.

5.2 Measurement set-up

At the transmitter side, once the discrete-time and complex-valued sequences to be transmitted have been generated, the resulting symbols are assembled into the frame structure plotted in Figure 6. A pseudo noise (PN) sequence of 119 BPSK symbols is appended for frame detection and synchronization. Next, a 50 symbol duration silence period is intentionally left to estimate the Rx-noise covariance matrix. Finally, 4 000 4-QAM modulated pilot symbols are transmitted per antenna in order to estimate the MIMO channel.

Figure 6

Frame structure used in the measurements.

The frame structure plotted in Figure 6 is suitable for estimating the covariance matrices of both receiver and transmitter noise. On the one hand, from the signals acquired during the silence period, the Rx-noise covariance matrix ${Ĉ}_{{\mathit{\eta }}_{\text{r}}}$ is estimated. On the other hand, both the MIMO channel Ĥ and the total noise covariance matrix, ${\widehat{\mathit{C}}}_{\mathit{\eta }}$, are estimated from the signals measured when the training sequence is transmitted. According to Equations (1) and (4), the total noise covariance matrix is ${\mathit{C}}_{\mathit{\eta }}=\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_{\text{t}}}\mathit{H}+{\mathit{C}}_{{\mathit{\eta }}_{\text{r}}}$. Hence, the estimate of the Tx-noise covariance matrix, ${\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{t}}}$, can be obtained as

$${\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{t}}}={\widehat{\mathit{H}}}^{-1}\left({\widehat{\mathit{C}}}_{\mathit{\eta }}-{\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{r}}}\right){\widehat{\mathit{H}}}^{-\text{H}},$$

(36)

where the superscript ^-H denotes the inverse transpose conjugate operator.

An up-sampling by a factor of 40 (i.e., 40 samples per symbol) and a pulse-shape filtering using a squared root-raised cosine filter with 12% roll-off are performed. Consequently, given that the sampling frequency of the DACs is set to 40 MHz, the signal bandwidth is 1.12 MHz, which leads--according to our tests--to a frequency-flat channel response. Note that DACs implement an internal, interpolating filter which improves the signal quality at its output, resulting in a sampling rate of 160 Msample/s. The obtained signals are I/Q modulated to form a passband signal at a carrier frequency of f_IF = 3 MHz. Such signals are then properly scaled in order to guarantee the transmit-energy constraint. Moreover, given that the DACs have a resolution of 16 bits, the signals are properly quantized, giving as a result 16-bit integer values for the samples. These signals are stored off-line in buffers available at the transmit nodes of the testbed.

Afterwards, the transmitter triggers all the DACs at the same time, so that the buffers are simultaneously and cyclically read in real-time by the corresponding DACs, hence generating signals at the IF. Next, these analog signals are sent to the RF front-end to be transmitted at the desired RF center frequency at a mean transmit power of 5dBm per transmit antenna.^c With the aim of obtaining statistically-rich channel realizations, and given that the RF front-end is frequency-agile, we measured at different RF carriers (frequency hopping) in the frequency interval ranging from 5219 MHz to 5 251 MHz and from 5 483 MHz to 5 703 MHz. Carrier spacing is 4MHz (greater than the signal bandwidth), which results in 65 different frequencies. Additionally, we repeated this measurement procedure for four different spatial positions of the receiver, giving as a result 260 different channel realizations. Channel matrices were normalized in order that the mean Frobenius norm after averaging over the 260 channel realizations be equal to N_tN_r = 16.

Note that 260 channel realizations constitute a rich set of independent statistical samples from the channel (covering both 2.5 and 5GHz ISM bands) because we analyze the results in relative terms, i.e., we always compare a certain curve to a so-called reference curve and we focus our attention in the difference (improvement) rather than in the absolute values obtained. In this case, when the difference is the target of the analysis and not the absolute values, both curves (the reference and the one corresponding to the proposed system) are correlated, thus the uncertainty associated to the results is much lower than that of the individual curves. An example can be found in [30].

At the receiver side, once the transmitter has been triggered, signals are processed as follows. First, the RF front-end down-converts the signals received by the selected antennas (up to four) to the baseband, generating the corresponding analog passband signals. All the IF signals are then digitized by the ADCs by sampling at 40MHz and they are taken in realtime to the corresponding buffers. Given that the signals are being cyclically transmitted and in order to guarantee that a frame is entirely captured, all the transmit frame is acquired twice. The signals are then properly scaled according to the 14 bits resolution exhibited by ADCs. Notice that this factor is constant during the whole measurement, hence not affecting the channel properties. Next, the acquired signals are filtered using a custom-designed passband filter, which eliminates all the undesired replicas of the signals. Next, time and frequency synchronization are carried out making use of the known preamble. As soon as the acquired frames are correctly synchronized, the resulting signals are I/Q demodulated and filtered again (matched filter) so that as a result, discrete-time, and complex-valued observations with 40 samples per symbol are obtained. After filtering, the signals are decimated. Finally, the frame is properly disassembled, and final observations are then sent to the channel estimator. The raw acquired signals, i.e., the discrete-time complex-valued observations (including those obtained during the silence period shown in Figure 6), as well as the channel estimates, are finally stored for subsequent evaluation.

5.3 Results with measured indoor channels

In total, 260 different realizations were obtained from the measurement campaign.^d The same measurement is utilized with two objectives. First, based on the frame structure shown in Figure 6, we estimate the channel coefficients. Second, making use of the silence period included in the frame, the noise variance at the receiver (Rx-noise) from each acquired frame is also estimated. Such a noise variance estimation is readily obtained by calculating the variance of the samples corresponding to the silence period after filtering and decimation at the receiver.

After averaging over all the 260 Rx-noise realizations obtained from the measurement campaign, the following (unnormalized) covariance matrix is obtained for the Rx-noise,

$${\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{r}}}=10^2\times \left[\begin{array}{c}\hfill 1.2058-0.0104+0.0079\text{i}-0.0030+0.0141\text{i}\text{0}\text{.0212}-0.0040\text{i}\hfill \\ \hfill -0.0104-0.0079\text{i}0.99730.0119+0.0061\text{i}\text{0}\text{.0043}-0.0038\text{i}\hfill \\ \hfill -0.0030-0.0141\text{i}0.0119-0.0061\text{i}0.6265-0.0007+0.0076\text{i}\hfill \\ \hfill 0.0212+0.0040\text{i}0.0043+0.0038\text{i}-0.0007-0.0076\text{i}\text{1}\text{.1025}\hfill \end{array}\right],$$

(37)

whereas the corresponding Tx-noise covariance matrix (also unnormalized) is obtained from 260 different measurements carried out connecting the transmitter and the receiver using coaxial cables plus attenuators (we repeat exactly the same procedure for measuring both the wireless channels and the noise at the transmitter). Notice that the configuration parameters of the transmitter are exactly the same as before, when measuring ${Ĉ}_{{\mathit{\eta }}_{\text{r}}}$ by means of wireless transmissions. However, by using cables plus attenuators, such a Tx-noise covariance matrix estimation is significantly more accurate as well as more precise because the channel matrix H in Equation (36) can be assumed to be known (actually, the cabled channel is estimated with very high precision and accuracy). Such a Tx-noise covariance matrix is given by

$${\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{t}}}=10^{-3}\times \left[\begin{array}{c}\hfill 2.60690.0810-0.0115\text{i}0.0058+0.0068\text{i}-0.0107+0.0001\text{i}\hfill \\ \hfill 0.0081+0.0115\text{i}2.64130.0020+0.0020\text{i}-0.0127+0.0035\text{i}\hfill \\ \hfill 0.0058-0.0068\text{i}0.0020-0.0020\text{i}2.55520.0037+0.0018\text{i}\hfill \\ \hfill -0.0107-0.0001\text{i}-0.0127-0.0035\text{i}0.0037-0.0018\text{i}3.4155\hfill \end{array}\right].$$

(38)

Thus, the STxNR is estimated by performing $N_{\text{t}}/\text{tr}\left({\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{t}}}\right)$, with ${\widehat{\mathit{C}}}_{{\mathit{\eta }}_{\text{t}}}$ given by Equation (38), obtaining as a result the value of 25.52 dB, which is in accordance with the practical range between 22 dB and 32 dB.

The figure of merit chosen for the performance evaluation is the uncoded BER versus SRxNR, as defined in Equation (3). The symbols are QPSK modulated and it is also assumed that C _u = I. The channel is quasi-static and remains unchanged during the transmission of a frame of 10⁵ symbols. Therefore, additive Gaussian Rx-noise and Tx-noise were artificially generated and spatially colored according to matrices of Equations (37) and (38), respectively, to recreate the measured scenario. Also note that additional noise was injected to change the operating SRxNR value of the BER curves.

Figures 7 to 10 show the results after averaging over all the 260 channel realizations obtained from this measurement campaign. Each figure plots the uncoded BER versus SRxNR curves corresponding to the following three different situations: i) system performance for the ideal case in which there is no Tx-noise (curves labeled as "Classical ... with no Tx-noise"); ii) system performance when there is Tx-noise but the system design does not take it into account (curves labeled as "Classical ... with Tx-noise"); and iii) system performance when there is Tx-noise which is considered for the system design (curves labeled as "Modified ... with Tx-noise").

Figure 7

BER versus SRxNR with MIMO linear and nonlinear (DF) receivers over measured indoor channels. Figure 7 shows the uncoded BER with respect to the SRxNR. A MIMO system with four transmit antennas and four receive antennas (N_t = N_r = 4) is considered. The performance evaluation is carried out for ZF as well as MMSE, considering linear receivers and nonlinear (DF) receivers, while all schemes are evaluated with and without Tx-noise. Finally, schemes specifically designed to deal with Tx-noise (labeled as "modified") are also evaluated. In all cases, QPSK signals over measured indoor channels are used, whereas the estimated STxNR is about 25dB.

Figure 7 shows the performance of the MIMO system with both MMSE and ZF linear receivers. As it can be seen from the figure, linear receivers are very little affected by the presence of Tx-noise. Figure 7 also plots the average BER versus SRxNR for MIMO systems with both MMSE DF and ZF DF receivers. Again, Tx-noise does not severely degrade system performance (except for the MMSE DF at SRxNR values greater than 30dB). Considering the Tx-noise in the DF receiver design only produces a small improvement in performance.

TX-noise, however, severely degrades the performance of MIMO systems with linear precoding (LP), as it is plotted in Figure 8. Notice how performance starts to degrade for SRxNR values above 20 dB. Tx-noise causes an error floor of about 10^-2, which can be reduced to 3 × 10^-3 when it is considered in the MMSE precoder design.

Figure 8

BER versus SRxNR with MIMO LP over measured indoor channels. Figure 8 shows the uncoded BER with respect to the SRxNR. A MIMO system with four transmit antennas and four receive antennas (N_t = N_r = 4) is considered. The performance evaluation is carried out considering LP for ZF as well as MMSE, while all schemes are evaluated with and without Tx-noise. Finally, schemes specifically designed to deal with Tx-noise (labeled as "modified") are also evaluated. In all cases, QPSK signals over measured indoor channels are used, whereas the estimated STxNR is about 25 dB.

Figure 9 plots the BER performance in terms of SRxNR for MIMO systems with both MMSE and ZF TH precoding. Note the high sensitivity of the performance of MIMO systems with THP with respect to Tx-noise, which produces that the curves go up again for high scenarios [31]. The performance of both MMSE and ZF TH precoding is considerably degraded due to the Tx-noise and an error floor arises at the significant value of 2 × 10^-3 (at 40dB of SRxNR). This error floor reduces to about 2 × 10^-6 (at 40 dB of SRxNR) when the Tx-noise is considered in the THP design. Therefore, the increasing effect in BER due to Tx-noise, specially for high SNR, is strongly mitigated when the design is robust against this type of impairments.

Figure 9

BER versus SRxNR with MIMO THP over measured indoor channels. Figure 9 shows the uncoded BER with respect to the SRxNR. A MIMO system with four transmit antennas and four receive antennas (N_t = N_r = 4) is considered. The performance evaluation is carried out considering THP for ZF as well as MMSE, while all schemes are evaluated with and without Tx-noise. Finally, schemes specifically designed to deal with Tx-noise (labeled as "modified") are also evaluated. In all cases, QPSK signals over measured indoor channels are used, whereas the estimated STxNR is about 25 dB.

The same behaviour is observed for VP although at lower values of BER since VP performs better than THP and it is not so influenced by noise impairments at the transmitter due to the correction performed by the lattice search. Figure 10 shows how Tx-noise degrades the performance of MIMO systems with MMSE VP for SRxNR values greater than 20dB introducing an error floor at 10^-5. This error floor reduces to 2 × 10^-6 when the Tx-noise is introduced in the MMSE VP design.

Figure 10

BER versus SRxNR with MIMO VP over measured indoor channels for N _t = N _r = 4. Estimated STxNR is about 25dB. Figure 10 shows the uncoded BER with respect to the SRxNR. A MIMO system with four transmit antennas and four receive antennas (N_t = N_r = 4) is considered. The performance evaluation is carried out considering VP for ZF as well as MMSE, while all schemes are evaluated with and without Tx-noise. Finally, schemes specifically designed to deal with Tx-noise (labeled as "modified") are also evaluated. In all cases, QPSK signals over measured indoor channels are used, whereas the estimated STxNR is about 25dB.

In general, in all cases--linear precoding, Tomlinson-Harashima, and vector precoding-- for SRxNR values smaller than 20dB, the noise at the receiver is the dominant factor, so that STxNR values around 25dB do not influence the uncoded BER performance. Note that the STxNR value of 25.52dB determines the SRxNR value at which the performance begins to be improved by considering Tx-noise in the system design.

Figures 9 and 10 present the performance evaluation results in terms of uncoded BER versus SRxNR for TH precoding as well as VP when QPSK mappings are utilized. However, higher modulation levels lead to a greater loss in performance when the Tx-noise is not included in the designs as it can be seen in Figures 11 and 12, where the performance of Tomlinson-Harashima and vector precoding using 16-QAM is, respectively, shown. In both cases, the proposed modified designs will mitigate that loss produced by Tx-noise.

Figure 11

The same as Figure 9 but for 16-QAM instead of QPSK. Figure 11 shows the uncoded BER with respect to the SRxNR. A MIMO system with four transmit antennas and four receive antennas (N_t = N_r = 4) is considered. The performance evaluation is carried out considering THP for ZF as well as MMSE, while all schemes are evaluated with and without Tx-noise. Finally, schemes specifically designed to deal with Tx-noise (labeled as "modified") are also evaluated. In all cases, 16-QAM signals over measured indoor channels are used, whereas the estimated STxNR is about 25dB.

Figure 12

The same as Figure 11 but for 16-QAM instead of QPSK. Figure 12 shows the uncoded BER with respect to the SRxNR. A MIMO system with four transmit antennas and four receive antennas (N_t = N_r = 4) is considered. The performance evaluation is carried out considering VP for ZF as well as MMSE, while all schemes are evaluated with and without Tx-noise. Finally, schemes specifically designed to deal with Tx-noise (labeled as "modified") are also evaluated. In all cases, 16-QAM signals over measured indoor channels are used, whereas the estimated STxNR is about 25 dB.

Finally, note also that for increasing SRxNR values, all curves corresponding to MMSE-based systems converge to those corresponding to ZF-based systems.

6 Conclusions

This study investigates the impact of transmitter noise on the performance of several MIMO systems, namely, linear and decision feedback receivers, as well as linear, Tomlinson-Harashima, and vector precoders. In all these cases, both MMSE and ZF designs were considered. We derived the expressions for all the transceiver designs taking into account the presence of transmitter noise. We also presented the results that illustrate the performance of all the aforementioned transmission schemes using MIMO channels and noise models, both obtained from an indoor measurement campaign carried out with a MIMO testbed developed at the University of A Coruña.

Basically, we conclude that transmitter noise has little impact on the performance--expressed in terms of uncoded BER versus signal-to-noise ratio at the receiver--of MIMO linear and decision feedback receivers, but marginal performance improvements can be expected when considering transmitter noise in the receiver design. On the contrary, transmitter noise considerably degrades the performance of precoded MIMO systems. Nevertheless, significant improvements on performance are obtained when transmitter noise is included in the precoder design. As expected, systems with VP produce the best results while the worst ones are obtained with LP.

Appendix 1: Details of robust optimizations

In this appendix, we will derive the expressions of all the linear and nonlinear filters shown throughout this article including the transmitter noise in the optimizations in order to obtain the MMSE robust designs appropriate to mitigate such transmitter impairments.

Appendix 1.1: Derivation of MIMO linear receiver design with transmitter noise

The mean square error with Tx-noise in the MMSE linear receiver design is given by

$$\begin{array}{ll}\hfill \text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]& =\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)-\text{tr}\left(p^{*}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)-\text{tr}\left(p\mathit{GH}{\mathit{C}}_{\mathit{u}}\right)\\ +\text{tr}\left({\left|p\right|}^2\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)+\text{tr}\left(\mathit{G}{\mathit{C}}_{{\mathit{\eta }}_r}{\mathit{G}}^{\text{H}}\right)+\text{tr}\left(\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right),\end{array}$$

which can be expressed similarly to Equation (9) by doing $\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]$, i.e., the mean square error without Tx-noise, equal to $\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)-\text{tr}\left(p^{*}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)-\text{tr}\left(p\mathit{GH}{\mathit{C}}_{\mathit{u}}\right)+\text{tr}\left({\left|p\right|}^2\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)+\text{tr}\left(\mathit{G}{\mathit{C}}_{{\mathit{\eta }}_r}{\mathit{G}}^{\text{H}}\right)$.

Then, we construct the Lagrangian function as follows

$$L_t\left(p,\mathit{G},\lambda \right)=E\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]+\lambda \left({\left|p\right|}^2\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)-E_{\text{tx}}\right),$$

with the Lagrangian multiplier λ ∈ ℝ^0,+.

We equate the derivatives with respect to p and G to zero, which leads to the following Karush-Kuhn-Tucker (KKT) optimality conditions [32–35]

$$\begin{array}{ll}\hfill \frac{\partial L_t\left(•\right)}{\partial \mathit{G}*}& =-p^{*}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}+{\left|p\right|}^2\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}+\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+\mathit{G}{\mathit{C}}_{{\mathit{\eta }}_r}=0\\ \hfill \frac{\partial L_t\left(•\right)}{\mathrm{\partial p}}& =-\text{tr}\left(\mathit{GH}{\mathit{C}}_{\mathit{u}}\right)+p^{*}\text{tr}\left(\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)+\lambda p^{*}\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)=0\\ \hfill {\left|p\right|}^2\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)& \le E_{\text{tx}}\\ \hfill \lambda \left({\left|p\right|}^2\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)-E_{\text{tx}}\right)& =0\text{with}\lambda \ge 0.\end{array}$$

From the first equation, we obtain the following expression for the receive filter G

$$\mathit{G}=p^{*}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\left({\left|p\right|}^2\mathit{H}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right)}^{-1}.$$

By plugging this result into the second KKT condition, it is easy to demonstrate that λ > 0, and therefore the energy transmit constraint is maintained. To ensure a unique solution, we restrict p ∈ ℝ⁺. Thus, p is obtained from the energy transmit constraint and we have that $p=\sqrt{\frac{E_{\text{tx}}}{\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)}}$. Applying the matrix inversion lemma to the above expression for the receive filter G_MMSE, it can be demonstrated that

$$\begin{array}{ll}\hfill {\mathit{G}}_{\text{MMSE}}& =p_{\text{MMSE}}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}\left({\mathit{C}}_{\mathit{\eta }}^{-1}-{\mathit{C}}_{\mathit{\eta }}^{-1}\mathit{H}{\left(p_{\text{MMSE}}^{-2}\mathbf{I}+{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\mathit{H}\right)}^{-1}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\right)\\ =p_{\text{MMSE}}\left({\mathit{C}}_{\mathit{u}}-{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\mathit{H}{\left(p_{\text{MMSE}}^{-2}\mathbf{I}+{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\mathit{H}\right)}^{-1}{\mathit{C}}_{\mathit{u}}\right){\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\\ =p_{\text{MMSE}}{\left({\mathit{C}}_{\mathit{u}}^{-1}+p_{\text{MMSE}}^{\text{2}}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\mathit{H}\right)}^{-1}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1},\end{array}$$

where ${\mathit{C}}_{\mathit{\eta }}={\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^H$.

Then, we easily reach the solution for the MMSE linear receiver in Equation (10).

Appendix 1.2: Derivation of MIMO linear precoder design with transmitter noise

The mean square error for the MMSE linear precoder design with Tx-noise is given by

$$\begin{array}{ll}\hfill \text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]& =\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)-\text{tr}\left(g^{*}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)-\text{tr}\left(g\mathit{HF}{\mathit{C}}_{\mathit{u}}\right)\\ +{\left|g\right|}^2\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)+{\left|g\right|}^2\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}\right)+{\left|g\right|}^2\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right),\end{array}$$

(39)

where it can be equated $\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]=\text{tr}\left({\mathit{C}}_{\mathit{u}}\right)-\text{tr}\left(g^{*}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)-\text{tr}\left(g\mathit{HF}{\mathit{C}}_{\mathit{u}}\right)+{\left|g\right|}^2\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)+{\left|g\right|}^2\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}\right)$ according to Equation (15).

Thus, we can form the following Lagrangian function

$$L_t\left(\mathit{F},g,\lambda \right)=\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]+\lambda \left(\text{tr}\left(\mathit{F}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right)-E_{\text{tx}}\right).$$

Setting the derivatives with respect to F and g to zero, we obtain the necessary KKT conditions

$$\begin{array}{ll}\hfill \frac{\partial L_t\left(•\right)}{\partial \mathit{F}*}& =-g^{*}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{u}}+{\left|g\right|}^2{\mathit{H}}^{\text{H}}\mathit{HF}{\mathit{C}}_{\mathit{u}}+\lambda \mathit{F}{\mathit{C}}_{\mathit{u}}=0\\ \hfill \frac{\partial L_t\left(•\right)}{\mathrm{\partial g}}& =-\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}\right)+g^{*}\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)\\ +g^{*}\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}\right)+g^{*}\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{H}^H\right)=0\\ \hfill \text{tr}\left(\mathit{F}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right)& \le E_{\text{tx}}\\ \hfill \lambda \left(\text{tr}\left(\mathit{F}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right)-E_{\text{tx}}\right)& =0\text{with}\lambda \ge 0.\end{array}$$

(40)

The gain g* obtained from the second equation is given by

$$g^{*}=\frac{\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}\right)}{\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}.$$

(41)

Multiplying the first KKT condition by F^H from the right and applying the trace operator, we get the following

$$g^{*}\text{tr}\left({\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right)-{\left|g\right|}^2\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)=\lambda \text{tr}\left(\mathit{F}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right).$$

And now, combining this result with the expression for g* in Equation (41) yields

$$\begin{array}{ll}\hfill \lambda \text{tr}\left(\mathit{F}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right)& =\frac{\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}\right)}{\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}\text{tr}\left({\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}\right)\\ -\frac{{\left|\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}\right)\right|}^2}{\text{t}{\text{r}}^{\text{2}}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}\text{tr}\left(\mathit{HF}{\mathit{C}}_{\mathit{u}}{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)\\ ={\left|g\right|}^2\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}+\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right).\end{array}$$

(42)

From the above result λ = |g|²ξ _t , and if we plug this result for λ into the first KKT condition, we get

$$\mathit{F}=\frac{1}{g}{\left({\mathit{H}}^{\text{H}}\mathit{H}+{\xi }_t\mathbf{I}\right)}^{-1}{\mathit{H}}^{\text{H}}.$$

(43)

By considering tr(FC _u F^H) = E_tx and the above expression for F, it is obtained that

$${\left|g\right|}^2=\frac{\text{tr}\left({\left({\mathit{H}}^{\text{H}}\mathit{H}+{\xi }_t\mathbf{I}\right)}^{-2}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{u}}H\right)}{E_{\text{tx}}},$$

which leads to a unique solution given by Equation (16) if we restrict g to being positive real.

Appendix 1.3: Derivation of MIMO decision feedback receiver design with transmitter noise

The mean square error is calculated as

$$\begin{array}{ll}\hfill \text{E}\left[{∥{\epsilon }_{t,p}\left[n\right]∥}_2^2\right]& =\text{tr}\left(\mathit{BP}{\mathit{C}}_{\mathit{u}}{\mathit{P}}^{\text{T}}{\mathit{B}}^{\text{H}}\right)-\text{tr}\left(\mathit{BP}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)-\text{tr}\left(\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{P}}^{\text{T}}{\mathit{B}}^{\text{H}}\right)\\ +\text{tr}\left(\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)+\text{tr}\left(\mathit{G}{\mathit{C}}_{{\mathit{\eta }}_r}{\mathit{G}}^{\text{H}}\right)+\text{tr}\left(\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right),\end{array}$$

(44)

where $\text{E}\left[{∥{\epsilon }_p\left[n\right]∥}_2^2\right]$ of Equation (20) is given by $\text{E}\left[{∥{\epsilon }_p\left[n\right]∥}_2^2\right]=\text{tr}\left(\mathit{BP}{\mathit{C}}_{\mathit{u}}{\mathit{P}}^{\text{T}}{\mathit{B}}^{\text{H}}\right)-\text{tr}\left(\mathit{BP}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)-\text{tr}\left(\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{P}}^{\text{T}}{\mathit{B}}^{\text{H}}\right)\text{ + tr}\left(\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}{\mathit{G}}^{\text{H}}\right)+\text{tr}\left(\mathit{G}{\mathit{C}}_{{\mathit{\eta }}_r}{\mathit{G}}^{\text{H}}\right)$.

This allows us to construct the Lagrangian function

$$L_t\left(\mathit{P},\mathit{G},\mathit{B},{\mathit{\mu }}_1,\dots ,{\mathit{\mu }}_k\right)=\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]+2\Re \left(\text{tr}\left(\sum _{i=1}^N\left({\mathit{e}}_i^{\text{T}}{\mathit{BS}}_i^{\text{T}}-{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}\right){\mathit{\mu }}_i\right)\right),$$

(45)

where the equality ${\mathit{e}}_i^{\text{T}}{\mathit{BS}}_i^{\text{T}}={\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}$, for i = 1,..., N, must hold because of the unit lower triangular structure of B.^eTo mathematically formulate this restriction, we included the selection matrix S _i defined as

$${\mathit{S}}_i=\left[0_{N-i+1\times i-1},{\mathbf{I}}_{N-i+1}\right]\in {\left\{0,1\right\}}^{N-i+1\times N}.$$

(46)

The Lagrangian multiplier μ _i , i = 1,..., N, is a column vector of dimension N - i + 1.

By setting its derivatives with respect to G and B to zero, we obtain the following KKT conditions

$$\begin{array}{ll}\hfill \frac{\partial L_t\left(•\right)}{\partial \mathit{G}*}& =-\mathit{BP}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}+\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{H}}^{\text{H}}+\mathit{GH}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+\mathit{G}{\mathit{C}}_{{\mathit{\eta }}_r}=0\\ \hfill \frac{\partial L_t\left(•\right)}{\partial \mathit{B}*}& =\mathit{BP}{\mathit{C}}_{\mathit{u}}{\mathit{P}}^{\text{T}}-\mathit{GH}{\mathit{C}}_{\mathit{u}}{\mathit{P}}^{\text{T}}+\sum _{i=1}^N{\mathit{e}}_i{\mathit{\mu }}_i^{\text{H}}{\mathit{S}}_i=0\\ \hfill {\mathit{e}}_i^{\text{T}}{\mathit{BS}}_i^{\text{T}}& ={\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}\forall i\in \left\{1,\dots ,N\right\}.\end{array}$$

(47)

From the second KKT condition, we obtain

$$\mathit{B}=\mathit{GH}{\mathit{P}}^{\text{T}}-\left(\sum _{i=1}^N{\mathit{e}}_i{\mathit{\mu }}_i^{\text{H}}{\mathit{S}}_i\right){\mathit{PC}}_{\mathit{u}}^{-1}{\mathit{P}}^{\text{T}}.$$

(48)

Plugging this expression for B into the first KKT condition, we get

$$\mathit{G}=-\left(\sum _{i=1}^N{\mathit{e}}_i{\mathit{\mu }}_i^{\text{H}}{\mathit{S}}_i\right)\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}.$$

(49)

Substituting into Equation (48) we obtain

$$\mathit{B}=-\left(\sum _{i=1}^N{\mathit{e}}_{\mathit{i}}{\mathit{\mu }}_i^{\mathit{H}}{\mathit{S}}_i\right)\mathit{P}\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right){\mathit{P}}^{\text{T}}.$$

(50)

Applying the restriction concerned with the unit lower triangular structure of B to the above result leads to

$${\mathit{e}}_i^{\text{T}}{\mathit{BS}}_i^{\text{T}}=-{\mathit{e}}_i^{\text{T}}\left(\sum _{j=1}^N{\mathit{e}}_j{\mathit{\mu }}_j^{\text{H}}{\mathit{S}}_j\right)\mathit{P}\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right){\mathit{P}}^{\text{T}}{\mathit{S}}_i^{\text{T}}={\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}.$$

Then, with ${\mathit{e}}_i^{\text{T}}{\mathit{e}}_j=0$, for j ≠ i, and 1, otherwise, ${\mathit{\mu }}_i^{\text{H}}$ reads as

$${\mathit{\mu }}_i^{\text{H}}=-{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\left[{\mathit{S}}_i\mathit{P}\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right){\mathit{P}}^{\text{T}}{\mathit{S}}_i^{\text{T}}\right]}^{-1}.$$

This result for ${\mathit{\mu }}_i^{\text{H}}$ gives us the following expressions for the filters G and B

$$\begin{array}{ll}\hfill \mathit{G}& =\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\left[{\mathit{S}}_i\mathit{P}\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right){\mathit{P}}^{\text{T}}{\mathit{S}}_i^{\text{T}}\right]}^{-1}\times \\ \times {\mathit{S}}_i\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\\ \hfill \mathit{B}& =\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\left[{\mathit{S}}_i\mathit{P}\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right){\mathit{P}}^{\text{T}}{\mathit{S}}_i^{\text{T}}\right]}^{-1}\times \\ \times {\mathit{S}}_i\mathit{P}\left({\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\mathit{H}+{\mathit{C}}_{\mathit{u}}^{-1}\right){\mathit{P}}^{\text{T}}.\end{array}$$

(51)

Bearing in mind the Cholesky factorization with symmetric permutation, the feedforward filter in Equation (51) reduces to

$$\begin{array}{ll}\hfill \mathit{G}& =\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\left({\mathit{S}}_i{\mathit{L}}^{-\text{H}}{\mathit{D}}^{-1}{\mathit{L}}^{-1}{\mathit{S}}_i^{\text{T}}\right)}^{-1}{\mathit{S}}_i\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\\ =\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\left({\mathit{S}}_i{\mathit{L}}^{-\text{H}}{\mathit{D}}^{-1}{\mathit{S}}_i^{\text{T}}{\mathit{S}}_i{\mathit{L}}^{-1}{\mathit{S}}_i^{\text{T}}\right)}^{-1}{\mathit{S}}_i\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\\ =\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\mathit{S}}_i{\mathit{LS}}_i^{\text{T}}{\mathit{S}}_i\mathit{D}{\mathit{L}}^{\text{H}}{\mathit{S}}_i^{\text{T}}{\mathit{S}}_i\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}\\ =\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}\mathit{D}{\mathit{L}}^{\text{H}}{\mathit{S}}_i^{\text{T}}{\mathit{S}}_i\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1}=\sum _{i=1}^N{\mathit{e}}_i{\mathit{e}}_i^{\text{T}}\mathit{D}{\mathit{L}}^{\text{H}}\mathit{P}{\mathit{H}}^{\text{H}}{\mathit{C}}_{\mathit{\eta }}^{-1}\\ =\mathit{D}{\mathit{L}}^{\text{H}}\mathit{P}{\mathit{H}}^{\text{H}}{\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}+{\mathit{C}}_{{\mathit{\eta }}_r}\right)}^{-1},\end{array}$$

(52)

where in the derivations we have used the following properties for the selection matrix S _i

$${\mathit{S}}_i\mathit{N}={\mathit{S}}_i{\mathit{NS}}_i^{\text{T}}{\mathit{S}}_i,{\mathit{e}}_i^{\text{T}}{\mathit{S}}_i^{\text{T}}{\mathit{S}}_i{\mathit{MS}}_i^{\text{T}}{\mathit{S}}_i={\mathit{e}}_i^{\text{T}},\text{and}{\mathit{e}}_i^{\text{T}}{\mathit{NS}}_i^{\text{T}}{\mathit{S}}_i={\mathit{e}}_i^{\text{T}}N,$$

with N being an upper triangular matrix and M a unit lower triangular matrix. Comparing this result with Equation (49) leads to the conclusion that $-{\sum }_{i=1}^N{\mathit{e}}_i{\mathit{\mu }}_i^{\text{H}}{\mathit{S}}_i=\mathit{D}{\mathit{L}}^{\text{H}}$. Hence, the feedback filter reduces to

$$\mathit{B}=\mathit{D}{\mathit{L}}^{\text{H}}{\mathit{L}}^{-\text{H}}{\mathit{D}}^{-1}{\mathit{L}}^{-1}={\mathit{L}}^{-1}.$$

(53)

Therefore, the filters B and G corresponding to the MMSE DF receiver solution are given by Equation (21).

Appendix 1.4: Derivation of MIMO Tomlinson-Harashima design with transmitter noise

The mean square error is given by

$$\begin{array}{ll}\hfill \text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]& =\text{tr}\left({\mathit{P}}^{\text{T}}\mathit{B}{\mathit{C}}_v{\mathit{B}}^{\text{H}}\mathit{P}\right)-g^{*}\text{tr}\left({\mathit{P}}^{\text{T}}\mathit{B}{\mathit{C}}_v{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)-g\text{tr}\left(\mathit{HF}{\mathit{C}}_v{\mathit{B}}^{\text{H}}\mathit{P}\right)\\ +{\left|g\right|}^2\text{tr}\left(\mathit{HF}{\mathit{C}}_v{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)+{\left|g\right|}^2\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}\right)+{\left|g\right|}^2\text{tr}\left(\mathit{H}{\mathit{C}}_{{\mathit{\eta }}_t}{\mathit{H}}^{\text{H}}\right),\end{array}$$

(54)

where the mean square error without taking into account in the optimizations the Tx-noise is given by $\text{E}\left[{∥\epsilon \left[n\right]∥}_2^2\right]=\text{tr}\left({\mathit{P}}^{\text{T}}\mathit{B}{\mathit{C}}_v{\mathit{B}}^{\text{H}}\mathit{P}\right)-g^{*}\text{tr}\left({\mathit{P}}^{\text{T}}\mathit{B}{\mathit{C}}_v{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)-g\text{tr}\left(\mathit{HF}{\mathit{C}}_v{\mathit{B}}^{\text{H}}\mathit{P}\right)+{\left|g\right|}^2\text{tr}\left(\mathit{HF}{\mathit{C}}_v{\mathit{F}}^{\text{H}}{\mathit{H}}^{\text{H}}\right)+{\left|g\right|}^2\text{tr}\left({\mathit{C}}_{{\mathit{\eta }}_r}\right)$ accordingly to Equation (25).

Then, the MSE in Equation (54) enables us to construct the Lagrangian function as follows

$$\begin{array}{ll}\hfill L_t\left(\mathit{P},\mathit{B},\mathit{F},g,\lambda ,{\mathit{\mu }}_1,\dots ,{\mathit{\mu }}_K\right)& =\text{E}\left[{∥{\epsilon }_t\left[n\right]∥}_2^2\right]+\lambda \left(\text{tr}\left(\mathit{F}{\mathit{C}}_v{\mathit{F}}^{\text{H}}\right)-E_{\text{tx}}\right)\\ +2\Re \left(\sum _{i=1}^K\text{tr}\left({\mathit{\mu }}_i^{\text{T}}\left({\mathit{S}}_i\mathit{B}{\mathit{e}}_i-{\mathit{S}}_i{\mathit{e}}_i\right)\right)\right),\end{array}$$

(55)

where S _i is a selection matrix defined as [cf. Equation (46)], λ ∈ ℝ^0,+, μ _i ∈ ℂⁱ, i = 1,..., K, and $2\Re \left({\sum }_{i=1}^K\text{tr(}{\mathit{\mu }}_i^T\left({\mathit{S}}_i\mathit{B}{\mathit{e}}_i-{\mathit{S}}_i{\mathit{e}}_i\right)\text{)}\right)$ comes from the restriction for the unit lower triangular structure of the feedback matrix B.

Setting the derivatives of the Lagrangian function with respect to B, F, and g to zero we obtain

$$\begin{array}{l}\frac{\partial L_t(•)}{\partial F*}=-g^{*}{\mathit{H}}^{\text{H}}{\mathit{P}}^{\text{T}}\mathit{B}{\mathit{C}}_v+{\left|g\right|}^2{\mathit{H}}^{\text{H}}\mathit{HF}{\mathit{C}}_v+\lambda \mathit{F}{\mathit{C}}_v=0\\ \frac{\partial L_t(•)}{\partial \mathit{B}*}=\mathit{B}{\mathit{C}}_v-g\mathit{PHF}{\mathit{C}}_v+{\displaystyle \sum _{i=1}^K{\mathit{S}}_i^{\text{T}}{\mathit{\mu }}_i^{*}{\mathit{e}}_i^{\text{T}}=0}\\ \frac{\partial {}_{}}{}\end{array}$$