In this section, we focus on MIMO systems that use either nonlinear transmitters or nonlinear receivers to recover the transmitted data. It is well known that maximum likelihood detection (MLD) is the optimum detection scheme in the sense of minimizing the probability of a symbol being erroneously detected. The computational complexity of MLD grows exponentially with the number of transmit antennas and the modulation alphabet size and, for this reason, suboptimum nonlinear detection schemes such as the decision feedback one are preferred in a large number of practical situations.
However, decision-feedback receivers suffer from the major drawback of error propagation caused by feeding back erroneous decisions. One way to avoid this harmful effect is to perform a nonlinear filtering similar to that in DF but at the transmitter side. This idea leads to the concept of Tomlinson-Harashima precoding (THP). Finally, VP is another form of nonlinear MIMO transmit processing which will be considered in this section. Similarly to MLD, VP consists in a lattice search carried out at the transmitter instead of at the receiver side.
The impact of transmitter noise on the performance of MLD has already been analyzed in [2]. In the following sections, we will derive the expressions of the filters for the remaining nonlinear transceivers when the Tx-noise is accounted for.
4.1 MIMO decision feedback receiver design with transmitter noise
Figure 2 plots the block diagram of a MIMO system with a DF receiver. Information symbols will be represented by , where denotes the modulation alphabet, which are directly sent by the transmit antennas. It is assumed that u[n] is zero mean with covariance matrix denoted by C
u
.
It is apparent from Equations (1) and (4) that the input signal at the receiver can be written as
(18)
where is the received signal if there is no Tx-noise [5].
In DF reception, the signals at the channel output are passed through the feedforward filter , which forces the ISI to be spatially causal and the error to be spatially white (i.e., minimum variance). By means of the feedback filter and of the feedback loop shown in Figure 2, ISI can be recursively canceled without changing the statistical properties of the noise provided that the noise variance is sufficiently small so that the symbol detector (represented by Q(●) in Figure 2) produces correctly detected symbols. By elaborating on the signal model according to Figure 2, the estimated signal û
t
[n] can be written as
(19)
where y
t
[n] is defined as in Equation (18) and denotes the detected symbols after the threshold quantizer.
The order in which symbols are detected has a significant influence on the performance of DF MIMO receivers. In the system model shown in Figure 2, the ordering is obtained with the multiplication of the detected symbols, ũ[n], by the permutation matrix PT. This multiplication produces ũ
p
[n], which constitutes the vector of detected symbols conveniently sorted. Having in mind that PPT = I, we have that ũ[n] = Pũ
p
[n] and hence, û
t
[n] can be rewritten as
The MMSE design of the DF MIMO receiver searches for the filtering and permutation matrices that minimize the variance of the error vector
Assuming correct decisions (i.e., ũ
p
[n] = u[n]) and according to Equation (18), this error vector can be rewritten as
where ε
p
[n] = BPu[n] - Gy[n] is the error vector when there is no Tx-noise. Since the Tx-noise is independent from the Rx-noise and the transmitted signals, the MSE cost function to be minimized can be written as
(20)
where is the MSE with no Tx-noise. Notice that is the cost function that is minimized in the conventional MMSE design, whereas the additional term is the MSE improvement caused by the inclusion of the Tx-noise.
An MMSE design of the MIMO link that accounts for the Tx-noise should minimize the MSE given by Equation (20). Similarly to the scenario without Tx-noise [22], minimization of Equation (20) is readily accomplished from the Cholesky factorization with symmetric permutation of
This factorization is given by PΦ
t
PT= LDLH, where L is a unit lower triangular matrix and D is a diagonal matrix. After this decomposition, it can be demonstrated that the filters G and B for the MMSE DF nonlinear MIMO receiver solution are
(21)
The minimum value of the MSE cost function is obtained plugging and into Equation (20). Hence, the MMSE value is
(22)
where D is the diagonal matrix obtained from the Cholesky factorization with symmetric permutation of Φ
t
.
Notice that the MMSE expression given by Equation (22) depends on the permutation matrix P. Brute force optimization of P can be carried out by computing the MMSE for all the Nt! possible permutation matrices and choosing the one that provides the minimum value of Equation (22). Alternatively, more efficient ordering algorithms (such as the one described in [22]) can be used.
From the MMSE design of the DF receiver, it is straightforward to obtain the expressions for the ZF DF receiver: it is the limiting case when . The final expressions for the ZF DF filters are exactly the same as before although L and D should be obtained from the Cholesky decomposition of
4.2 MIMO Tomlinson-Harashima precoder design with transmitter noise
Figure 3 shows the block diagram of a MIMO system employing THP. THP is a nonlinear precoding technique made up of a feedforward filter , a feedbackward filter , and a modulo operator, represented in Figure 3 by M(●). The modulo operator is introduced to avoid the increase in transmit power due to the feedback loop [11]. Data symbols sent from the transmitter will be represented by , where denotes the modulation alphabet. The ordering considerably affects the performance of THP and, for this reason, transmit symbols are passed through a permutation filter P. Minimization is carried out under the restriction of B being a spatially causal filter and Etx being the transmitted energy, i.e., , where is the transmitted signal, with v[n] representing the output of the modulo operator. At reception, we assume that all the receive antennas apply the same positive real value denoted by g. These assumptions are necessary in order to arrive at closed-form, unique solutions for the MMSE THP design.
In order to carry out the THP optimization, and taking into account the linear representation of THP [5, 11], the desired signal is denoted by d[n] and it is expressed as
(23)
The received signal under the presence of Tx-noise is rewritten as
(24)
where is the received signal when there is no Tx-noise. At the receivers, the modulo operator is applied again to invert the effect of this operator at the transmitter and the resulting signal is passed through a symbol detector (represented by Q(●) in Figure 3) to produce the detected symbols .
As explained in [5], the MMSE THP design searches for the filtering and permutation matrices that minimize the variance of the error vector
where ε[n] = PTB v[n] - gy[n] is the error vector when there is no Tx-noise.
Since the Tx-noise is independent from the transmitted signal and the Rx-noise, the MSE can be decomposed as
(25)
where is the MSE when there is no Tx-noise, which constitutes the cost function that is minimized in the conventional MMSE design of THP.
Following similar derivations as in [12], the minimization of the MSE cost function in Equation (25), subject to the mentioned constraints, can be carried out from the factorization of
where
(26)
with . The symmetrically permuted Cholesky decomposition of this matrix is
(27)
where L and D are, respectively, unit lower triangular and diagonal matrices. Finally, the MMSE solution for the THP filters that account for the Tx-noise is given by
(28)
The receive scalar weight is directly obtained from the transmit-energy constraint. Assuming that it is real and positive, it is obtained that
(29)
where C
v
is the covariance matrix of v[n], which is diagonal with entries depending on the modulation alphabet [11].
The minimum value for the MSE cost function given by Equation (25) can be obtained by substituting the expressions obtained for the optimum filters and , and for the gain factor . It is easy to show that the final MMSE under the presence of Tx-noise is
(30)
where ξt is given by Equation (26) and D is the diagonal matrix that results from the permuted Cholesky factorization of Equation (27).
As it is done in [12], instead of testing all the possible permutation matrices to find the one that minimizes the cost function of Equation (30), the ordering optimization can be included in the computation of the Cholesky decomposition of Equation (27).
Again, it is straightforward to obtain the expressions for the ZF THP design as the limiting case when ξt → 0. The expressions for the filters and are equal to those obtained for and , respectively, although the matrices P, L, and D should be obtained from the symmetrically permuted Cholesky factorization of
4.3 MIMO vector precoder design with transmitter noise
Figure 4 shows the block diagram of a MIMO system with VP. The transmitter has the freedom to add an arbitrary perturbation signal to the data signal prior to the linear transformation with the filter . This perturbation will be later on removed by the modulo operator M(●) at the receiver. Here, τ denotes a constant that depends on the modulation alphabet.a This constant is associated with the nonlinear modulo operator M(●), defined as
(31)
where ⌊●⌋ denotes the floor operator which gives the largest integer smaller than or equal to the argument. The corresponding fundamental Voronoi region is
which means that the modulo operator constrains the real and imaginary part of x to the interval [-τ/2, τ/2] by adding integer multiples of τ and j τ to the real and imaginary part, respectively.
As it can be seen from Figure 4, the data vector is first superimposed with the perturbation vector a[n], and the resulting vector is then processed by the linear filter F to form the transmitted signal , n = 1,..., NB, where d[n] is the desired signal given by u[n] + a[n] and n is the symbol index in a block size of NB data symbols. The transmit-energy constraint is expressed as since transmit-symbols statistics are unknown.
The weight g in Figure 4 is assumed to be constant throughout the block of NB symbols. Again, note that a common weight for all the receivers is used. Thus, the weighted estimated signal is given by
with . The modulo operator at the receiver compensates the effect of adding the perturbation a[n] at the transmitter.
Since a[n] is discrete, their optimum values cannot be obtained after derivation. The optimization procedure is as follows. We start by fixing a[n], after which x[n] and g are optimized taking into account the transmit power constraint. For these optimum x[n] and g we choose the best a[n] in order to minimize the following MSE criterion [13]
(32)
With Tx-noise being present, the previous MSE cost function can be expressed as
(33)
where is the MSE cost function when the Tx-noise is away. Following an optimization procedure similar to that described in [13], we arrive at the MMSE VP solution given by
(34)
where is directly obtained from the transmit-energy constraint.
By defining the matrix Φt = (HHH + ξtI)-1 and applying the matrix inversion lemma to Equation (34), the MSE cost function given by Equation (33) is reduced to [13]
Since Φt is positive definite, we can use the Cholesky factorization to obtain a lower triangular matrix L and a diagonal matrix D with the following relationship,
Thus, the perturbation signal can be found by means of the following search [13]
(35)
This search can be solved using the Schnorr-Euchner sphere-decoding algorithm [23]. It is interesting to note that THP can be interpreted as a suboptimum approach to VP where a[n] is successively computed.
The ZF constraint , for n = 1,..., NB, leads to similar expressions for and as those obtained for the MMSE VP design when ξ
t
→ 0. Following similar steps as before, the cost function for ZF VP has exactly the same form as that of MMSE VP but considering Φt = (HHH)-1. Finally, the optimum perturbation vectors are found by the following closest point search in a lattice