A new blind algorithm for channel estimation in OFDM-based amplify-and-forward two-way relay networks

Abstract

In this paper, we propose a blind channel estimation algorithm for the amplify-and-forward (AF) two-way relay network (TWRN) which consists of two terminal nodes and one relay node. The orthogonal frequency division multiplexing (OFDM) modulation is adopted for frequency selective channel. Both cyclic prefix (CP) and zero padding (ZP) are considered. The two cascaded channels are estimated in two steps. First, the cascaded channel causing the self-interference is estimated using a proposed power reduction method. Then, the other cascaded channel from source to destination is estimated by subspace method. Closed-form formulas for channel estimates are derived. In addition, we also carry out the theoretical mean square error analysis and derive the approximated Cramer-Rao bounds.

1 Introduction

Research on wireless relay networks became popular since the pioneering work [1] developed low-complexity cooperative diversity strategies. In [1], data streams flow unidirectionally from the source to the relay and then to the destination. This network structure is known as the one-way relay network (OWRN). However, since most communication systems are bidirectional, it is necessary to consider the situation when the source node and the destination node exchange their roles. Such a relay network is known as the two-way relay network (TWRN). In TWRN, the relay treats the received signals in a “network coding”-like manner [2], and the terminals can recover the signal collision since they know their own transmitted signals. As a result, the overall communication rate between two source terminals in TWRN is approximately twice that achieved in OWRN [3].

Despite its throughput advantage, TWRN faces more challenges in terms of transceiver design, relay processing optimization, and transmission protocol development. In [4], the capacity analysis and the achievable rate region for amplify-and-forward (AF) and decode-and-forward (DF) TWRN are explored. In [5], the authors point out that the throughput of AF-TWRN is 1.5 times of DF-TWRN. The distributed space-time code (STC) at relays for both AF-TWRN and DF-TWRN has been developed in [6]. Moreover, the optimal beamforming with full channel knowledge at the multi-antenna relay that maximizes the overall system capacity of AF-TWRN is derived in [7]. In [8], the authors address the problem of robust linear relay precoder and destination equalizer design for multiple-input multiple-output relay systems. In [9], the authors compare several network-coding AF-TWRN and consider imperfect time synchronization. Most existing works on TWRN [2–9] have assumed perfect channel state information (CSI) at the relay node and/or the source terminals. While traditional channel estimation methods can be applied to DF-TWRN, the channel estimation problem for AF-TWRN is more challenging due to the self-interfering signals.

In traditional channel estimation methods for point-to-point systems, they can be divided into two groups: data-aided (DA) [10–17] and non data-aided (blind) [18–26]. In general, DA channel estimation methods differ in the way they interpolate or filter punctual DA least square (DA-LS) channel estimates over data subcarriers. This can be accomplished using time-frequency Wiener filtering [10, 11], which is optimal in the minimum mean square error (MMSE) sense if knowledge of the channel statistics (KCS) is available. On the other hand, channel estimation can be accomplished by elaborating raw estimates in the time domain using a discrete Fourier transform (DFT)-based scheme. In [12], the MMSE channel estimator working in the time domain has been proposed. In order to reduce computational complexity, using the singular value decomposition and several low-rank approximations to the MMSE estimator has been proposed in [13] and [14]. Li et al. [12–14] also require complete KCS. In [15], the authors compare the MMSE approach with maximum likelihood (ML) channel estimation, where complete KCS is not required. This latter approach works well with dense multipath channels and quasi-uniform profiles. In practice, after the inverse DFT (IDFT), not all the channel impulse response (CIR) samples are significant because many may correspond to delays where no propagation channel paths are actually present. Therefore, the authors in [16] exploit this idea to estimate channel. In [17], the authors propose a method to approach the MMSE channel estimation performance, while avoiding the need for a priori KCS.

For blind channel estimation methods, earlier works require either higher order statistics (HOS) of the received data [18] or over-sampling at the receiver [19]. By exploiting linear redundant precoding, only second-order statistics (SOS) of the received data is required and these methods are robust to channel order overestimation [20, 21]. Another popular blind algorithm is the so-called subspace-based algorithm which was originally developed in [19]. The subspace method has simple structure and achieves good performance. In [22], a blind channel identification method by exploiting virtual carriers (VC) is derived. In [23], a generalization in cyclic prefix (CP) systems is proposed. By arranging the received data appropriately, [23] generates a rank-deduction matrix, and thus, subspace method can work. In [24], the authors propose another simpler arrangement of the received data. Pan and Phoong [25] and [26] utilize the repetition method to reduce the number of required received data and consider the existence of VCs.

As in the traditional point-to-point systems, study of channel estimation algorithm is also demanded for AF-TWRN systems [27–34]. DA channel estimation methods for AF-TWRN are proposed in [27–30]. Gao et al. [27] develops an optimal training design for flat-fading environment. The authors also combine their algorithm with orthogonal frequency division multiplexing (OFDM) to estimate the channel impulse responses for frequency selective environment in [28]. The case of multiple-input multiple-output is considered in [29], and [30] provides two channel training algorithms for channel estimation.

On the other hand, [31–34] are blind channel estimation methods. In [31], the authors propose a ML approach to estimate the flat-fading channels blindly, but the transmitted signals are limited to constant modulus modulation. Zhao et al. [32] find a closed-form solution and thus provides a low-complexity ML algorithm. For non-constant modulus modulation, [33] gives an iterative algorithm, which is based on the maximum a posteriori (MAP) approach, and it requires a large number of received blocks. In [34], the authors consider the frequency selective environment. They apply a non-unitary linear precoding at both terminals and derive a blind channel estimation algorithm from SOS of the received signals. However, the use of non-unitary linear precoding leads to degradation in bit error rate (BER) performance.

In this paper, we develop a blind channel estimation algorithm for AF-TWRN under OFDM modulation. Our method consists of two steps. The first step is to estimate the cascaded channel causing the self-interference. Since the terminal knows its own transmitted signal, we choose the method based on power reduction to estimate the channel, which is also named LS method. The self-interference signal can be removed by using the estimated channel. The second step is to estimate the cascaded channel from source to destination. We utilize the rank reduction method, which is also known as subspace-based algorithm [23–26]. This is because subspace methods do not require complete KCS, work well with all multipath channels, and achieve good performance. Closed-form formulas for these two cascaded channel estimates are derived. The theoretical performance analysis and approximated Cramer-Rao bounds (ACRB) are given as well. The proposed method can be applied to both CP-based and zero padding (ZP)-based OFDM systems. Simulation results will be provided to show the performance of the proposed method.

The rest of this paper is organized as follows. The system model for CP-OFDM AF-TWRN is introduced in Section 2. Section 3 describes the proposed algorithm for blind channel estimation. In Section 4, we analyze the performance of the proposed channel estimation methods and the ACRBs. Simulation results are presented in Section 5, and concluding remarks are made in Section 6. The results in Section 3.1 and 3.2 of this paper have appeared in a conference paper [35].

Notation In this paper, E{x} stands for the statistical expectation of the random variable x. The symbols A^T, A^∗, and A^† denote the transpose, the complex conjugate, and the conjugate-transpose of matrix A, respectively. ∥A∥_F is the Frobenius norm of matrix A. If A is a square, tr(A) denotes the trace of matrix A. I_m is the m×m identity matrix, whereas 0 represents an all-zero matrix with appropriate dimension. $\jmath =\sqrt {-1}$ is the imaginary unit. T_m(c) and $\tilde {\mathbf {T}}_{m}(\mathbf {c})$ are two Toeplitz matrices respectively defined as

$$ \mathbf{T}_{m}(\mathbf{c})\triangleq\left.\left[ \begin{array}{cccccc} c_{n}&\cdots&c_{1}&0&\cdots&0\\ 0&c_{n}&\cdots&c_{1}&\ddots&\vdots\\ \vdots&\ddots&\ddots&\ddots&\ddots&0\\ 0&\cdots&0&c_{n}&\cdots&c_{1} \end{array}\right]\right\}m\ \text{rows} $$

(1)

and

$$ \tilde{\mathbf{T}}_{m}(\mathbf{c})\triangleq\underbrace{\left[ \begin{array}{cccc} c_{1}&0&\cdots&0\\ \vdots&c_{1}&\ddots&\vdots\\ c_{n}&\vdots&\ddots&0\\ 0&c_{n}&\ddots&c_{1}\\ \vdots&\ddots&\ddots&\vdots\\ 0&\cdots&0&c_{n}\end{array}\right]}_{m\ \text{columns}}, $$

(2)

where c=[c₁,c₂,…,c_n]^T is an arbitrary vector.

2 System model

Consider a TWRN with two terminal nodes $\mathbb {T}_{1}$ and $\mathbb {T}_{2}$, and one relay node $\mathbb {R}$, as shown in Fig. 1. Each node has one antenna which cannot transmit and receive simultaneously. The channel from $\mathbb {T}_{i}$ to $\mathbb {R}$ is denoted as $\mathbf {f}_{i}=[f_{i,0}^{},f_{i,1}^{},\ldots,f_{i,L}^{}]^{T}$, whereas the one from $\mathbb {R}$ back to $\mathbb {T}_{i}$ is denoted as $\mathbf {g}_{i}=\left [g_{i,0}^{},g_{i,1}^{},\ldots,g_{i,L}^{}\right ]^{T}$ for i=1 and 2. For notational simplicity, we assume that the lengths of f₁, f₂, g₁, and g₂ do not exceed L+1.¹ Similar to most other algorithms, we assume that the channels do not change when the channel estimation is performed.

2.1 OFDM modulation at terminals

Denote the kth OFDM block from $\mathbb {T}_{i}$ as $\mathbf {s}_{k}^{(i)}=\left [s_{k,0}^{(i)},s_{k,1}^{(i)},\ldots,s_{k,N-1}^{(i)}\right ]^{T}$, where N is the OFDM block length. The corresponding time domain signal block is obtained from the normalized IDFT as

$$ \mathbf{x}_{k}^{(i)}=\mathbf{W}^{\dag}\mathbf{s}_{k}^{(i)}=\left[ \begin{array}{cccc} x_{k,0}^{(i)}&x_{k,1}^{(i)}&\cdots&x_{k,N-1}^{(i)} \end{array}\right]^{T}, $$

(3)

where W is the N×N normalized DFT matrix with the (m,n)th entry given by $\frac {1}{\sqrt {N}}e^{-\jmath 2\pi mn/N}$. To maintain the subcarrier orthogonality during the overall transmission, we propose to add a CP of length 2L.² This implicitly requires N≥2L which is nevertheless satisfied by most OFDM systems. Define $\mathbf {x}_{k,cp}^{(i)}=[x_{k,N-2L}^{(i)},\ldots,x_{k,N-1}^{(i)}]^{T}$. The signal sent out from $\mathbb {T}_{i}$ is expressed as $\left [\begin {array}{cc}\mathbf {x}_{k,cp}^{(i)T}&\mathbf {x}_{k}^{(i)T}\end {array}\right ]^{T}$ for i=1 and 2.

2.2 Relay processing

The relay $\mathbb {R}$ receives the signal [34]

$$ \mathbf{r}_{k}=\left[ \begin{array}{c} r_{k,0}^{}\\ r_{k,1}^{}\\ \vdots\\ r_{k,N+2L-1}^{} \end{array}\right] =\sum_{i=1}^{2}\mathbf{T}_{N+2L}(\mathbf{f}_{i})\left[ \begin{array}{l} \mathbf{x}_{k-1,isi}^{(i)}\\ \mathbf{x}_{k,cp}^{(i)}\\ \mathbf{x}_{k}^{(i)}\end{array}\right]+\mathbf{n}_{k,r}, $$

(4)

where $\mathbf {x}_{k-1,isi}^{(i)}$ is the term which causes the inter-symbol interference (ISI):

$$ \mathbf{x}_{k-1,isi}^{(i)}=\left[ \begin{array}{c}x_{k-1,N-L}^{(i)}\\ \vdots\\x_{k-1,N-1}^{(i)} \end{array}\right]. $$

(5)

Moreover, each element in the noise vector n_k,r is assumed to be independent and identically distributed (i.i.d.) zero-mean complex white Gaussian.

We assume that the relay $\mathbb {R}$ employs the amplify-and-forward scheme. It scales r_k by the factor of

$$ \alpha=\sqrt{\frac{P_{r}}{\mathrm{E}\left\{\|\mathbf{r}_{k}\|_{F}^{2}\right\}}}=\sqrt{\frac{P_{r}}{\|\mathbf{f}_{1}\|_{F}^{2}\sigma_{1}^{2}+\|\mathbf{f}_{2}\|_{F}^{2}\sigma_{2}^{2}+\sigma_{n_{r}}^{2}}}, $$

(6)

where P_r is the average transmission power of $\mathbb {R}$. In the second equality, we have made the assumptions that the transmitted signals $\mathbf {x}_{k}^{(1)}$, $\mathbf {x}_{k}^{(2)}$, and the received noise n_k,r are uncorrelated with variances $\sigma _{1}^{2}$, $\sigma _{2}^{2}$, and $\sigma _{n_{r}}^{2}$, respectively. Then, the relay broadcasts αr_k to both terminals.

2.3 Signal reformulation at terminals

Due to symmetry, we only illustrate the processing at $\mathbb {T}_{1}$. The (N+2L)×1 vector received at $\mathbb {T}_{1}$ can be expressed as

$$ \mathbf{y}_{k}=\left[ \begin{array}{c} y_{k,0}^{}\\ y_{k,1}^{}\\ \vdots\\ y_{k,N+2L-1}^{} \end{array}\right] =\mathbf{T}_{N+2L}(\mathbf{g}_{1})\left[ \begin{array}{l}\alpha\mathbf{r}_{k-1,isi}\\ \alpha\mathbf{r}_{k}\end{array}\right]+\mathbf{n}_{k,t}, $$

(7)

where r_k−1,isi is similar to (5)

$$ \mathbf{r}_{k-1,isi}=\left[ \begin{array}{c}r_{k-1,N+L}^{}\\ \vdots\\ r_{k-1,N+2L-1}^{} \end{array}\right], $$

(8)

and each element in the noise vector n_k,t is assumed to be i.i.d. zero-mean complex white Gaussian, with variance $\sigma _{n_{t}}^{2}$. Substituting (4) into (7), we have

$$ \mathbf{y}_{k}\,=\,\mathbf{T}_{N\,+\,2L}(\mathbf{h}_{1})\!\left[\! \begin{array}{l} \mathbf{x}_{k-1,cp}^{(1)}\\ \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\!\right]\!+ \mathbf{T}_{N\,+\,2L}(\mathbf{h}_{2})\!\left[\! \begin{array}{l} \mathbf{x}_{k-1,cp}^{(2)}\\ \mathbf{x}_{k,cp}^{(2)}\\ \mathbf{x}_{k}^{(2)} \end{array}\!\right]\!+\mathbf{n}_{k,e}, $$

(9)

where h₁=α(g₁∗f₁) and h₂=α(g₁∗f₂) with ∗ being the linear convolution between two vectors by the fact that the multiplication of two Toeplitz matrices is still a Toeplitz matrix. The last term n_k,e denotes the equivalent noise

$$ \mathbf{n}_{k,e}=\alpha\mathbf{T}_{N+2L}(\mathbf{g}_{1})\left[ \begin{array}{c} n_{k-1,r}^{}(N+L)\\ \vdots\\ n_{k-1,r}^{}(N+2L-1)\\ \mathbf{n}_{k,r} \end{array}\right]+\mathbf{n}_{k,t}. $$

(10)

When N≫L, n_k,e can be approximated as white noise.

2.4 Data detection at terminals

After removing the first 2L elements of y_k in (9), we obtain a vector of size N:

$$ \bar{\mathbf{y}}_{k}=\mathbf{T}_{N}(\mathbf{h}_{1})\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\right]+ \mathbf{T}_{N}(\mathbf{h}_{2})\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(2)}\\ \mathbf{x}_{k}^{(2)} \end{array}\right]+\bar{\mathbf{n}}_{k,e}, $$

(11)

where $\bar {\mathbf {n}}_{k,e}$ is the last N elements of n_k,e. If the cascaded channel h₁ is known to $\mathbb {T}_{1}$, then the first term on the right-hand side of (11) can be removed since $\mathbb {T}_{1}$ knows its own signal $\mathbf {x}_{k}^{(1)}$. If h₂ is known, the regular OFDM detection can be efficiently performed using fast Fourier transform. So $\mathbb {T}_{1}$ can recover the data from $\mathbb {T}_{2}$ if both h₁ and h₂ are available. Hence, our goal is to estimate h₁ and h₂. Below, we will show how to blindly estimate these two cascaded channels from the received signal y_k.

3 Proposed method for channel estimation

In this paper, we assume that $\mathbf {x}_{k}^{(1)}$ and $\mathbf {x}_{k}^{(2)}$ are uncorrelated. Moreover, the transmitted signals and the noises are uncorrelated as well. Under these two assumptions, we propose an algorithm to estimate h₁ and h₂ blindly. Though our derivations are based on CP-OFDM system, the results can be also extended to ZP-OFDM system. The details will be discussed later.

3.1 The estimation of h ₁

Let us look at the received vector $\bar {\mathbf {y}}_{k}$ in (11). Notice that $\mathbf {x}_{k}^{(1)}$ is known at $\mathbb {T}_{1}$. If we have a perfect estimate of h₁, then the first term at the right-hand side of (11) can be eliminated completely from $\bar {\mathbf {y}}_{k}$. Due to uncorrelatedness of $\mathbf {x}_{k}^{(1)}$, $\mathbf {x}_{k}^{(2)}$, and $\bar {\mathbf {n}}_{k,e}$, the power of $\bar {\mathbf {y}}_{k}$ will be reduced when $\mathbf {x}_{k}^{(1)}$ is eliminated from $\bar {\mathbf {y}}_{k}$. Based on this power reduction, we are able to derive a closed-form formula for an estimate of the (2L+1)×1 vector h₁, as shown below.

Define a cost function

$$ J\left(\hat{\mathbf{h}}_{1}\right)=\mathrm{E}\left\{\left\|\bar{\mathbf{y}}_{k}-\mathbf{T}_{N}\left(\hat{\mathbf{h}}_{1}\right)\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\right]\right\|{~}_{F}^{2}\right\}, $$

(12)

where $\bar {\mathbf {y}}_{k}$ is the N×1 vector in (11) and $\hat {\mathbf {h}}_{1}$ is an estimate of h₁. Substituting (11) into (12), we get

$$ {\begin{aligned} J\left(\hat{\mathbf{h}}_{1}\right) & = \mathrm{E}\left\{\left\|\left(\mathbf{T}_{N}(\mathbf{h}_{1})-\mathbf{T}_{N}\left(\hat{\mathbf{h}}_{1}\right)\right)\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\right]\right.\right.\\ & \left.\left.\quad + \mathbf{T}_{N}(\mathbf{h}_{2})\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(2)}\\ \mathbf{x}_{k}^{(2)} \end{array}\right]+\bar{\mathbf{n}}_{k,e}\right\|{~}_{F}^{2}\right\}\\ &=\mathrm{E}\left\{\left\|\mathbf{T}_{N}\left(\mathbf{h}_{1}-\hat{\mathbf{h}}_{1}\right)\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\right]\right\|{~}_{F}^{2}\right\}\\ & \quad + \mathrm{E}\left\{\left\|\mathbf{T}_{N}(\mathbf{h}_{2})\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(2)}\\ \mathbf{x}_{k}^{(2)} \end{array}\right]\right\|{~}_{F}^{2}\right\}+\mathrm{E}\left\{\left\|\bar{\mathbf{n}}_{k,e}\right\|{~}_{F}^{2}\right\}. \end{aligned}} $$

(13)

Using the assumptions mentioned above to simplify the expression, we have

$$ {\begin{aligned} J\left(\hat{\mathbf{h}}_{1}\right)&=N\left(\sigma_{1}^{2}\left\|\mathbf{h}_{1}-\hat{\mathbf{h}}_{1}\right\|{~}_{F}^{2}\! +\!\sigma_{2}^{2}\left\|\mathbf{h}_{2}\left\|{~}_{F}^{2}\,+\,|\alpha|^{2}\sigma_{n_{r}}^{2}\right\|\mathbf{g}_{1}\right\|{~}_{F}^{2}\,+\, \sigma_{n_{t}}^{2}\!\right)\\ &\geq N\left(\sigma_{2}^{2}\left\|\mathbf{h}_{2}\left\|{~}_{F}^{2}+|\alpha|^{2}\sigma_{n_{r}}^{2}\right\|\mathbf{g}_{1}\right\|{~}_{F}^{2}+\sigma_{n_{t}}^{2}\right). \end{aligned}} $$

(14)

Obviously, the cost function has the minimum if and only if $\|\mathbf {h}_{1}-\hat {\mathbf {h}}_{1}\|{~}_{F}^{2}=0$, or equivalently, $\hat {\mathbf {h}}_{1}=\mathbf {h}_{1}$. Assume that $\mathbb {T}_{1}$ has collected K blocks. For mean-ergodic processes, the ensemble average (or statistical average) can be well approximated by the time average:

$$ {\begin{aligned} \bar{J}\left(\hat{\mathbf{h}}_{1}\right)&= \frac{1}{K}\sum_{k=0}^{K-1}\left\|\bar{\mathbf{y}}_{k}-\mathbf{T}_{N}\left(\hat{\mathbf{h}}_{1}\right)\left[ \begin{array}{l} \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)}\end{array}\right] \right\|{~}_{F}^{2}\\ &=\frac{1}{K}\sum_{k=0}^{K-1}\left\|\bar{\mathbf{y}}_{k}-\sqrt{N}\mathbf{W}^{\dag}\mathbf{D}\left(\mathbf{s}_{k}^{(1)}\right)\mathbf{W}_{2L+1}\hat{\mathbf{h}}_{1}\right\|{~}_{F}^{2}, \end{aligned}} $$

(15)

where $\mathbf {D}\left (\mathbf {s}_{k}^{(1)}\right)$ is a diagonal matrix with the elements of $\mathbf {s}_{k}^{(1)}$ on the main diagonal, and W_2L+1 is the first 2L+1 columns of the DFT matrix W. Let

$$ \mathbf{y}=\left[ \begin{array}{cccc} \bar{\mathbf{y}}_{0}^{T} & \bar{\mathbf{y}}_{1}^{T} & \cdots & \bar{\mathbf{y}}_{K-1}^{T} \end{array}\right]^{T} $$

(16)

and

$$ \mathbf{S}=\left[ \begin{array}{cccc} \mathbf{D}\left(\mathbf{s}_{0}^{(1)}\right) & \mathbf{D}\left(\mathbf{s}_{1}^{(1)}\right) & \cdots & \mathbf{D}\left(\mathbf{s}_{K-1}^{(1)}\right) \end{array}\right]^{T}. $$

(17)

Then, (15) can be rewritten as

$$ \bar{J}\left(\hat{\mathbf{h}}_{1}\right)=\frac{1}{K}\left\|\mathbf{y}-\sqrt{N}\left(\mathbf{I}_{K}\otimes\mathbf{W}^{\dag}\right) \mathbf{S}\mathbf{W}_{2L+1}\hat{\mathbf{h}}_{1}\right\|{~}_{F}^{2}, $$

(18)

where the symbol ⊗ denotes the Kronecker product. The least squares solution of (18) can be calculated as

$$ {\begin{aligned} \hat{\mathbf{h}}_{1}=\frac{1}{\sqrt{N}}\left(\mathbf{W}_{2L+1}^{\dag}\mathbf{S}^{\dag}\mathbf{S}\mathbf{W}_{2L+1}\right)^{-1} \mathbf{W}_{2L+1}^{\dag}\mathbf{S}^{\dag}\left(\mathbf{I}_{K}\otimes\mathbf{W}\right)\mathbf{y}. \end{aligned}} $$

(19)

When K>>1, we have $\mathbf {S}^{\dag }\mathbf {S}\approx K\sigma _{1}^{2}\mathbf {I}_{N}$ as the modulation symbols are statistically independent. In this case, (19) can be approximated as

$$ \hat{\mathbf{h}}_{1}\approx\frac{1}{\sqrt{N}K\sigma_{1}^{2}}\mathbf{W}_{2L+1}^{\dag}\sum_{k=0}^{K-1} \left(\mathbf{s}_{k}^{(1)}\right)^{*}\odot\left(\mathbf{W}\bar{\mathbf{y}}_{k}\right), $$

(20)

where the symbol ⊙ denotes the Hadamard product. Notice that there is no scalar ambiguity in the estimation of h₁ since $\mathbf {s}_{k}^{(1)}$ and $\bar {\mathbf {y}}_{k}$ are known at $\mathbb {T}_{1}$.

3.2 The estimation of h ₂

In order to estimate the (2L+1)×1 vector h₂, we first remove the self-interfering signal from the received vector. Define

$$ \mathbf{z}_{k}=\mathbf{y}_{k}-\mathbf{T}_{N+2L}\left(\hat{\mathbf{h}}_{1}\right)\left[ \begin{array}{l} \mathbf{x}_{k-1,cp}^{(1)}\\ \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\right]. $$

(21)

Assuming that the estimation of h₁ is perfect (i.e., $\hat {\mathbf {h}}_{1}=\mathbf {h}_{1}$), from (9) and (21), we have

$$ \mathbf{z}_{k}=\mathbf{T}_{N+2L}(\mathbf{h}_{2})\left[ \begin{array}{l} \mathbf{x}_{k-1,cp}^{(2)}\\ \mathbf{x}_{k,cp}^{(2)}\\ \mathbf{x}_{k}^{(2)} \end{array}\right]+\mathbf{n}_{k,e}. $$

(22)

Note that the vector z_k is simply the received vector in a usual CP-OFDM system with channel h₂ and transmitted vector $\mathbf {x}_{k}^{(2)}$. Many blind estimation methods have been proposed for the estimation of h₂ from z_k. Below, we will adopt the subspace-based algorithm in [24]. Define the re-modulated vector

$$ \tilde{\mathbf{z}}_{k}=\left[ \begin{array}{c} z_{k-1,2L}^{}\\ \vdots\\ z_{k-1,N+2L-1}^{}\\ z_{k,0}^{}\\ \vdots\\ z_{k,2L-1}^{} \end{array}\right], $$

(23)

where $z_{k,i}^{}$ is the ith entry of z_k. That is, $\tilde {\mathbf {z}}_{k}$ is a (N+2L)×1 vector formed by the last N entries of z_k−1 and the first 2L entries of z_k. Next, we construct the vector

$$ \mathbf{v}_{k}=\mathbf{z}_{k}-\tilde{\mathbf{z}}_{k}. $$

(24)

Substituting (9) and (21) into (24), we have

$$ \mathbf{v}_{k}=\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2})\underbrace{\left(\left[ \begin{array}{c} \mathbf{x}_{k,cp}^{(2)}\\ x_{k,0}^{(2)}\\ \vdots\\ x_{k,N-2L-1}^{(2)} \end{array}\right] -\mathbf{x}_{k-1}^{(2)}\right)}_{\triangleq\mathbf{d}_{k}}+\boldsymbol{\eta}_{k}, $$

(25)

where $\tilde {\mathbf {T}}_{N}(\cdot)$ is defined in (2) and η_k is colored noise. The covariance matrix of η_k is [24]

$$\begin{array}{*{20}l} \mathrm{E}\{\boldsymbol{\eta}_{k}\boldsymbol{\eta}_{k}^{\dag}\}=\sigma_{n_{e}}^{2}\underbrace{\left[ \begin{array}{ccc} 2\mathbf{I}_{2L}&\mathbf{0}&-\mathbf{I}_{2L}\\ \mathbf{0}&2\mathbf{I}_{N-2L}&\mathbf{0}\\ -\mathbf{I}_{2L}&\mathbf{0}&2\mathbf{I}_{2L} \end{array}\right]}_{\triangleq\mathbf{R}_{w}}, \end{array} $$

where $\sigma _{n_{e}}^{2}$ is the average power of n_k,e. It can be verified that

$$ \mathbf{R}_{w}^{-1/2}=\left[ \begin{array}{ccc} c_{1}\mathbf{I}_{2L} & \mathbf{0}&c_{2}\mathbf{I}_{2L}\\ \mathbf{0} & \frac{1}{\sqrt{2}}\mathbf{I}_{N-2L}&\mathbf{0}\\ c_{2}\mathbf{I}_{2L}&\mathbf{0} & c_{1} \mathbf{I}_{2L} \end{array}\right] $$

(26)

with

$$\begin{array}{*{20}l} \begin{array}{ccc} c_{1}=\sqrt{\frac{2/3+\sqrt{1/3}}{2}} & \text{and} & c_{2}=\sqrt{\frac{2/3-\sqrt{1/3}}{2}}. \end{array} \end{array} $$

Carrying out the whitening process on v_k, we get the whitened vector $\mathbf {v}_{k}^{(w)}=\mathbf {R}_{w}^{-1/2}\mathbf {v}_{k}$ and its covariance matrix is

$$ \mathbf{R}_{v}^{(w)}=\mathbf{R}_{w}^{-1/2}\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2}) \mathbf{R}_{d}\tilde{\mathbf{T}}_{N}^{\dag}(\mathbf{h}_{2})\mathbf{R}_{w}^{-1/2}+\sigma_{n_{e}}^{2}\mathbf{I}_{N+2L}, $$

(27)

where $\mathbf {R}_{d}=\mathrm {E}\left \{\mathbf {d}_{k}\mathbf {d}_{k}^{\dag }\right \}$ is the covariance matrix of d_k defined in (25). A necessary condition that R_d has full rank is that $\mathbb {T}_{1}$ collects K≥N blocks. Utilizing eigenvalue decomposition, (27) can be computed as

$$ \mathbf{R}_{v}^{(w)}=\mathbf{U}_{s}\mathbf{\Sigma}\mathbf{U}_{s}^{\dag}+\sigma_{n_{e}}^{2}\mathbf{U}_{o}\mathbf{U}_{o}^{\dag}, $$

(28)

where Σ is an N×N diagonal matrix and the (N+2L)×N matrix U_s spans the signal subspace. On the other hand, the (N+2L)×2L matrix U_o spans the noise subspace. That is,

$$ \mathbf{U}_{o}^{\dag}\mathbf{R}_{w}^{-1/2}\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2})=\mathbf{0}. $$

(29)

Let

$$ \begin{array}{cc} \mathbf{J}_{i}=\left[ \begin{array}{c} \mathbf{0}_{i\times(2L+1)}\\ \mathbf{I}_{2L+1}\\ \mathbf{0}_{(N-1-i)\times(2L+1)} \end{array}\right] & \text{for }i=0,1,\ldots,N-1. \end{array} $$

(30)

Then, (29) can be rewritten as

$$ \underbrace{\left[ \begin{array}{c} \mathbf{U}_{o}^{\dag}\mathbf{R}_{w}^{-1/2}\mathbf{J}_{0}\\ \vdots\\ \mathbf{U}_{o}^{\dag}\mathbf{R}_{w}^{-1/2}\mathbf{J}_{N-1} \end{array}\right]}_{\triangleq\mathbf{U}}\mathbf{h}_{2} =\mathbf{0}. $$

(31)

Hence, we can estimate h₂ (up to a scalar ambiguity) by calculating the eigenvector corresponding to the smallest eigenvalue of U^†U.

In summary, our algorithm is as follows.

• 1. Estimate h₁ by (20).

• 2. Eliminate the interference from $\mathbb {T}_{1}$ by (21).

• 3. Calculate $\mathbf {v}_{k}^{(w)}=\mathbf {R}_{w}^{-1/2}\mathbf {v}_{k}$ by (24) and (26) and obtain the (N+2L)×2L matrix U_o spanning the noise subspace by eigenvalue decomposition.

• 4. Estimate h₂ (up to a scalar ambiguity) by calculating the eigenvector corresponding to the smallest eigenvalue of U^†U.

3.3 A note on the identifiability issue

Note that the estimate of h₁ is unique because the cost function in (14) has a unique minimum at $\hat {\mathbf {h}}_{1}=\mathbf {h}_{1}$. The second channel h₂ is estimated by the subspace method. Let us look at the vector z_k in (22). When the self-interfering signal is completely eliminated, the remaining part z_k is identical to the case of single-input single-output (SISO) CP-OFDM system in [24]. The identifiability issue of this method has been studied in [24]. It has been shown that if h_2,0≠0, then the vector h₂ is uniquely determined (up to a scalar ambiguity).

3.4 Comparison with an existing work

A blind channel estimation algorithm in OFDM-based TWRN was proposed in [34]. Comparing our method with that in [34], there are two major differences. One is that [34] requires a precoding matrix P, where

$$\begin{array}{*{20}l} \mathbf{P}\mathbf{P}^{\dag}=\left[ \begin{array}{cccc} 1 & \theta & \cdots & \theta\\ \theta & 1 & \ddots & \vdots\\ \vdots & \ddots & \ddots & \theta\\ \theta & \cdots & \theta & 1 \end{array}\right]. \end{array} $$

A necessary condition on θ is $-\frac {1}{N-1}\leq \theta \leq 1$. In other words, the kth transmitted vector from $\mathbb {T}_{i}$ is the precoded vector $\mathbf {P}\mathbf {s}_{k}^{(i)}$ instead of $\mathbf {s}_{k}^{(i)}$. Notice that for θ≠0, P is not a unitary matrix. The channel noise can be amplified when the receiver performs the operation P⁻¹. It was shown in [34] that when θ increases from 0 to 1, the mean square error (MSE) of channel estimate decreases. Due to noise amplification, larger θ does not necessarily yield smaller BER, so there exists a compromise between channel estimation error and BER. Another difference between our method and [34] is that there is a 2×2 ambiguity matrix in [34], or equivalently, there are four ambiguity scalars. On the other hand, there is only one ambiguity scalar in our algorithm. In terms of complexity, we can see that the main complexity of our method is the computation of the eigenvalue decomposition of an N×N matrix in (28), whereas the eigenvalue decomposition in [34] is for a (2L+1)×(2L+1) matrix. Hence, our method is more complicated than [34].

3.5 Repeated use of the remodulated vector v _k

To obtain U_o in (28), $\mathbb {T}_{1}$ has to collect K≥N blocks. In OFDM systems, N is usually large. The number of blocks, K, needed for the channel estimation is large. In order to reduce the required block number K, we can use the repetition method proposed in [23, 25, 26]. Define the repetition parameter Q and form the matrix $\tilde {\mathbf {T}}_{Q}(\mathbf {v}_{k})$, where v_k is defined in (24). According to (25), $\tilde {\mathbf {T}}_{Q}(\mathbf {v}_{k})$ can be represented as

$$ \tilde{\mathbf{T}}_{Q}(\mathbf{v}_{k})=\tilde{\mathbf{T}}_{N+Q-1}(\mathbf{h}_{2})\tilde{\mathbf{T}}_{Q}(\mathbf{d}_{k})+\tilde{\mathbf{T}}_{Q}(\boldsymbol{\eta}_{k}). $$

(32)

It was shown in [25] that $\tilde {\mathbf {T}}_{Q}(\boldsymbol {\eta }_{k})$ is colored noise, and its covariance matrix can be calculated as

$$ {\begin{aligned} \mathrm{E}\left\{\tilde{\mathbf{T}}_{Q}\left(\boldsymbol{\eta}_{k}\right)\tilde{\mathbf{T}}_{Q}^{\dag}(\boldsymbol{\eta}_{k})\right\}&=\sigma_{n_{e}}^{2}\sum_{q=1}^{Q} \left[\!\begin{array}{ccc} \mathbf{0}_{(q-1)\times(q-1)} & \mathbf{0} & \mathbf{0}\\ \mathbf{0} & \mathbf{R}_{w} & \mathbf{0}\\ \mathbf{0} & \mathbf{0} & \mathbf{0}_{(Q-q) \times (Q-q)} \end{array}\!\right]\\ &=\sigma_{n_{e}}^{2}\mathbf{E}\mathbf{\Lambda}\mathbf{E}^{\dag}, \end{aligned}} $$

(33)

where we have applied the eigenvalue decomposition in the second equality. Therefore, we need to whiten the matrix $\tilde {\mathbf {T}}_{Q}(\mathbf {v}_{k})$ by EΛ^−1/2E^†. Since each vector v_k is repeated Q times in (32), the required number of blocks becomes $K\geq \frac {N-1}{Q}+1$ blocks [23]. Collecting these K blocks, we can follow the procedure in (28)–(31) to estimate h₂ (up to a scalar ambiguity).

3.6 Multiple relay nodes

The extension to the case of multiple relay nodes is straight forward as shown in Fig. 2. Suppose that we have M relay nodes $\mathbb {R}_{1},\mathbb {R}_{2},\ldots,\mathbb {R}_{M}$. Let the channels from $\mathbb {T}_{i}$ to $\mathbb {R}_{m}$ be denoted as $\mathbf {f}_{i}^{(m)}$ and the channels from $\mathbb {R}_{m}$ to $\mathbb {T}_{i}$ be denoted as $\mathbf {g}_{i}^{(m)}$. Then, (4) becomes

$$ \mathbf{r}_{k}^{(m)}=\sum_{i=1}^{2}\mathbf{T}_{N+2L}\left(\mathbf{f}_{i}^{(m)}\right)\left[ \begin{array}{l} \mathbf{x}_{k-1,isi}^{(i)}\\ \mathbf{x}_{k,cp}^{(i)}\\ \mathbf{x}_{k}^{(i)} \end{array}\right] +\mathbf{n}_{k,r}^{(m)}, $$

(34)

Fig. 2

System configuration for multiple relay network. It shows the case of multiple relay nodes. Suppose that we have M relay nodes $\mathbb {R}_{1},\mathbb {R}_{2},\ldots,\mathbb {R}_{M}$. Let the channels from $\mathbb {T}_{i}$ to $\mathbb {R}_{m}$ be denoted as $\mathbf {f}_{i}^{(m)}$ and the channels from $\mathbb {R}_{m}$ to $\mathbb {T}_{i}$ be denoted as $\mathbf {g}_{i}^{(m)}$

where $\mathbf {r}_{k}^{(m)}$ is the signal received by relay node $\mathbb {R}_{m}$ and $\mathbf {n}_{k,r}^{(m)}$ is the noise at $\mathbb {R}_{m}$. When $\mathbb {T}_{1}$ receives the signal, (7) becomes

$$ \mathbf{y}_{k}=\sum_{m=1}^{M}\mathbf{T}_{N+2L}\left(\mathbf{g}_{1}^{(m)}\right)\left[ \begin{array}{l} \alpha_{m}\mathbf{r}_{k-1,isi}^{(m)}\\ \alpha_{m}\mathbf{r}_{k}^{(m)} \end{array}\right]+\mathbf{n}_{k,t}, $$

(35)

where α_m is the amplification scalar in the relay node $\mathbb {R}_{m}$. Combining (34) with (35), the received vector at $\mathbb {T}_{1}$ continues to have the form given in (9), but now the cascaded channels are $\mathbf {h}_{1}=\sum _{m=1}^{M}\alpha _{m}\left (\mathbf {g}_{1}^{(m)}\ast \mathbf {f}_{1}^{(m)}\right)$ and $\mathbf {h}_{2}=\sum _{m=1}^{M}\alpha _{m}\left (\mathbf {g}_{1}^{(m)}\ast \mathbf {f}_{2}^{(m)}\right)$, and the equivalent noise n_k,e becomes

$${\begin{aligned} \mathbf{n}_{k,e} \,=\, \sum_{m=1}^{M} \alpha_{m}\mathbf{T}_{N + 2L}\left(\mathbf{g}_{1}^{(m)}\right)\left[\! \begin{array}{c} n_{k-1,r}^{(m)}(N+L)\\ \vdots\\ n_{k-1,r}^{(m)}(N+2L-1)\\ \mathbf{n}_{k,r}^{(m)} \end{array}\right]\,+\,\mathbf{n}_{k,t}. \end{aligned}} $$

Hence, the above methods can be applied to the case of multiple relay nodes.

3.7 The case of ZP-OFDM systems

The proposed method can be also applied to TWRN ZP-OFDM system. In this case, 2L zeros are padded at the end of $\mathbf {x}_{k}^{(i)}$ in (3) instead of adding the cyclic prefix of length 2L. Due to the padded zeros, the received vector does not suffer from ISI. Therefore, (9) can be rewritten as

$$ \mathbf{y}_{k}=\tilde{\mathbf{T}}_{N}(\mathbf{h}_{1})\mathbf{x}_{k}^{(1)}+\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2})\mathbf{x}_{k}^{(2)}+\mathbf{n}_{k,e}. $$

(36)

To estimate h₁, we modify the cost function in (12) as

$$ J\left(\hat{\mathbf{h}}_{1}\right)=\mathrm{E}\left\{\left\|\mathbf{y}_{k}-\tilde{\mathbf{T}}_{N}\left(\hat{\mathbf{h}}_{1}\right)\mathbf{x}_{k}^{(1)}\right\|{~}_{F}^{2}\right\}. $$

(37)

Following a procedure similar to (12)–(20), an estimate of h₁ can be obtained by

$$ {\begin{aligned} \hat{\mathbf{h}}_{1}=\frac{1}{\frac{N}{\sqrt{N+2L}}K\sigma_{1}^{2}}\tilde{\mathbf{W}}_{2L+1}^{\dag}\sum_{k=0}^{K-1} \left(\tilde{\mathbf{W}}_{N}\mathbf{x}_{k}^{(1)}\right)^{*}\odot\left(\tilde{\mathbf{W}}\mathbf{y}_{k}\right), \end{aligned}} $$

(38)

where $\tilde {\mathbf {W}}$ is the (N+2L)×(N+2L) normalized DFT matrix with the (m,n)th entry given by $\frac {1}{\sqrt {N+2L}}e^{-\jmath 2\pi mn/(N+2L)}$, whereas $\tilde {\mathbf {W}}_{2L+1}$ and $\tilde {\mathbf {W}}_{N}$ are respectively the first 2L+1 and N columns of $\tilde {\mathbf {W}}$.

Assume that the estimation of h₁ is perfect so that we can eliminate the interference from $\mathbb {T}_{1}$. Similar to (21), define

$$ \mathbf{z}_{k}=\mathbf{y}_{k}-\tilde{\mathbf{T}}_{N}\left(\hat{\mathbf{h}}_{1}\right)\mathbf{x}_{k}^{(1)}. $$

(39)

Substituting (36) into (39), we have

$$ \mathbf{z}_{k}=\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2})\mathbf{x}_{k}^{(2)}+\mathbf{n}_{k,e}. $$

(40)

This form is similar to (25), so we can follow the procedure in (25)–(31) to estimate h₂. Note that the noise n_k,e is (almost) white. Similar to the previous discussion, a necessary condition is K≥N. To reduce the limitation of a large K, we exploit the repetition method in [26]. That is, we utilize $\tilde {\mathbf {T}}_{Q}(\mathbf {z}_{k})$ instead of z_k to estimate h₂, and the necessary condition becomes $K\geq \frac {N-1}{Q}+1$. In this case, the noise term $\tilde {\mathbf {T}}_{Q}(\mathbf {n}_{k,e})$ is colored (though n_k,e is white) and the covariance matrix is [26]

$${\begin{aligned} \mathrm{E}\left\{\tilde{\mathbf{T}}_{Q}\left(\mathbf{n}_{k,e}\right)\tilde{\mathbf{T}}_{Q}^{\dag}\left(\mathbf{n}_{k,e}\right)\right\}\,=\, \sigma_{n_{e}}^{2} \mathbf{D}\left(\left[1,2,\ldots, \underbrace{Q',\ldots,Q'}_{|N+2L-Q|+1},\ldots,2,1\right]\right), \end{aligned}} $$

where D(·) is defined in (15) and Q^′= min{Q,N+2L}. Therefore, we need to whiten the matrix $\tilde {\mathbf {T}}_{Q}(\mathbf {z}_{k})$ by

$$\begin{array}{*{20}l} \mathbf{D}\left(\left[1,\frac{1}{\sqrt{2}},\ldots,\underbrace{\frac{1}{\sqrt{Q'}},\ldots,\frac{1}{\sqrt{Q'}}}_{|N+2L-Q|+1},\ldots,\frac{1}{\sqrt{2}},1\right]\right). \end{array} $$

Following a procedure similar to Section 3.5, one can obtain a blind estimate of h₂ (up to a scalar ambiguity).

4 Analysis of MSE performance and Cramer-Rao bound

In this section, we will derive the theoretical MSE about channel estimation for h₁ and h₂ respectively. In the following analysis, we assume that the channel taps are uncorrelated and the transmitted vectors $\mathbf {x}_{k}^{(i)}$ are also uncorrelated for different k or i.

4.1 The analysis of h ₁ estimate

In the estimation of h₁, we regard the signal from $\mathbb {T}_{2}$ as interference. Since (20) is the least squares solution of (18), the difference between $\hat {\mathbf {h}}_{1}$ and h₁ can be calculated as

$$\begin{array}{*{20}l} \Delta\mathbf{h}_{1}&\triangleq\hat{\mathbf{h}}_{1}-\mathbf{h}_{1}\notag\\ &=\frac{1}{\sqrt{N}K\sigma_{1}^{2}}\mathbf{W}_{2L+1}^{\dag}\sum_{k=0}^{K-1}\left(\mathbf{s}_{k}^{(1)}\right)^{*}\odot\left(\mathbf{W}\boldsymbol{\xi}_{k}\right), \end{array} $$

(41)

where ξ_k denotes the interference and noise. From (11), we have

$$ \boldsymbol{\xi}_{k}=\mathbf{C}(\mathbf{h}_{2})\mathbf{x}_{k}^{(2)}+\bar{\mathbf{n}}_{k,e}, $$

(42)

where C(h₂) is an N×N circulant matrix having $\left [\begin {array}{cc}\mathbf {h}_{2}^{T}&\mathbf {0}_{1\times (N-2L-1)}\end {array}\right ]^{T}$as its first column. Assuming that $\mathbf {x}_{k}^{(2)}$ and n_k,e are uncorrelated, the covariance matrix of ξ_k can be computed as

$$\begin{array}{*{20}l} \mathrm{E}\left\{\boldsymbol{\xi}_{k}\boldsymbol{\xi}_{k}^{\dag}\right\} & = \sigma_{2}^{2}\mathbf{C}\left(\mathbf{h}_{2}\right)\mathbf{C}^{\dag}(\mathbf{h}_{2})+\sigma_{n_{e}}^{2}\mathbf{I}_{N}\notag\\ &=\mathbf{W}^{\dag}\left(\sigma_{2}^{2}\mathbf{D}\left(\mathbf{h}_{2,f}\right)\mathbf{D}^{\dag}\left(\mathbf{h}_{2,f}\right)+\sigma_{n_{e}}^{2}\mathbf{I}_{N}\right)\mathbf{W}, \end{array} $$

(43)

where D(h_2,f) is the N×N diagonal matrix with diagonal entries from the N×1 frequency response vector $\mathbf {h}_{2,f}=\sqrt {N}\mathbf {W}_{2L+1}\mathbf {h}_{2}$. Then, the covariance matrix of Δh₁ can be computed as

$$ {\begin{aligned} \mathbf{R}_{\Delta\mathbf{h}_{1}}&\triangleq\mathrm{E}\left\{\Delta\mathbf{h}_{1}\Delta\mathbf{h}_{1}^{\dag}\right\} &\\ &=\frac{1}{NK\sigma_{1}^{2}}\mathbf{W}_{2L + 1}^{\dag}\left(\sigma_{2}^{2}\mathbf{D}\left(\mathbf{h}_{2,f}\right)\mathbf{D}^{\dag}\left(\mathbf{h}_{2,f}\right) + \sigma_{n_{e}}^{2}\mathbf{I}_{N}\right)\mathbf{W}_{2L +1}\\ &=\frac{1}{NK\sigma_{1}^{2}}\left(\sigma_{2}^{2}\mathbf{A}+\sigma_{n_{e}}^{2}\mathbf{I}_{2L+1}\right), \end{aligned}} $$

(44)

where A is a (2L+1)×(2L+1) Toeplitz and Hermitian matrix with $A_{m,n}=\sum _{l=0}^{2L+m-n}h_{2,l}h_{2,l-m+n}^{*}$ if m≤n and $A_{m,n}=\sum _{l=m-n}^{2L}h_{2,l}h_{2,l-m+n}^{*}$ if m≥n. Therefore, the theoretical MSE can be calculated as

$$ {\begin{aligned} \mathrm{E}\left\{\left\|\Delta\mathbf{h}_{1}\right\|{~}_{F}^{2}\right\}=tr\left\{\mathbf{R}_{\Delta\mathbf{h}_{1}}\right\} =\frac{2L+1}{NK}\frac{\sigma_{2}^{2}\|\mathbf{h}_{2}\|{~}_{F}^{2}+\sigma_{n_{e}}^{2}}{\sigma_{1}^{2}}, \end{aligned}} $$

(45)

where $tr\left \{\mathbf {R}_{\Delta \mathbf {h}_{1}}\right \}$ is the sum of the diagonal elements of $\mathbf {R}_{\Delta \mathbf {h}_{1}}\phantom {\dot {i}\!}$. Define the signal-to-noise ratio (SNR) as

$$\begin{array}{*{20}l} \text{SNR}\triangleq\frac{\sigma_{2}^{2}}{\sigma_{n_{e}}^{2}}= \frac{\sigma_{2}^{2}}{\alpha^{2}\sigma_{n_{r}}^{2}\|\mathbf{g}_{1}\|{~}_{F}^{2}+\sigma_{n_{t}}^{2}}, \end{array} $$

(46)

where the second equality is obtained by using (10). Then, (45) can be written as

$$\begin{array}{*{20}l} \mathrm{E}\left\{\left\|\Delta\mathbf{h}_{1}\right\|{~}_{F}^{2}\right\}= \frac{2L+1}{NK}\frac{\sigma_{2}^{2}}{\sigma_{1}^{2}}\left(\|\mathbf{h}_{2}\|{~}_{F}^{2}+\frac{1}{\text{SNR}}\right). \end{array} $$

(47)

Note from the above equation that the MSE is proportional to the signal power from $\mathbb {T}_{2}$ but inversely proportional to the signal power from $\mathbb {T}_{1}$ and the number of the received signal blocks. Moreover, for high SNR, the MSE floors at the value of $\frac {2L+1}{NK}\frac {\sigma _{2}^{2}}{\sigma _{1}^{2}}\|\mathbf {h}_{2}\|{~}_{F}^{2}$.

4.2 The analysis of h ₂ estimate

During the estimation of h₂ in Section 3.2, it is assumed that the estimate of h₁ is perfect. However, the estimation error Δh₁ will affect the accuracy of the estimation of h₂. From (21), if Δh₁≠0, the interference and noise terms can be written as

$$ \mathbf{T}_{N+2L}(\Delta\mathbf{h}_{1})\left[ \begin{array}{l} \mathbf{x}_{k-1,cp}^{(1)}\\ \mathbf{x}_{k,cp}^{(1)}\\ \mathbf{x}_{k}^{(1)} \end{array}\right]+\mathbf{n}_{k,e}. $$

(48)

Next, we look at v_k in (25). Following the procedure (21)–(26), the whitened vector $\mathbf {v}_{k}^{(w)}$ now becomes

$$\begin{array}{*{20}l} \mathbf{v}_{k}^{(w)}=\mathbf{R}_{w}^{-1/2}\mathbf{v}_{k}=\mathbf{R}_{w}^{-1/2}\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2})\mathbf{d}_{k}+\boldsymbol{\zeta}_{k}, \end{array} $$

where

$$ {\begin{aligned} \boldsymbol{\zeta}_{k}\,=\,\mathbf{R}_{w}^{-1/2}\tilde{\mathbf{T}}_{N}(\Delta\mathbf{h}_{1})1\left(\!\left[\! \begin{array}{c} \mathbf{x}_{k,cp}^{(1)}\\ x_{k,0}^{(1)}\\ \vdots\\ x_{k,N-2L-1}^{(1)} \end{array}\right] -\mathbf{x}_{k-1}^{(1)}\!\right)\,+\,\mathbf{R}_{w}^{-1/2}\boldsymbol{\eta}_{k}. \end{aligned}} $$

(49)

Recall from the subspace method in Section 3.2 that the estimate $\hat {\mathbf {h}}_{2}$ is obtained from the noise subspace U_o in (28). Let λ₁≤λ₂≤⋯≤λ_N+2L be the eigenvalues of $\mathbf {R}_{v}^{(w)}$. The noise subspace U_o is the eigenspace corresponding to the smallest 2L eigenvalues λ₁,λ₂,…,λ_2L. The error vector ζ_k can cause two effects: (i) it perturbs the noise subspace U_o and (ii) it also perturbs the eigenvalues, i.e., λ₁+Δλ₁,λ₂+Δλ₂,…,λ_N+2L+Δλ_N+2L. Note that λ_2L belongs to the noise subspace U_o and λ_2L+1 belongs to the signal subspace U_s. Their difference λ_2L+1−λ_2L is usually large. Nevertheless, the perturbation on eigenvalues may lead to the case λ_2L+Δλ_2L>λ_2L+1+Δλ_2L+1, especially when the SNR is low. In this case, the noise subspace will be polluted by the signal subspace and this will cause a large error in the estimation of h₂. Below, we derive the MSE by studying the following two cases separately.

Case I: λ_2L+Δλ_2L<λ_2L+1+Δλ_2L+1

In this case, we can exploit the first-order approximation of the perturbation to U_o. In [24], the channel estimation error has been derived. However, the theoretical MSE derived in [24] is based on white noise. As the noise ζ_k is colored, the formula derived in [24] is not applicable. For the case of colored noise ζ_k, we have derived a new formula and the theoretical MSE of the h₂ estimate can be calculated as

$$ {\begin{aligned} \mathrm{E}\left\{\!\left\|\Delta\mathbf{h}_{2}\right\|{~}_{F}^{2}\right\}\,=\, \frac{1}{2K\sigma_{2}^{2}}tr\left\{\!\mathbf{U}^{\sharp}\left(\mathbf{I}_{N}\otimes\mathbf{U}_{o}^{\dag}\mathbf{R}_{\zeta}\mathbf{U}_{o}\right) \left(\mathbf{U}^{\dag}\right)^{\sharp}\right\}, \end{aligned}} $$

(50)

where $\Delta \mathbf {h}_{2}\triangleq \hat {\mathbf {h}}_{2}-\mathbf {h}_{2}$, U^♯ is the Moore-Penrose pseudoinverse matrix of U defined in (31), and $\mathbf {R}_{\zeta }\triangleq \mathrm {E}\{\boldsymbol {\zeta }_{k}\boldsymbol {\zeta }_{k}^{\dag }\}$ is the covariance matrix of ζ_k and it can be written as

$$ \mathbf{R}_{\zeta}=2\sigma_{1}^{2}\mathbf{R}_{w}^{-1/2}\mathrm{E} \left\{\tilde{\mathbf{T}}_{N}(\Delta\mathbf{h}_{1})\tilde{\mathbf{T}}_{N}^{\dag}(\Delta\mathbf{h}_{1})\right\}\mathbf{R}_{w}^{-1/2}\!+ \!\sigma_{n_{e}}^{2}\!\mathbf{I}_{N+2L}. $$

(51)

Notice that $\tilde {\mathbf {T}}_{N}(\Delta \mathbf {h}_{1})$ can be rewritten as

$$\begin{array}{*{20}l} \tilde{\mathbf{T}}_{N}(\Delta\mathbf{h}_{1})=\left[ \begin{array}{cccc} \mathbf{J}_{0}\Delta\mathbf{h}_{1} & \mathbf{J}_{1}\Delta\mathbf{h}_{1} & \cdots & \mathbf{J}_{N-1}\Delta\mathbf{h}_{1} \end{array}\right], \end{array} $$

where J_i is defined in (30). Hence, (51) can be rewritten as

$$ {\begin{aligned} \mathbf{R}_{\zeta}=2\sigma_{1}^{2}\mathbf{R}_{w}^{-1/2} \left(\sum_{i=0}^{N-1}\mathbf{J}_{i}\mathbf{R}_{\Delta\mathbf{h}_{1}}\mathbf{J}_{i}^{T}\right)\mathbf{R}_{w}^{-1/2}+\sigma_{n_{e}}^{2}\mathbf{I}_{N+2L}, \end{aligned}} $$

(52)

where $\mathbf {R}_{\Delta \mathbf {h}_{1}}\phantom {\dot {i}\!}$ is defined in (44).

Case II: λ_2L+Δλ_2L≥λ_2L+1+Δλ_2L+1

In this case, our algorithm cannot find the accurate noise subspace U_o because it has been polluted by signal subspace U_s. Thus, we assume that the eigenvector of U corresponding to the smallest eigenvalue is random and uncorrelated to the true cascaded channel h₂. Define this unit-norm eigenvector as $\tilde {\mathbf {h}}_{2}$. The scalar ambiguity can be calculated as $\alpha =\left (\tilde {\mathbf {h}}_{2}^{\dag }\mathbf {h}_{2}\right)/\left (\tilde {\mathbf {h}}_{2}^{\dag }\tilde {\mathbf {h}}_{2}\right) =\tilde {\mathbf {h}}_{2}^{\dag }\mathbf {h}_{2}$. That is,

$$ \hat{\mathbf{h}}_{2}=\tilde{\mathbf{h}}_{2}\alpha=\tilde{\mathbf{h}}_{2}\left(\tilde{\mathbf{h}}_{2}^{\dag}\mathbf{h}_{2}\right). $$

(53)

The estimation error is $\Delta \mathbf {h}_{2}=\hat {\mathbf {h}}_{2}-\mathbf {h}_{2}=\left (\tilde {\mathbf {h}}_{2}\tilde {\mathbf {h}}_{2}^{\dag }-\mathbf {I}_{2L+1}\right)\mathbf {h}_{2}$. Hence, the theoretical MSE of the h₂ estimate can be calculated as

$$\begin{array}{*{20}l} \mathrm{E}\left\{\left\|\Delta\mathbf{h}_{2}\right\|{~}_{F}^{2}\right\}= \mathbf{h}_{2}^{\dag}\mathrm{E}\left\{\left(\tilde{\mathbf{h}}_{2}\tilde{\mathbf{h}}_{2}^{\dag}-\mathbf{I}_{2L+1}\right)^{2}\right\}\mathbf{h}_{2}. \end{array} $$

(54)

Since the unit-norm vector $\tilde {\mathbf {h}}_{2}$ is assumed to be random, $\mathrm {E}\left \{\tilde {\mathbf {h}}_{2}\tilde {\mathbf {h}}_{2}^{\dag }\right \}$ can be approximated as $\frac {1}{2L+1}\mathbf {I}_{2L+1}$. Therefore, (54) can be written as

$$\begin{array}{*{20}l} \mathrm{E}\left\{\left\|\Delta\mathbf{h}_{2}\right\|{~}_{F}^{2}\right\}=\frac{2L}{2L+1}\|\mathbf{h}_{2}\|{~}_{F}^{2}. \end{array} $$

(55)

Overall MSE: Utilizing Bayes’ theorem, the theoretical MSE of the h₂ estimate can be written as

$$ {\begin{aligned} \mathrm{E}\left\{\left\|\Delta\mathbf{h}_{2}\right\|{~}_{F}^{2}\right\}&=P_{err}\mathrm{E}\big\{\!\!\left.\Delta\mathbf{h}_{2}\|{~}_{F}^{2}\right|\lambda_{2L}+ \Delta\lambda_{2L}\\ & \geq\lambda_{2L+1}+\Delta\lambda_{2L+1}\big\}\\ & \quad + (1-P_{err}) \mathrm{E} \big\{\!\!\left.\|\Delta\mathbf{h}_{2}\|{~}_{F}^{2}\right|\lambda_{2L}+\Delta\lambda_{2L}\\ &<\lambda_{2L+1}+\Delta\lambda_{2L+1}\big\}, \end{aligned}} $$

(56)

where P_err is the probability of λ_2L+Δλ_2L≥λ_2L+1+Δλ_2L+1 and it can be expressed by

$$ P_{err}=Q\left(\sqrt{\frac{K}{2}}\frac{\lambda_{2L+1}-\lambda_{2L}}{\sigma_{n_{e}}^{2}}\right), $$

(57)

where Q(·) is the Q-function:

$$\begin{array}{*{20}l} Q(x)=\frac{1}{\sqrt{2\pi}}\int_{x}^{\infty} e^{-\frac{u^{2}}{2}}du. \end{array} $$

The derivation of P_err is given in Appendix Appendix A. Substituting (50) and (55) into (56), the theoretical MSE of the h₂ estimate can be represented as

$$ {\begin{aligned} \mathrm{E}\left\{\left\|\Delta\mathbf{h}_{2}\right\|{~}_{F}^{2}\right\}&=P_{err}\frac{2L}{2L+1}\|\mathbf{h}_{2}\|{~}_{F}^{2}\\ &\quad+(1-P_{err})\frac{1}{2K\sigma_{2}^{2}}tr\left\{\mathbf{U}^{\sharp} \left(\mathbf{I}_{N}\otimes\mathbf{U}_{o}^{\dag}\mathbf{R}_{\zeta}\mathbf{U}_{o}\right) \left(\mathbf{U}^{\dag}\right)^{\sharp}\right\}, \end{aligned}} $$

(58)

where R_ζ is given in (52).

4.3 Approximated Cramer-Rao bound

When we estimate h₁, the signal from $\mathbb {T}_{2}$ can be viewed as interference, and the signal from $\mathbb {T}_{1}$ can be seen as pilot. To simplify the derivation, we assume that ξ_k in (42) is white. Hence, an ACRB of h₁ estimation is [36]

$$ \text{ACRB}_{1}=\frac{\sigma_{\xi}^{2}}{N}tr\left\{\left(\left(\mathbf{S}\mathbf{W}_{2L+1}\right)^{\dag}\mathbf{S}\mathbf{W}_{2L+1}\right)^{-1}\right\}, $$

(59)

where W_2L+1 and S are defined in (15) and (17), respectively, and $\sigma _{\xi }^{2}$ is the average power of ξ_k. From (43), we have $\sigma _{\xi }^{2}=\sigma _{2}^{2}\|\mathbf {h}_{2}\|{~}_{F}^{2}+\sigma _{n_{e}}^{2}$. Therefore, (59) can be simplified as

$$ \text{ACRB}_{1}=\frac{2L+1}{NK}\frac{\sigma_{2}^{2}\|\mathbf{h}_{2}\|{~}_{F}^{2}+\sigma_{n_{e}}^{2}}{\sigma_{1}^{2}}. $$

(60)

Notice that this form is the same as (45).

Next, we consider the ACRB of h₂. In [24], the authors have derived an ACRB and concluded that the ACRB is the same as the channel estimation MSE. Hence, from (50), an ACRB of h₂ estimation is

$$ \text{ACRB}_{2}=\frac{1}{2K\sigma_{2}^{2}}tr\left\{\mathbf{U}^{\sharp}\left(\mathbf{I}_{N}\otimes\mathbf{U}_{o}^{\dag}\mathbf{R}_{\zeta}\mathbf{U}_{o}\right) \left(\mathbf{U}^{\dag}\right)^{\sharp}\right\}. $$

(61)

In the derivations of the ACRBs, the noises are assumed to be white even though they are actually colored. Therefore, the ACRBs in (60) and (61) are in general larger than or equal to the true Cramer-Rao bounds.

5 Simulation results

In the simulation, we consider a TWRN with one relay node. The channel taps $f_{i,l}^{}$ and $g_{i,l}^{}$ are generated as independent and identically distributed zero-mean complex Gaussian random variables with variances equal to 1/9. The order of these channels is L=8, so the order of the cascaded channels is 2L=16. The channels are normalized so that $\|\mathbf {f}_{1}\|{~}_{F}^{2}=\|\mathbf {f}_{2}\|{~}_{F}^{2}=\|\mathbf {g}_{1}\|{~}_{F}^{2}=\|\mathbf {g}_{2}\|{~}_{F}^{2}=1$. The channel does not change while the channel estimation is performed. The channel noise is additive white Gaussian noise (AWGN), and the transmission symbols are 16-QAM with gray code. The size of the DFT matrix is N=64, and the length of CP is 2L=16. In all plots, we set $\sigma _{1}^{2}=\sigma _{2}^{2}$ and $\sigma _{n_{r}}^{2}=\sigma _{n_{t}}^{2}$. The SNR is defined in (46), and the normalized MSE is defined as

$$\begin{array}{*{20}l} \begin{array}{cc} \frac{1}{M_{c}}\sum_{m=1}^{M_{c}}\frac{\|\hat{\mathbf{h}}_{i}^{(m)}-\mathbf{h}_{i}\|{~}_{F}^{2}}{\|\mathbf{h}_{i}\|{~}_{F}^{2}}&\text{for }i=\text{1 and 2,} \end{array} \end{array} $$

where $\hat {\mathbf {h}}_{i}^{(m)}$ represents the estimated h_i in the mth trial. M_c=2000 denotes the total number of Monte-Carlo trials.

First, we look at the MSE performance of the proposed methods. The number of received blocks is K=500. In Fig. 3, we plot the normalized MSEs for h₁ and h₂. The “simulation” curves of h₁ and h₂ are obtained by (20) and (31), respectively, whereas the “theory” curves of h₁ and h₂ are calculated by (47) and (58), respectively. Moreover, we also display the ACRBs of h₁ and h₂ according to (60) and (61). From Fig. 3, it can be seen that the simulated result, the theoretical MSE, and the ACRB of h₁ is close. Moreover, the proposed method can give a good estimate of h₁, even at very low SNR of 0 dB. One can see that the MSE floors at $\frac {2L+1}{NK}=5.3\times 10^{-4}$ at high SNR, and this confirms our analysis in (47). For h₂, the MSE performance is worse than that of h₁ for SNR< 25 dB, but the MSE of h₂ floors at a much smaller value of 1.2×10⁻⁵. This flooring happens at very high SNR, and the estimation error of h₁ affects the accuracy of h₂ estimate. The gap between numerical and theoretical results is small at high SNR. At low SNR, the gap between simulation result and ACRB becomes very large, but the theoretical curve is still close to the numerical curve. Recall that the theoretical MSE value is a combination of two cases in Section 4.2, and the ACRB of h₂ in (61) is equal to the theoretical MSE when we do not consider the perturbation on eigenvalues, i.e., case II in Section 4.2 (case II usually happens at low SNR). Therefore, the difference between the theoretical MSE and ACRB of h₂ at low SNR is caused by the perturbation of eigenvalues. From Fig. 3, we conclude that the change of eigenvalue sequence dominates the performance degradation at low SNR. In addition, the assumption of white noise in the derivation of ACRB also affects the accuracy, especially when the SNR is low.

Fig. 3

Comparison of the numerical and theoretical normalized MSE for K=500. We plot the normalized MSEs for h₁ and h₂. The “simulation” curves of h₁ and h₂ are obtained by (20) and (31), respectively, whereas the “Theory” curves of h₁ and h₂ are calculated by (47) and (58), respectively. Moreover, we also display the ACRBs of h₁ and h₂ according to (60) and (61). The number of received blocks is K=500

Next, we compare the performances of our method with the method proposed by Liao et al. in [34]. As mentioned in Section 3.4, Liao’s algorithm has a compromise between channel estimation error and BER. The parameter θ in Liao’s algorithm is set to 0.2, 0.4, 0.6, and 0.8. From [34], it is found that θ=0.4 yields a good BER performance when SNR=25 dB. In Figs. 4 and 5, the number of received blocks is K=500. Figure 4 shows the MSE performances. Since the MSEs of h₁ and h₂ by Liao’s algorithm are the same, we plot one MSE curve only. From the figure, we see that as θ increases from 0.2 to 0.8, the MSE of Liao’s algorithm decreases. For the estimation of h₁, our method is better than Liao’s methods for θ=0.2 and 0.4, but worse than that for θ=0.6 and θ=0.8. As we will see in Fig. 5, the BER performance for θ=0.8 is not good due to severe noise amplification. For h₂, Liao’s method is better at low SNR whereas our method is better at high SNR. In Fig. 5, we show BER performances. Zero-forcing equalizers are used at the receiver. The “perfect compensation” represents the case that the channel taps are perfectly known at the receiver. It is seen that among the four curves of θ=0.2, 0.4, 0.6, 0.8, Liao’s method has the best BER performance when θ is set as 0.4 for SNR=25 dB. Though the MSE of Liao’s method is the smallest when θ=0.8, its BER performance is not good due to the noise amplification problem of the precoding matrix. These results are matched with [34]. From Fig. 5, we see that the proposed algorithm outperforms Liao’s methods when SNR≥15 dB, and the performance of our method is close to the perfect compensation.

Fig. 4

Comparison of the normalized MSE for K=500. We compare the normalized MSE performances of our method with the method proposed by Liao et al. in [34]. The parameter θ in Liao’s algorithm is set to 0.2, 0.4, 0.6, and 0.8. The number of received blocks is K=500

Fig. 5

Comparison of the BER for K=500. We compare the BER performances of our method with the method proposed by Liao et al. in [34]. The parameter θ in Liao’s algorithm is set to 0.2, 0.4, 0.6, and 0.8. The number of received blocks is K=500

Figures 6 and 7 show the simulation results when the number of blocks is K=50. In this case, K<N, and thus, the estimation of h₂ by (31) does not work. We exploit the repetition method discussed in Section 3.5 to solve this issue. We set the repetition parameter Q=10, and the necessary condition $K\geq \frac {N-1}{Q}+1$ is satisfied. In Fig. 6, the MSE performance is shown. We can observe that the repetition method is extremely useful when the terminal receives few blocks. On the other hand, h₁ estimation by (20) and Liao’s algorithm are based on the power reduction, so there is no limitation on the number of blocks K. From the figure, we see that the proposed method outperforms Liao’s algorithms with θ=0.2 and 0.4 for all SNR. In Fig. 7, the performance is measured by BER. It can be seen that the proposed algorithm performs better than Liao’s method when SNR≥15 dB. Comparing Fig. 7 with Fig. 5, we find that the BER performance degrades when K reduces from 500 to 50. This is due to the larger channel estimation errors for K=50 and imperfect interference cancelation by h₁ using (21).

Fig. 6

Comparison of the normalized MSE for K=50. We compare the normalized MSE performances of our method with the method proposed by Liao et al. in [34]. The parameter θ in Liao’s algorithm is set to 0.2, 0.4, 0.6, and 0.8. The number of received blocks is K=50

Fig. 7

Comparison of the BER for K=50. We compare the BER performances of our method with the method proposed by Liao et al. in [34]. The parameter θ in Liao’s algorithm is set to 0.2, 0.4, 0.6, and 0.8. The number of received blocks is K=50

Finally, we compare the proposed algorithm for CP-OFDM and ZP-OFDM systems. In Fig. 8, the solid curves and the dashed curves represent the MSEs for CP-OFDM and ZP-OFDM, respectively. We can find that the performances are almost the same. In other words, our method works well for both CP and ZP systems.

Fig. 8

Comparison of the normalized MSE for CP-OFDM and ZP-OFDM systems for K=500. We compare the proposed algorithm for CP-OFDM and ZP-OFDM systems. The solid curves and the dashed curves represent the MSEs for CP-OFDM and ZP-OFDM, respectively. The number of received blocks is K=500

6 Conclusions

In this paper, we propose a blind channel estimation method in OFDM-based amplify-and-forward two-way relay networks. The first cascaded channel h₁ is estimated by the power reduction method whereas the second cascaded channel h₂ is estimated by the subspace method. Close-form formulas are derived. We also analyze the theoretical performance and derive the ACRBs for channel estimation. Our algorithm can be applied to both CP-OFDM and ZP-OFDM systems, and it can use repetition method to handle the case of few received blocks. Simulation results verify our analysis.

7 Appendix A

7.1 A proof of (57)

To simplify our derivation, we utilize the fact that this condition usually occurs at low SNR. From (47) and the simulation in Section 5, it can be seen that the estimate of h₁ is still quite accurate at low SNR, so the second term n_k,e in (48) is dominant. Let λ₁≤λ₂≤⋯≤λ_N+2L be the eigenvalues of $\mathbf {R}_{v}^{(w)}$ and the corresponding unit-norm eigenvectors are respectively b₁,b₂,…,b_N+2L. By (27) and (52), $\mathbf {R}_{v}^{(w)}$ can be expressed by

$$\begin{array}{*{20}l} \mathbf{R}_{v}^{(w)}&=2\sigma_{2}^{2}\mathbf{R}_{w}^{-1/2}\tilde{\mathbf{T}}_{N}(\mathbf{h}_{2})\tilde{\mathbf{T}}_{N}^{\dag}(\mathbf{h}_{2})\mathbf{R}_{w}^{-1/2}\notag\\ &+2\sigma_{1}^{2}\mathbf{R}_{w}^{-1/2}\left(\sum_{i=0}^{N-1}\mathbf{J}_{i}\mathbf{R}_{\Delta\mathbf{h}_{1}}\mathbf{J}_{i}^{T}\right)\mathbf{R}_{w}^{-1/2}+\sigma_{n_{e}}^{2}\mathbf{I}_{N+2L}. \end{array} $$

(62)

Since the received signals are finite and the second term in (48) is dominant, we have the following approximation:

$$\begin{array}{*{20}l} \frac{1}{K}\sum_{k=1}^{K}&\mathbf{v}_{k}^{(w)}\left(\mathbf{v}_{k}^{(w)}\right)^{\dag}\approx\mathrm{E}\{\mathbf{R}_{w}^{-1/2}\mathbf{v}_{k}\mathbf{v}_{k}^{\dag}\mathbf{R}_{w}^{-1/2}\}\notag\\ &\quad+\underbrace{\frac{1}{K}\sum_{k=1}^{K}\mathbf{R}_{w}^{-1/2}\boldsymbol{\eta}_{k}\boldsymbol{\eta}_{k}^{\dag}\mathbf{R}_{w}^{-1/2}-\sigma_{n_{e}}^{2}\mathbf{I}_{N+2L}}_{\triangleq\mathbf{N}}, \end{array} $$

(63)

and the corresponding eigenvalues become λ₁+Δλ₁,λ₂+Δλ₂,…,λ_N+2L+Δλ_N+2L. Notice that N is a Hermitian matrix with mean 0. For large K, the central limit theorem indicates that the diagonal entries of N are real normal distributed and the other entries are circularly symmetric complex normal distributed [37]. According to the result in [38], all entries of N have the same variance $\frac {1}{K}\sigma _{n_{e}}^{4}$.

From matrix theory [39], the eigenvalue perturbation Δλ_i can be approximated as $\mathbf {b}_{i}^{\dag }\mathbf {N}\mathbf {b}_{i}$. Then, the mean is

$$ \mathrm{E}\{\Delta\lambda_{i}\}=\mathrm{E}\left\{\mathbf{b}_{i}^{\dag}\mathbf{N}\mathbf{b}_{i}\right\}= \mathbf{b}_{i}^{\dag}\mathrm{E}\{\mathbf{N}\}\mathbf{b}_{i}=0, $$

(64)

and the variance is

$$ {\begin{aligned} \mathrm{E}\left\{\left|\Delta\lambda_{i}\right|^{2}\right\} &= \mathrm{E}\left\{\mathbf{b}_{i}^{\dag}\mathbf{N}\mathbf{b}_{i}\mathbf{b}_{i}^{\dag}\mathbf{N}^{\dag}\mathbf{b}_{i}\right\}\\ &=\mathrm{E}\left\{\sum_{j}\sum_{l}\sum_{m}\sum_{n}b_{i}^{*}(j)N(j,l)b_{i}(l)b_{i}(m)N^{*}(m,n)b_{i}^{*}(n)\right\}\\ &=\sum_{j}\sum_{l}\sum_{m}\sum_{n}b_{i}^{*}(j)b_{i}(l)b_{i}(m)b_{i}^{*}(n)\mathrm{E}\{N(j,l)N^{*}(m,n)\}. \end{aligned}} $$

(65)

Because $\mathbf {R}_{w}^{-1/2}\boldsymbol {\eta }_{k}$ is white, all entries of N are uncorrelated, so the last term E{N(j,l)N^∗(m,n)} is equal to $\frac {1}{K}\sigma _{n_{e}}^{4}\delta (j-m)\delta (l-n)$, where δ(·) is the Kronecker delta function. Thus, (65) can be rewritten as

$$\begin{array}{*{20}l} \mathrm{E}\left\{\left|\Delta\lambda_{i}\right|^{2}\right\}&=\frac{1}{K}\sigma_{n_{e}}^{4}\sum_{j}\sum_{l}b_{i}^{*}(j)b_{i}(l)b_{i}(j)b_{i}^{*}(l)\notag\\ &=\frac{1}{K}\sigma_{n_{e}}^{4}\left|\mathbf{b}_{i}^{\dag}\mathbf{b}_{i}\right|^{2}=\frac{1}{K}\sigma_{n_{e}}^{4}. \end{array} $$

(66)

The last equality holds since $\mathbf {b}_{i}^{\dag }\mathbf {b}_{i}=1$ for i=1,2,…,N+2L.

Notice that N is normal distributed and b_i is constant, so the random variable Δλ_i is normal distributed as well. It means that the probability of λ_2L+Δλ_2L≥λ_2L+1+Δλ_2L+1 can be computed as

$$\begin{array}{*{20}l} P_{err}&\triangleq\text{Pr}\left\{\lambda_{2L}+\Delta\lambda_{2L}\geq\lambda_{2L+1}+\Delta\lambda_{2L+1}\right\}\notag\\ &=Q\left(\frac{\lambda_{2L+1}-\lambda_{2L}}{\sqrt{\mathrm{E}\left\{\left|\Delta\lambda_{2L+1}\right|^{2}\right\}+ \mathrm{E}\left\{\left|\Delta\lambda_{2L}\right|^{2}\right\}}}\right). \end{array} $$

(67)

Substituting (66) into (67), we obtain (57).