mm-Wave channel estimation with accelerated gradient descent algorithms

Abstract

The availability of millimeter wave (mm-Wave) band in conjunction with massive multiple-input-multiple-output (MIMO) technology is expected to boost the data rates of the fifth-generation (5G) cellular systems. However, in order to achieve high spectral efficiencies, an accurate channel estimate is required, which is a challenging task in massive MIMO. By exploiting the small number of paths that characterize the mm-Wave channel, the estimation problem can be solved by compressed-sensing (CS) techniques. In this paper, we propose a novel CS channel estimation method based on the accelerated gradient descent with adaptive restart (AGDAR) algorithm exploiting a ℓ₁-norm approximation of the sparsity constraint. Moreover, a modified re-weighted compressed-sensing (RCS) technique is considered that iterates AGDAR using a weighted version of the ℓ₁-norm term, where weights are adapted at each iteration. We also discuss the impact of cell sectorization and tracking on the channel estimation algorithm. We compare the proposed solutions with existing channel estimations with an extensive simulation campaign on downlink third-generation partnership project (3GPP) channel models.

1 Introduction

Due to its huge spectrum availability, the millimeter wave (mm-Wave) band is currently considered for the fifth generation (5G) of cellular networks [1–3]. The high attenuation incurred at those frequencies imposes the use of multiple antennas at each device, typically resulting in massive multiple-input-multiple-output (MIMO) systems, giving rise to various challenges. We focus here on channel estimation that is needed for proper transmit beamforming. In fact, the least square (LS) estimate using short training sequences and limited transmit power (to reduce overhead in massive MIMO systems) is not accurate enough for capacity achieving beamforming. However, the mm-Wave MIMO channel comprises a small number of dominant clusters of paths and even with many antennas a small set of parameters characterizes the entire channel. This induces a sparsity of the mm-Wave channel matrix when transformed by a Fourier transform into the so-called virtual channel, and compressed-sensing (CS) techniques can be used for channel estimation.

Various solutions have been proposed for channel estimation in mm-Wave communication systems, and the reader may refer to [4] for their survey. Part of the literature has considered transceivers with hybrid beamformers (cascade of beamformers before and after the digital to analog converters): the joint optimization of both training and estimation has been pursued in [5] for these structures using a feedback channel. Orthogonal matching pursuit (OMP) solutions have been considered with both single path [6, 7] and multiple-path cancelation [8, 9]. In [10], an enhanced approach for generating the beamforming codebook has been proposed, using the continuous basis pursuit (CBP) method, while [11] considers a fast iterative shrinkage-thresholding algorithm (FISTA) approach.

For fully digital beamformers, [12] considers the sparse channel estimation as a least absolute shrinkage and selection operator (LASSO) problem. In [13], OMP is used to estimate the channel by iteratively detecting and canceling paths from the virtual channel estimate. In [14], a basis pursuit denoise (BPDN) approach is suggested where a weighted version of ℓ₁-norm term is considered in the LASSO problem and weights are iteratively adapted. A sparsity adaptive matching pursuit (SSAMP) approach is instead used in [15], while in [16] the LASSO problem is solved by applying a generalized approximate message passing (GAMP) algorithm exploiting the Bernoulli-Gaussian distribution of paths in the virtual channel.

In this paper, we propose a novel sparse channel estimation method based on the accelerated gradient descent with adaptive restart (AGDAR) algorithm [17]. Focusing on a scenario where the receiver obtains first the LS estimate of the narrowband mm-Wave MIMO channel, we relax the sparse optimization problem using LASSO, wherein the ℓ₀-norm is replaced by the ℓ₁-norm. We apply then the AGDAR algorithm [17] to solve the sparse channel estimation problem. In order to further enhance the channel estimation procedure, a re-weighted ℓ₁-norm problem is considered leading to the re-weighted compressed-sensing (RCS) algorithm [18], which iterates AGDAR with different weights of the ℓ₁-norm term. We also discuss the impact of cell sectorization and channel tracking on the channel estimation algorithm. We compare the proposed solutions with OMP solutions [6, 8, 13]. With respect to the rest of the literature we reduce the complexity (with respect to the random search of A-LASSO in [12]), we swap the objective functions and the constraints with respect to [14], effectively minimizing the mean square error (MSE) and providing details on the implementation of the optimization algorithm. Compared to [15], we use different algorithms (AGDAR and RCS instead of SSAMP) that trade-off between sparsity and noise reduction. Lastly, we consider a single user and a static pilot transmission for the initial estimate, while [16] considers the adaptation of the transmit and receive beamformers to allow channel estimation simultaneously for more users. An extensive simulation campaign on third-generation partnership project (3GPP) channel models [3] for a downlink scenario has been conducted to show the merits of the proposed approach in terms of both estimate MSE and computational complexity.

The rest of the paper is organized as follows. We introduce the system model in Section 2, providing the description of both the mm-Wave channel model and the existing OMP solutions. The sparse channel estimation problem is introduced in Section 3, together with a discussion on sectorization and channel tracking. The proposed AGDAR technique is described in Section 4, together with the refined RCS approach. Numerical results are presented in Section 5 to assess the performance of the considered techniques in a 5G scenario, before conclusions are driven in Section 6.

2 System model

We consider a massive MIMO narrowband communication system with N_t antennas at the transmitter and N_r antennas at the receiver. This models indifferently either the uplink or the downlink of a cellular communication system. Let $\mathbf {H} \in \mathbb {C}^{N_{\mathrm {r}} \times N_{\mathrm {t}} }$ be the channel matrix with complex entries. Antennas are organized into either uniform linear arrays (ULAs) [19] or uniform planar arrays (UPAs) [20] at both the transmitter and receiver: ULA antennas are uniformly spaced along the z axis while UPA antennas are uniformly tiled over the yz-plane¹. For the sake of a clearer explanation in the main body of the paper, we only provide derivations for ULA, while we report in Appendix A the results for an UPA with D₂×D₃=N_t transmit antennas and D₀×D₁=N_r receive antennas.

We indicate with L the number of paths for the signal from the transmitter to the receiver, so that the channel matrix entries can written as

$$ [ \boldsymbol{H}]_{i_{1},i_{2}} =\sum\limits_{l=1}^{L} \alpha_{l} e^{2\pi \eta_{1}^{l}i_{1}j}e^{2\pi \eta_{2}^{l}i_{2}j}\,, $$

(1)

where i₁=0,…,N_r−1, i₂=0,…,N_t−1, $|\eta _{d}^{l}|\leq \frac {\delta }{\lambda }$, d=1,2, λ is the carrier wave length, δ is the antenna spacing, and α_l is the l-path amplitude including path loss, shadowing, and fading. Note that parameters $\eta _{d}^{l}$ are related to the angles of departure and arrival of the l-th path. By assuming δ≤λ/2, we have $\eta _{d}^{l} \in \left [-\frac 12, \frac 12\right ]$. The statistics of each parameter depend on the considered propagation scenario, and various relevant cases can be found for example in the 3GPP mm-Wave channel model [3] including channel models with clustered sub-paths [3], where L becomes the total number of (sub-)paths from all clusters. Typically, in mm-Wave systems, the number of paths (or sub-paths) L is small [21].

Figure 1 shows an example of receiver with N_r=3 receive antennas and a single path arriving at the antennas with an angle 𝜗 from a distance D: in this case $\eta _{1}^{1} = \frac {\delta }{\lambda } \cos \vartheta$ and $\alpha _{1} = \frac {1}{D^{2}} e^{-j2\pi \frac {D}{\lambda }}$.

Fig. 1

ULA receiver. Single path received by ULA with N_r=3 antennas at distance D from the source. In this case $\eta _{1}^{1} = \frac {\delta }{\lambda } \cos \vartheta$ and $\alpha _{1} = \frac {1}{D^{2}} e^{-j2\pi \frac {D}{\lambda }}$

2.1 LS estimate

The considered channel estimation techniques in this paper are all based on the LS channel estimate, briefly summarized here.

The set of N_t training symbols² transmitted with the N_t antennas are collected into the N_t×N_t matrix S, assumed here to be unitary. The corresponding received N_t×N_r matrix signal is

$$ \boldsymbol{R} = \boldsymbol{H} \boldsymbol{S} + \boldsymbol{N}\,, $$

(2)

where $\boldsymbol {N} \in \mathbb {C}^{N_{\mathrm {r}} \times N_{\mathrm {t}} }$ is the noise matrix with independent and identically distributed (iid) zero-mean complex Gaussian entries, each with power σ². The LS channel estimate at the receiver is obtained as [22]

$$ \boldsymbol{H}' = \boldsymbol{R}\boldsymbol{S}^{-1} = \boldsymbol{H}+\boldsymbol{N}', $$

(3)

where N^′ is a matrix with iid zero-mean complex Gaussian entries having power σ² (thanks to the unitary property of S). Note that this estimation procedure may yield a significant overhead for the transmission of training sequences only when the number of transmit antennas grows large [23], since the number of transmitted symbols (the columns of S) is N_t. We will further address this problem in Section 3.

2.2 OMP methods

We will compare our channel estimation algorithm with two OMP techniques: single peak cancelation (SPC) [6] and joint peak cancelation (JPC) [8]. Both methods use the Fourier transform of the channel matrix, in what is usually denoted as virtual channel or angular domain representation [24, Sec. 7.3.3]. With reference to ULA, let $I_{n}(x)=\frac {\sin (\pi x)}{n \sin \left ({\pi \frac {x}{n}}\right)}$ be the 1D-periodic sinc function and let

$$ \begin{aligned} \left[\boldsymbol{W}\left(\boldsymbol{\Omega}^{l}\right)\right]_{\boldsymbol{f}} \,=\, &\left[ \frac{N_{\mathrm{r}}}{M_{1}} I_{N_{\mathrm{r}}}\left(\frac{N_{\mathrm{r}}\left(f_{1}+\Omega_{1}^{l}\right)}{M_{1}}\right) e^{j\pi \left(f_{1}+\Omega_{1}^{l}\right)\frac{N_{\mathrm{r}}-1}{M_{1}}}\right] \times \\ &\left[ \frac{N_{\mathrm{t}}}{M_{2}} I_{N_{\mathrm{t}}}\left(\frac{N_{\mathrm{t}}(f_{2}+\Omega_{2}^{l})}{M_{2}}\right) e^{j\pi \left(f_{2}+\Omega_{2}^{l}\right)\frac{N_{\mathrm{t}}-1}{M_{2}}}\right], \end{aligned} $$

(4)

be the two-dimensional (2D) sampled periodic sinc function, where M₁ and M₂ are the number of samples per period in the 2D virtual channel domain, f=(f₁,f₂), f_d=0,…,M_d−1, d=1,2, are the indices of the samples, and $\boldsymbol {\Omega }^{l}=\left (\Omega _{1}^{l}, \Omega _{2}^{l}\right)=\left (M_{1}\eta _{1}^{l},M_{2}\eta _{2}^{l}\right)$.

The virtual channel matrix $\boldsymbol {V} \in \mathbb C^{M_{1} \times M_{2}}$ is the 2D-discrete Fourier transform (DFT) of H with entries [6]

$$\begin{array}{*{20}l} [\!\boldsymbol{V}]_{\boldsymbol{f}} &= \frac{1}{M_{1}M_{2}} \sum\limits_{i_{1}=0}^{{N_{\mathrm{r}}}-1} \sum\limits_{i_{2}=0}^{{N_{\mathrm{t}}}-1} [\boldsymbol{H} ]_{(i_{1},i_{2})} e^{-\frac{2\pi f_{1}i_{1}j}{M_{1}}} e^{-\frac{2 \pi f_{2}i_{2}j}{M_{2}}} \\&= \sum\limits_{l=1}^{L} \alpha_{l} \left[\boldsymbol{W}(\boldsymbol{\Omega}^{l})\right]_{\boldsymbol{f}}\,, \end{array} $$

(5)

where f=(f₁,f₂) are the indices of the channel sample in the virtual domain.

The SPC method [6] reported in Algorithm 1 (for algo = SPC), iteratively estimates the amplitude α_l and the discrete positions Ω^l of I paths in the virtual channel and cancels their corresponding periodic sinc functions in the virtual channel. After I iterations, the channel estimate $\boldsymbol {\widehat {H}}$ is obtained by taking the 2D-inverse discrete Fourier transform (IDFT) of the estimated virtual channel $\boldsymbol {\widehat {V}}$ reconstructed by summing the contributions of all the detected paths.

The number of iterations (i.e., the number of detected paths) is a trade-off between L (the number of paths) and the noise level. On the one hand, it is advisable to estimate all L paths, and on the other hand, noise can make small-power paths not detectable; therefore, it is better not to estimate all of them by using I<L. In Section 5, we determine by simulations the optimal I that minimizes the MSE of the channel estimate. Note that SPC provides an intrinsically approximated solution even in the absence of noise, since the peak positions Ω^l are estimated on a fixed discrete grid.

The JPC algorithm of [8] reported in Algorithm 1 (for algo= JPC) is a modification of SPC that at each iteration jointly estimates the amplitudes of all previously detected peaks by the LS approach and cancels the corresponding periodic sinc’s from the virtual channel. In particular, x=vec(X) stacks the columns of matrix X into the column vector x; at iteration l, one peak is detected (line 3) and then the amplitudes of all previously detected peaks are jointly estimated (line 8), and the new virtual channel with removed peaks is obtained (lines 9–11). This is achieved by building matrix w that contains in column l the vector version of $\boldsymbol {W}\left (\widehat {\boldsymbol {\Omega }}^{l}\right)$ (line 6).

This algorithm has the advantage over the SPC that each amplitude estimate is refined at each iteration thus taking advantage also of the peaks detected in further iterations.

3 Sparse dual channel estimation

In order to obtain an efficient and simple channel estimator, we exploit the specific channel structure described in the previous section. In particular, we use the fact that the channel is composed of a small number of paths with respect to the typically large number of transmit and receive antennas.

In this paper, we directly refer to the representation (1) and interpret it as 2D-IDFT of a sparse matrix having only L non-zero entries. First, the channel H is rearranged into the channel column vector ${\boldsymbol {h}}=\text {vec}(\boldsymbol {H}) \in \mathbb {C}^{N_{\mathrm {r}}N_{\mathrm {t}} \times 1}$ with entries

$$ [\boldsymbol{h}]_{i_{1}+N_{\mathrm{r}} i_{2}} = [ \boldsymbol{H} ]_{i_{1},i_{2}}, $$

(6)

where i₁=0,…,N_r−1, i₂=0,…,N_t−1, while the 2D-IDFT matrix is $\boldsymbol {F} \in \mathbb {C}^{N_{\mathrm {t}} N_{\mathrm {r}} \times M_{2} M_{1}}$ with entries

$$ [\boldsymbol{F}]_{(i_{1}+N_{\mathrm{r}}i_{2},f_{1}+M_{1}f_{2})} = \prod_{d \in \{1, 2\}} e^{\frac{2 \pi j f_{d}i_{d}}{M_{d}}}\,, $$

(7)

where f_d=0,…,M_d−1, d=1,2. Lastly, we define the column vector v of length M₁M₂ with L non-zero entries at position $\bar {\Omega }_{1}^{l} + M_{1} \bar {\Omega }_{2}^{l}$, for l=1,…,L, i.e.,

$$ [\boldsymbol{v}]_{\bar{\Omega}_{1}^{l}+M_{1}\bar{\Omega}_{2}^{l}} = \alpha_{l}, $$

(8)

with

$$ \bar{\Omega}_{1}^{l} = \langle \eta_{1}^{l} M_{1} \rangle \,, \bar{\Omega}_{2}^{l} = \langle \eta_{2}^{l} M_{2} \rangle \,, $$

(9)

and 〈x〉 denotes the integer part of x. From (1), we can approximate the channel vector as

$$ \boldsymbol{h} \approx \boldsymbol{F}\boldsymbol{v}. $$

(10)

We will denote with v as the dual channel in the M₁×M₂ domain. Note that the dual channel is sparse as it contains only L non-zero entries.

Remark 1

The approximation (10) stems from the rounding of (9), i.e., from approximating $\Omega _{d}^{l}$ with $\bar {\Omega }_{d}^{l}$. As M_d→∞, the approximation becomes more accurate. Moreover, we have used DFTs with M_d points along dimension d, as for the dual channel representation, in order to make a simpler comparison among various channel estimation schemes. Lastly, note that v is not the vectorial representation of the virtual channel, since the DFT used to obtain the virtual channel does not invert the IDFT of (10): in fact, the DFT is taken on the reduced set of N_r×N_t samples, thus yielding the periodic sinc’s of (4).

Similarly to the vectorial representation of the channel, we define

$$ \boldsymbol{h}' = \text{vec}(\boldsymbol{H'})\approx \boldsymbol{F}\boldsymbol{v}+\boldsymbol{n'}, $$

(11)

where indices of H^′ to obtain h^′ are selected similarly to (6). From (11), we observe that the LS estimate is a noisy version of a linear transformation of the dual channel v.

We propose an algorithm that improves the LS channel estimate by exploiting the sparsity of v. In particular, we define by $\hat {\boldsymbol {v}}$ the new estimate of the dual channel v and write the sparse channel estimation problem as

$$ \widehat{\boldsymbol{v}}=\underset{\boldsymbol{v}'}{\text{argmin}} \left(\Vert \boldsymbol{F}\boldsymbol{v}'- \boldsymbol{h}' \Vert_{2}^{2}+ \rho\Vert \boldsymbol{v}' \Vert_{0}\right), $$

(12)

where ∥v^′∥₀ is the ℓ₀-norm that counts the non-zero elements in v^′ and ρ is a parameter that controls the sparsity of the solution. This problem formulation aims at minimizing the MSE between the estimated channel and the LS channel estimate, under a constraint on the sparsity of vector $\hat {\boldsymbol {v}}$, imposed by the norm-zero term.

Unfortunately, problem (12) is non-convex and NP-hard [25]. Thus, we relax the problem by replacing the ℓ₀-norm with the ℓ₁-norm obtaining the LASSO problem

$$ \widehat{\boldsymbol{v}}=\underset{\boldsymbol{v}}{\text{argmin}} \left(\Vert \boldsymbol{F}\boldsymbol{v}-\boldsymbol{h}' \Vert^{2}_{2}+ \rho\Vert \boldsymbol{v} \Vert_{1}\right), $$

(13)

with $||\boldsymbol {v}||_{1} = {\sum \nolimits }_{i=0}^{M_{1}M_{2}-1} |v_{i}|$, which is now convex.

Remark 2

Note that estimating the dual channel opens the possibility of reducing the training overhead. We observe that systems with different numbers of antennas (placed at the same position) share the same dual channel. Thus, once we have an estimate of v, we can change F to obtain the channel estimate for a different antenna setting. Indeed, we can use a fewer transmit antennas to transmit the training sequence, then obtain an estimate of the dual channel and finally project the estimate into a larger number of antennas by modifying the size of the IDFT matrix in (10). Typically paths are concentrated in clusters on part of the dual channel, thus by an iterative channel estimation procedure, we can beamform training signals in the part of the dual channel covered by the clusters.

4 Solution of the sparse channel estimation problem

A vast literature is available for the solution of the sparse channel estimation problem (13), see for example the survey [26]. We propose here to use two recent and efficient methods based on the gradient descent algorithm with improved convergence speed, namely the AGDAR algorithm (also named FISTA with adaptive restart [27]) and the RCS algorithm [18].

4.1 Accelerated gradient descent with adaptive restart

The AGDAR algorithm [27] has been developed to solve problems where the objective function is the sum of a differentiable function and a general but simple closed convex function.

Here, we briefly summarize the motivation of the AGDAR algorithm. We first observe that the minimization problem $\min _{\boldsymbol {x} \in \mathbb R^{N}} f(\boldsymbol {x})$ when f(·) is convex and smooth can be solved by the gradient descent algorithm that iteratively updates the solution, computing at iteration p

$$ \boldsymbol{x}_{p} = \boldsymbol{x}_{p-1} - t \nabla f(\boldsymbol{x}_{p-1})\,, $$

(14)

where t is the step size and ∇f(x) is the gradient of f(·) computed in x. An alternative formulation of (14) is provided by the proximal form [28]

$$ \boldsymbol{x}_{p} = \text{argmin}_{\boldsymbol{x}} \nabla f(\boldsymbol{x}_{p-1})^{T}(\boldsymbol{x}-\boldsymbol{x}_{p-1}) + \frac{1}{2t} ||\boldsymbol{x}- \boldsymbol{x}_{p-1}||^{2}_{2}\,. $$

(15)

Now, in order to minimize f(x)+g(x) with f(·) still convex and smooth but g(·) convex, nondifferentiable, and lower semicontinuous, the proximal form must be modified as follows [28]

$$\begin{array}{*{20}l} {}\boldsymbol{x}_{p} =& \text{argmin}_{\boldsymbol{x}} \nabla f(\boldsymbol{x}_{p-1})^{T}(\boldsymbol{x}-\boldsymbol{x}_{p-1}) + \frac{1}{2t} ||\boldsymbol{x}- \boldsymbol{x}_{p-1}||^{2}_{2} \\ {}&+\! g(\boldsymbol{x}) \,=\, \text{argmin}_{\boldsymbol{x}} g(\boldsymbol{x})\,+\,\! \frac{1}{2t} ||\boldsymbol{x}\,-\,\! (\boldsymbol{x}_{p-1}\,-\, t \nabla f(\boldsymbol{x}_{p-1}))||^{2}_{2} \,. \end{array} $$

(16)

In general, this optimization problem may be hard to solve; however, when g(x)=ρ||x||₁, problem (16) is efficiently solved by splitting it into N separate one-dimensional problems for each entry of $\boldsymbol {x} \in \mathbb R^{N}$, i.e.,

$$ [\boldsymbol{x}_{p}]_{n} = \text{argmin}_{x}\; \rho |x| + \frac{1}{2t} |x - z_{n}|^{2} \,, $$

(17)

with z_n=[x_p−1−t∇f(x_p−1)]_n. Problem (17) can be solved in closed form [29] as $[\boldsymbol {x}_{p}]_{n} = \mathcal T_{\rho t}(z_{n})$, with the shrinkage operator defined as

$$ \mathcal T_{a} (x) = \text{sgn}(x) \text{max}(|x| -a,0)\,, $$

(18)

obtaining the iterative shrinkage-thresholding algorithm (ISTA) algorithm. This solution can be made faster by applying the Nesterov acceleration principle [30]: instead of using the gradient descent (14), x_p is updated as a linear combination of the gradient descent terms (14) at the current and previous iterations, i.e.,

$$ \boldsymbol{y}_{p} = \boldsymbol{x}_{p-1} - t \nabla f(\boldsymbol{x}_{p-1})\,, $$

(19a)

$$ \boldsymbol{x}_{p} = (1- \gamma_{p-1}) \boldsymbol{y}_{p} + \gamma_{p-1} \boldsymbol{y}_{p-1}\,. $$

(19b)

Combining this approach with ISTA, we obtain the FISTA algorithm where $[\boldsymbol {y}_{p}]_{n} = \mathcal T_{\rho t}(z_{n})$ and x_p is updated using (19b) It turns out that this approach fastens the convergence of the algorithm for example by choosing as linear combination coefficients [30]

$$ \gamma_{p} = \left\{\begin{array}{ll} 0 &\ p= 0 \\ \frac{1 - \theta_{p-1}}{\theta_{p}} &\ p >0\,, \end{array}\right. $$

(20)

$$ \theta_{p} =\left\{\begin{array}{ll} 1 &\ p = 0 \\ \left(1+\sqrt{1+4 \theta_{p-1}^{2}}\right)/2 &\ p > 0\,. \end{array}\right. $$

(21)

The explanation of the Nesterov iteration is not very intuitive, and the interested reader can find more details in [30].

The parameter choice (20) is not in general optimal while its optimization is a difficult task. An alternative approach is the adaptive restart technique [27], in which γ_p is set according to (20) (thus in a suboptimal way) but the FISTA algorithm is restarted whenever the objective function is locally increasing (thus the iterative solution is moving in the wrong direction), i.e., when

$$ \nabla f(\boldsymbol{x}_{p-1})^{T} (\boldsymbol{y}_{p} - \boldsymbol{y}_{p-1}) > 0\,, $$

(22)

From (19a) we obtain that the restarting condition (22) can be written as

$$ (\boldsymbol{x}_{p-1}-\boldsymbol{y}_{p})(\boldsymbol{y}_{p}-\boldsymbol{y}_{p-1}) >0\,. $$

(23)

The restart consists in resetting θ_p=1 and using as initial point the last point produced by the algorithm.

4.2 Re-weighted compressed sensing

The RCS method is proposed [18] to improve the sparsity of the gradient descent solution of (13). The algorithm weights the entries of x_p in the ℓ₁-norm in (13) in order to better approximate the ℓ₀-norm term in (12). Therefore, instead of solving (12), the RCS method aims at solving problem

$$ \widehat{{\boldsymbol{v}}}=\underset{{\boldsymbol{v}'}}{\text{argmin}} \left(\Vert \boldsymbol{F}{\boldsymbol{v}'}-{\boldsymbol{h}'} \Vert_{2}+ \rho\Vert \boldsymbol{D}{\boldsymbol{v}'} \Vert_{1}\right)\,, $$

(24)

where the diagonal matrix D contains the weights. This problem can be seen as a ℓ₁-norm relaxation of a weighted version of the ℓ₀-norm problem (12), i.e.,

$$ \widehat{\boldsymbol{v}}=\underset{\boldsymbol{v}'}{\text{argmin}} \left(\Vert \boldsymbol{F}\boldsymbol{v}'- \boldsymbol{h}' \Vert_{2}^{2}+ \rho\Vert \boldsymbol{D} \boldsymbol{v}' \Vert_{0}\right). $$

(25)

As the ℓ₀-norm counts the non-zero entries, regardless of their amplitude, for non-zero weights, the two problems (12) and (25) have the same solutions.

About the choice of the weights, they are meant to provide a good approximation of the ℓ₀-norm using the (weighted) ℓ₁-norm. Therefore, imposing that at solution

$$ \Vert \boldsymbol{D} \hat{\boldsymbol{v}} \Vert_{1} = \Vert \hat{\boldsymbol{v}} \Vert_{0}\,, $$

(26)

we obtain the optimum weights (diagonal entries of matrix D)

$$ [\boldsymbol{D}]_{k,k} = \left\{\begin{array}{ll} \frac{1}{|\hat{v}_{k}|}\,, &\ \hat{v}_{k} \neq 0\,,\\ * &\ \hat{v}_{k} = 0\,, \end{array}\right. $$

(27)

where ∗ denotes any non-zero value. However, this choice requires the knowledge of the problem solution $\hat {\boldsymbol {v}}$, which is not available while solving the problem.

In [18], an iterative approach has been proposed, where the weights are adapted to converge to (27) without knowing the optimal solution. In particular, RCS runs q₂ times the AGDAR algorithm, using at each iteration a different set of weights chosen according to (28). It has been shown by extensive simulations over a variety of examples that the following weight adaptation strategy is performing well: starting from D₀=diag{[1,…,1]}, for which (24) corresponds to (13), and then at iteration i+1 update the weights as

$$ [\boldsymbol{D}_{i+1}]_{j,j}=\frac{1}{\vert{[\boldsymbol{x}_{p}]_{j}\vert}+\zeta}, \quad j=1, \ldots, M_{1} M_{2}\,, $$

(28)

where x_p is the solution of (24) for weights D_i and ζ is a small number. Note that at convergence (for ζ≈0) we obtain (27).

The implementation of RCS is obtained by running q₂ times the AGDAR algorithm and computing the shrinkage function with a weighted parameter, i.e., $\mathcal T_{w_{n} \rho t}(z_{n})$. The resulting procedure is reported in Algorithm 3.

It has been shown [18] that the RCS algorithm is a majorization-minimization algorithm that iteratively minimizes a simple surrogate function majoring the objective function, and indeed provides in general a better approximation to the original ℓ₀-norm problem.

4.3 Sectorization and channel tracking

When the antennas at either or both the transmitter and the receiver are transmitting/receiving in a focused direction (in what is known as cell sectorization), the departure and arrival angles are within sub-intervals of [0,2π); therefore, also $\eta _{d}^{l}$ will take value in sub-intervals of [−1/2,1/2) and the rounding of $\eta _{d}^{l} M_{d}$ will be in a sub-interval of $\left (-\frac {M_{d}}{2}, \frac {M_{d}}{2}\right)$. Therefore, vector v can be reduced by eliminating the entries corresponding to values of $\eta _{d}^{l}$ that are never taken by the channel realization. Correspondingly, the columns of F are removed and the AGDAR algorithm is run over a reduced space, thus increasing its accuracy.

About channel tracking, once the channel has been estimated, it may slowly change due to the variations of the propagation environment. In this case, we can reduce the complexity of the channel estimation and make it more effective by simply tracking its changes rather than starting from scratch its estimation. We propose to focus the search of the paths in the dual channel within intervals around the initial estimates of arrival and departure angles. Therefore, for both AGDAR and RCS approaches, we have a reduction of the dual channel vector v. Indeed, this is similar to sectorization; however, for channel tracking we must consider multiple angle intervals, one for each initially estimated path.

Sectorization or tracking are also possible for SPC and JPC, wherein the search of the peak positions Ω^l will be done on a sub-grid of the M₁×M₂ grid, according to the intervals of $\eta _{1}^{l}$ and $\eta _{2}^{l}$. Also in this case, the benefits for the channel estimation process will be a lower probability of periodic sinc misplacement, as the search space is smaller.

4.4 Computational complexity

The computational complexity of the considered algorithms is evaluated in terms of complex multiplications (CMUX), complex additions (CADD), and comparisons (COM). Let $M_{tot}=\prod _{d} M_{d}$, where the product is along all dimensions, depending on the use of either UPA or ULA. Hence, using the fast Fourier transform algorithm, a (I)DFT of M_tot samples requires M_tot log2M_tot CMUX, M_tot log2M_tot CADD and no comparison. A summary of the computational complexity of the various algorithms is reported in Table 1, where q₃ denotes the effective number iterations over the variable p of the AGDAR and RCS algorithms. In the following section, we will present numerical results also for a complexity comparison among the various techniques.

Table 1 Computational complexity of the channel estimation methods

5 Numerical results

In this section, we compare the proposed channel estimation techniques by evaluating the MSE in decibels (dB)

$$ \text{MSE} =10\log_{10} \mathbb{E}\left[ || \widehat{\boldsymbol{H}} - \boldsymbol{H} ||_{2}^{2}\right]\,, $$

(29)

where $\mathbb E[\cdot ]$ denotes the expectation operator and $\hat {\boldsymbol {H}}$ is the estimated channel matrix.

We consider the urban macro cell (UMa), urban micro cell (UMi), rural macro cell (RMa), and Indoor Hotspot (InH) 3GPP channel models [3], with both line-of-sight (LOS) and non-line-of-sight (NLOS). In these scenarios, the number of clusters (typically from 4 to 20) depends on the channel model and the number of sub-paths is 20 per cluster, thus totaling L in the range of tens to hundreds. Note that although the number of sub-paths is large, only three or four sub-paths have a notable power. Therefore de facto, we find the sparse channel model described in this paper and in many literature papers and measurement campaign results. Channels are obtained for a downlink, where the base station (BS) and user equipment (UE) are on the same plane, with parameters defined in Table 2. The average channel gain is unitary, so we assume that transmit power has been adapted to compensate for the path loss; therefore, the average SNR is the reciprocal of the noise power. This also provides that the MSE for the LS estimate is simply the reciprocal of the SNR, which is then not reported in the figures.

Table 2 Simulation parameters

For the proposed AGDAR and RCS algorithms, we use parameters as in Table 2. In the following, we will always consider the same antenna geometry (either ULA or UPA) at both BS and UE, with a different number of antennas at the two ends.

5.1 Parameters setting

We first evaluate the impact of the parameters on the channel estimate MSE. The performance of both AGDAR and RCS is determined by the parameter ρ that weights the ℓ₁-norm term in the objective function and should be chosen according to the channel sparsity and the operating signal to noise ratio (SNR). For each value of SNR, we have assessed the optimum value of ρ that minimizes the average MSE of the channel estimate. The results are reported in Fig. 2 for both (at both devices) with N_r=16=4×4 (D₀=D₁=4) and N_t=4=2×2 (D₂=D₃=2) and M₀=M₁=M₂=M₃=8, and ULAs (at both devices) with N_r=16, N_t=4 and M₁=M₂=32. In this case the channel model is UMi LOS for which the number of clusters is random, between 3 and 12, while the number of sub-paths per cluster is 20, for a maximum total of L=240 paths. We can see a smooth behavior of ρ with respect to the average SNR, that can be described with simple functions for its adaptation to operating conditions. Moreover, as the average SNR increases, the optimal ρ decreases, since the LS estimate is less noisy and a limited sparsification of the channel is required. We have optimized the value of this parameter also for other conditions (e.g., different number of antennas) and forthcoming results are obtained with the optimized ρ.

Fig. 2

Choice of ρ. Optimized value of ρ vs SNR, for both UPA, with N_r=16=4×4 (D₀=D₁=4) and N_t=4=2×2 (D₂=D₃=2) and M₀=M₁=M₂=M₃=8, and ULAs with N_r=16, N_t=4 and M₁=M₂=32. UMi LOS model

A second relevant parameter for both AGDAR and RCS is the maximum number of iterations q₁. Figure 3 shows the average MSE as a function of the number of maximum allowed iterations q₁ for ULAs with N_r=16, N_t=4, and various DFT sizes. We note that for all DFT sizes the MSE is flooring as q₁ increases: in particular, with log10q₁=2.5, all algorithms converge to the minimum MSE. We also observe that the required number of iterations grows with (M₁,M₂), and RCS achieves a lower asymptotic MSE and converges faster.

Fig. 3

Effects of iterations q₁. MSE vs the maximum number of iterations q₁ for ULA with N_r=32,N_t=4 using AGDAR (solid lines) and RCS (dotted lines). UMi LOS model

For a comparison with SPC and JPC, we have also optimized parameter I, i.e., the number of detected paths. Note that the existing literature typically assumes the knowledge of L and sets I=L. We instead observe that I does not necessarily correspond to the number of paths, since small paths can be neglected as may be easily confused with noise artifacts. This is particularly true in the 3GPP channel model, where many clusters and sub-paths are present, most of which however have a very limited power. Figure 4 shows the value of I that minimizes the average MSE versus the average SNR for various values of M=M₁=M₂, using ULAs with N_r=16, N_t=4 and SPC. The channel model is UMi LOS. We observe that we need a large value of I when the SNR is high, as the considered channel model has many (sub-)paths, and at high SNRs, they can be distinguished from the noise. Also, note that the optimal value of I is decreasing as M increases: indeed, for higher M the approximation between positions Ω_l and $\hat {\boldsymbol {\Omega }}_{l}$ becomes more accurate, thus fewer sinc functions (closer to the actual number of paths) is enough (and better) for channel estimation. The results reported in the rest of this section are obtained with the optimized value of I.

Fig. 4

Choice of I. Optimal I for both SPC and JPC methods and various values of M=M₁=M₂ using ULA with N_r=16, N_t=4. UMi LOS model

Lastly, we consider the optimization of the DFT size, i.e., the number of points used to approximate $\eta _{d}^{l}$ in all considered channel estimation methods. In Appendix B, we provide the analytical derivation of the MSE as a function of M, for ULA and a Rayleigh fading channel with a small number of taps. Figure 5 shows the MSE as a function of M=M₁=M₂ for ULA with N_t=8, N_r=2 and the channel model described in Appendix B. As expected the MSE decreases for a higher value of M, as the quantization error is reduced, flooring for high values of M. We then have considered a ULA system with N_r=32 and N_t=4 and a DFT size multiple μ of the number of antennas, i.e., M₁=μN_r and M₂=μN_t. For a UMi LOS model, Fig. 6 shows the average MSE for an SNR of 0, 10, and 20 dB, and different channel estimation techniques. We observe that in all cases by increasing μ we obtain a better channel estimate, thanks to a better quantization precision of either the virtual channel domain (for SPC and JPC) or the values of $\eta _{d}^{l}$ (for both AGDAR and RCS). Moreover, both AGDAR and RCS methods achieve a lower MSE than SPC and JPC techniques at both low and high SNRs, thanks to their better exploitation of compact channel representation in the dual domain. The RCS has almost negligible performance improvement with respect to AGDAR, as both have a gain from 3 to 5 dB with respect to interference cancelation techniques. Note that the gain is more remarkable at a lower SNR, showing that the compressed-sensing techniques are able to better reject the noise. Lastly, note that JPC has an almost negligible improvement over SPC; thus, we can conclude that the detection of the peaks is already accurate when performed sequentially rather than in parallel. Overall, we conclude that μ=2 already provides close-to-optimal results for all methods. Similar observations can be drawn from Fig. 7, where we report the MSE for a UPA configuration with 2×2 antennas at the UE and 6×6 antennas at BS in a UMa LOS channel model.

Fig. 5

Analysis of choice of M. MSE vs M=M₁=M₂ for the scenario of Appendix B, with N_t=16 and N_r=4

Fig. 6

Choice of M for N_r=32 and N_t=4. MSE vs μ for ULA with N_r=32 and N_t=4 at an SNR of 0 (dotted lines), 10 (solid lines) and 20 dB (dashed lines) and M₁=μN_r and M₂=μN_t. UMi LOS model

Fig. 7

Choice of M for N_r=32 and N_t=4. MSE vs μ for UPA with N_r=36 and N_t=4 at an SNR of 0 (dotted lines), 10 (solid lines) and 20 dB (dashed lines) and M₁=μN_r and M₂=μN_t. UMa LOS model

In order to show the performance of the proposed solution in a scenario with a large number of antennas, Fig. 8 shows the MSE as a function of μ for ULA with N_r=128 and N_t=16 and the UMi LOS model. Also in this case, we can appreciate the advantage of all techniques with respect to the LS method, as we recall that for LS the MSE is the reciprocal of the SNR, thus 0 dB in correspondence of the dotted lines, −10 dB for the solid lines and −20 dB for the dashed lines. Indeed, a higher number of antennas with respect to Fig. 6 increases the gain of the other channel estimation techniques with respect to LS. About the comparison among the various methods, we can derive similar conclusions as those of Fig. 6, confirming also that other results are representative of a massive MIMO scenario.

Fig. 8

Choice of M for N_r=32 and N_t=4. MSE vs μ for ULA with N_r=128 and N_t=16 at an SNR of 0 (dotted lines), 10 (solid lines), and 20 dB (dashed lines) and M₁=μN_r and M₂=μN_t. UMi LOS model

5.2 Sectorization and channel tracking

As we already discussed, sectorization provides a faster and more accurate search of the channel paths. Here, we consider a system where channel angles are uniformly distributed in intervals of 6, 60, and 180 degrees. We have L=14 paths, with independent Gaussian-distributed amplitudes α_l: by this simple channel model, we better capture the effects of sectorization and channel tracking.

Figure 9 shows the resulting MSE as a function of M=M₁=M₂ for the various systems, when the average SNR is 10 dB. We observe that sectorization indeed reduces the MSE of all channel estimates, and sectors of 6° provide a MSE of 10 to 16 dB smaller than that of 6° sectors. Comparing the various techniques we observe that with large (180°) and small (6°) sectors all techniques take advantage of the sectorization in a similar way, while for intermediate values (60°) the compressed-sensing methods have a higher gains than interference cancelation methods.

Fig. 9

Effects of sectorization. MSE vs M=M₁=M₂ for ULAs and various channel estimation techniques, when the average SNR is 10 dB when sectorization is used, with 6 (solid lines), 60 (dashed lines), and 180° (dotted lines). N_r=N_t=16

We also consider channel tracking where, after an initial channel estimation performed according to the various considered techniques and with an angle span of 360°, when channels are time-invariant. Estimators are run using an angle span of 6° around the angles of each path. Figure 10 shows the MSE of the channel estimates at an average SNR of 10 dB, and for various values of M₁ and M₂. We observe that, thanks to the search over a smaller angle span both SPC and JPC achieve similar performance to the proposed approaches, a phenomenon that we already observed with sectorization. Also, the difference between RCS and AGDAR is further reduced, again because of the easier task of channel estimation in this case. We still note instead a high sensitivity to the DFT-size, which corresponds to an accuracy in the estimate of the angles of arrival and departure. Lastly, both sectorization and channel tracking reduce the complexity of the proposed solutions, as path search operations can be performed on a reduced space.

Fig. 10

Effects of tracking. MSE for ULAs of various channel estimation techniques, when the average SNR is 10 dB with channel tracking at 6°, and various values of M. N_r=16 and N_t=4

5.3 3GPP channel scenarios comparison

Until now, we considered only the UMi LOS channel model: in this section instead, we consider also the other 3GPP channel models. Figure 11 shows the average MSE for various algorithms and UPAs with N_t=8×8 at the BS and N_r=2×2 at the UE, and M₀=M₁=10 and M₂=M₃=4. We compare various channel estimation techniques for an average SNR of 10 dB. At this intermediate SNR value, we observe that the proposed AGDAR and RCS significantly outperform both SPC and JPC for all the considered channel models, by 6 to 7 dB. The indoor-office model (InH Mixed) has a few significant taps with reduced dispersion, a favorable condition for SPC and JPC, which exhibit a reduced gap with respect to the proposed (still better performing) techniques. On the other hand, dispersive channels with many low-power taps (UMi Street Canyon NLOS) make the channel estimation more problematic for SPC and JPC, while can be handled very efficiently by the compressed-sensing techniques, thanks to their ability to better distinguish between noise and channel components. This provides a gain of 7 dB between SPC and AGDAR. We also note that both AGDAR and RCS have comparable performance across all channel models. Similar results are obtained at low and high SNR (results are not reported here for the sake of conciseness).

Fig. 11

Effects of channel models. MSE under different channel models with SNR of 10 dB for UPA with N_t=8×8 at the BS and N_r=2×2 at the UE, and M₀=M₁=10 and M₂=M₃=4

5.4 Complexity comparison

In order to assess the complexity of the various channel estimation methods, we first report in Fig. 12 the effective number of iterations q₃ of AGDAR as a function of the maximum allowed number of iterations q₁, for ULAs with N_r=32, N_t=4, and average SNR of 10 dB on a UMi LOS channel model. As expected, when the number of allowed iterations increases, also the number of effective iterations increases, until reaching a floor. Moreover, a higher value of M₁ and M₃ requires a higher number of iterations. We also observe that RCS requires fewer iterations (while achieving a better performance in terms of MSE) as the reweighting fastens the convergence.

Fig. 12

Choice of q₃. Number of iterations q₃ (in log scale) vs the number iterations q₁ (in log scale) for a ULA system with N_r=32 and N_t=4 for AGDAR (solid lines) and RCS (dotted lines). UMi LOS model

In Fig. 13, we compare the complexity of the various scenarios, by considering the number of complex multiplications, as derived in Section 4.4, as a function of the number of canceled paths I. Parameters are those of Fig. 3, so that we can compare the achieved MSE of the various schemes. We observe that the number of multiplications grows exponentially for the SPC and JPC techniques. We also report the number of complex multiplications for the AGDAR and RCS methods that do not depend on I. We note that the AGDAR has a remarkably lower complexity than other methods. However, we also notice that for M₁=64 and M₂=8, RCS has a significantly higher complexity with respect to the other methods. When comparing the MSE performance (Fig. 3), we see that AGDAR achieves a much lower MSE than SPC and JPC methods for a lower complexity (in terms of CMUX and CADD).

Fig. 13

Computational complexity. Average number of complex multiplications as function of I for various systems with (M₁=32, M₂=4) (solid lines) and (M₁=64, M₂=8) (dotted lines), ULA system with (N_r=32, N_t=4) and SNR of 10 dB

6 Conclusions

In this paper, we have proposed channel estimation techniques for mm-Wave massive MIMO systems, based on a CS approach, where we have exploited the sparse nature of the channel, considering in particular the small number of channel paths at those frequencies. Efficient innovative solutions based on the adaptive restart of the Nesterov accelerated gradient algorithm have been explored. Numerical results have shown the superiority of the proposed approaches with respect to existing procedures, with similar or lower computational complexity. We have also considered the effects of sectorization and proposed a channel tracking technique that exploits slow channel variations.

7 Appendix A: CS channel estimation for UPA

For a UPA system, we denote as D₂ and D₃ the number of transmit antennas along the y- and z-axes, respectively; therefore, N_t=D₂×D₃. Similarly D₀ and D₁ are the numbers of antennas at the receiver along the two axes; therefore, N_r=D₀×D₁. The channel matrix has entries

$$ \left[ \boldsymbol{H}^{\text{UPA}} \right]_{i_{1}+D_{1}i_{0},i_{3}+D_{3}i_{2}} \,=\, \sum\limits_{l=1}^{L} \alpha_{l}e^{2\pi \eta_{0}^{l}i_{0}j} e^{2\pi \eta_{1}^{l}i_{1}j} e^{2\pi \eta_{2}^{l}i_{2}j}e^{2\pi \eta_{3}^{l}i_{3}j}. $$

(30)

The channel H^UPA is transformed into the channel column vector ${\boldsymbol {h}^{\text {UPA}}}=\text {vec}\left (\boldsymbol {H}^{\text {UPA}}\right) \in \mathbb {C}^{N_{r}N_{\mathrm {t}} \times 1}$ with entries

$$ \left[\boldsymbol{h}^{\text{UPA}}\right]_{i_{0}+D_{0}i_{1}+D_{0}D_{1}i_{2}+D_{2}D_{1}D_{0}i_{3}} = \left[ \boldsymbol{H}^{\text{UPA}} \right]_{i_{1}+D_{1}i_{0},i_{3}+D_{3}i_{2}}. $$

(31)

We also define the 4D-DFT matrix as $\boldsymbol {F}^{\text {4D}} \in \mathbb {C}^{N_{r}N_{\mathrm {t}} \times (M_{0} M_{1} M_{2}M_{3})}$ with entries

$$\begin{array}{*{20}l} &\left[\boldsymbol{F}^{4D}\right]_{(i_{0}+D_{0}i_{1}+D_{1}D_{0}i_{2}+D_{2}D_{1}D_{0}i_{3},f_{0}+M_{0}f_{1}+M_{1}M_{0}f_{2}+M_{2}M_{1}M_{0}f_{3})}\\&= \prod_{d=0}^{3} e^{-\frac{2 \pi i_{d}f_{d}j}{M_{d}}}. \end{array} $$

(32)

Lastly, we define the column vector v^UPA of length M₀M₁M₂M₃ with L non-zero entries, namely

$$ \left[\boldsymbol{v}^{\text{UPA}}\right]_{\bar{\Omega}_{0}^{l}+M_{0}\bar{\Omega}_{1}^{l}+M_{1}M_{0}\bar{\Omega}_{2}^{l}+M_{2}M_{1}M_{0}\bar{\Omega}_{3}^{l}} = \alpha_{l} $$

(33)

where

$$ \bar{\Omega}_{0}^{l} \,=\, \left\langle \eta_{0}^{l} M_{0} \right\rangle\,, \bar{\Omega}_{1}^{l} \,=\, \left\langle \eta_{1}^{l} M_{1} \right\rangle\,, \bar{\Omega}_{2}^{l} \,=\, \left\langle \eta_{3}^{l} M_{3} \right\rangle, \bar{\Omega}_{3}^{l} \,=\, \left\langle \eta_{3}^{l} M_{3} \right\rangle. $$

(34)

From (30), we can approximate the channel as

$$ \boldsymbol{h}^{\text{UPA}} \approx \boldsymbol{F}^{\text{4D}}\boldsymbol{v}^{\text{UPA}}. $$

(35)

Similarly, we can define $\boldsymbol {h}^{'\text {UPA}}\phantom {\dot {i}\!}$ and $\boldsymbol {v}^{'\text {UPA}}\phantom {\dot {i}\!}$ for the LS estimate of the channel and its dual representation. The AGDAR and RCS algorithms for UPA can be obtained as described in Section 4, with F, h^′, and v^′ replaced by F^4D, $\phantom {\dot {i}\!}\boldsymbol {h}^{'\text {UPA}}$, and $\boldsymbol {v}^{'UPA}\phantom {\dot {i}\!}$, respectively.

8 Appendix B: On the choice of M _d

The choice of the number of DFT points per dimension M_d is important to determine the performance of the channel estimation algorithm. From (34), we have that the AGDAR solution approximates $\eta _{d}^{l}$ with a quantized value taken over M_d possible points. Therefore, we can write the quantization error on the l-th path as

$$ \epsilon^{l}_{d} = \eta^{l}_{d} - \frac{\left\langle \eta_{d}^{l} M_{d} \right\rangle}{M_{d}} \,. $$

(36)

Focusing now on a scenario wherein ULAs are used at both transmitter and receiver, assuming that all other estimates (i.e., the amplitude angle estimates of each path) are correct and the only imperfection is the quantization error, from (36), the estimated channel can be written as (compared with (1))

$$ [ \hat{\boldsymbol{H}}]_{i_{1},i_{2}} =\sum\limits_{l=1}^{L} \alpha_{l} e^{2\pi \left(\eta_{1}^{l} - \epsilon^{l}_{1}\right) i_{1}j}e^{2\pi \left(\eta_{2}^{l} - \epsilon^{l}_{2}\right) i_{2}j} $$

(37)

and using the definition of H of (1), we have that the MSE of the channel estimate is

$$ \begin{aligned} \gamma^{\text{(q)}} & = \frac{1}{N_{\mathrm{r}}N_{\mathrm{t}}}\sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1} {\mathbb E}\left[|[\boldsymbol{H}]_{i_{1},i_{2}}- \hat{\boldsymbol{H}}]_{i_{1},i_{2}} |^{2}\right] \\ & = \frac{1}{N_{\mathrm{r}}N_{\mathrm{t}}}\sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1} {\mathbb E}\left[ \left|\sum\limits_{l=1}^{L} \alpha_{l} e^{2\pi \eta_{1}^{l}i_{1}j}e^{2\pi \eta_{2}^{l}i_{2}j}\left(1 - e^{-2\pi \epsilon^{l}_{1} i_{1}j}e^{-2\pi \epsilon^{l}_{2}i_{2}j}\right) \right|^{2}\right]. \end{aligned} $$

(38)

Now, assuming that the amplitudes and angles are independent random variables, we have

$$ \begin{aligned} \gamma^{\text{(q)}} = & \frac{1}{N_{r}N_{\mathrm{t}}} \sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1}\sum\limits_{l=1}^{L} \sigma_{\alpha}^{2}(l) {\mathbb E}\left[ \left| 1 - e^{-2\pi \epsilon^{l}_{1} i_{1}j}e^{-2\pi \epsilon^{l}_{2}i_{2}j} \right|^{2} \right]\,, \end{aligned} $$

(39)

where $\phantom {\dot {i}\!}\sigma _{\alpha }^{2}(l) = {\mathbb E}[|\alpha _{l}|^{2}]$. Expanding the expectation we have

$$ \begin{aligned} \gamma^{\text{(q)}} = & \frac{1}{N_{\mathrm{r}}N_{\mathrm{t}}} \sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1}\sum\limits_{l=1}^{L}\sigma_{\alpha}^{2}(l) {\mathbb E}\left\{ \left[1 - \cos\left(2\pi \left(\epsilon^{l}_{1} i_{1}+ \epsilon^{l}_{2}i_{2}\right)\right)\right]^{2}\right.\\&\left. + \sin^{2}\left(2\pi \left(\epsilon^{l}_{1} i_{1}+ \epsilon^{l}_{2}i_{2}\right)\right) \right\} \\ = & \frac{1}{N_{\mathrm{r}}N_{\mathrm{t}}} \sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1}\sum\limits_{l=1}^{L}\sigma_{\alpha}^{2}(l) {\mathbb E}\left\{ 2\left[1 - \cos\left(2\pi \left(\epsilon^{l}_{1} i_{1}+ \epsilon^{l}_{2}i_{2}\right)\right)\right]\right\} \\ = & \frac{1}{N_{\mathrm{r}}N_{\mathrm{t}}} \sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1} \sum\limits_{l=1}^{L} \sigma_{\alpha}^{2}(l) 2 \times \\ & {\mathbb E}\left\{ 1 - \cos\left[2\pi \left(\eta^{l}_{1} - \frac{\langle \eta_{1}^{l} M_{1} \rangle}{M_{1}}\right) i_{1}+ \left(\eta^{l}_{2} - \frac{\left\langle \eta_{2}^{l} M_{2} \right\rangle}{M_{2}} \right) i_{2} \right]\right\}\,. \end{aligned} $$

(40)

Let us assume that arrival and departure angles ($\vartheta ^{r}_{l}$ and $\vartheta ^{t}_{l}$) are uniformly distributed in the interval [0,2π) and $\eta _{1}^{l}={\frac {\delta }{\lambda }\cos \vartheta ^{r}_{l}}, \eta _{2}^{l}=-{\frac {\delta }{\lambda }\cos \vartheta ^{t}_{l}}$. Then, the probability density function of $\eta _{1}^{l}$ and $\eta _{2}^{l}$ for $\delta =\frac {\lambda }{2}$ becomes

$$ {p}_{{\eta}_{1}^{l}}(a) = \left\{\begin{aligned} &\frac{\lambda}{\delta \pi \sqrt{1-\lambda^{2} {a}^{2}/{d}^{2}}},\ a \in \left[-1/2,\right. \left.1/2\right) \\ &0 \ \qquad\qquad\qquad\quad\text{otherwise,} \end{aligned}\right. $$

(41)

and (40) becomes

$$ \begin{aligned} \gamma^{\text{(q)}} = & \frac{1}{N_{\mathrm{r}}N_{\mathrm{t}}} \sum\limits_{i_{1}=0}^{N_{\mathrm{r}}-1} \sum\limits_{i_{2}=0}^{N_{\mathrm{t}}-1}\sum\limits_{l=1}^{L} \sigma_{\alpha}^{2}(l) \sum\limits_{m_{1}=0}^{M_{1}-1} \sum\limits_{m_{2}=0}^{M_{2}-1} \\ & \int_{m_{1}/M_{1}-1/2}^{(m_{1}+1)/M_{1}-1/2} \int_{m_{2}/M_{2}-1/2}^{(m_{2}+1)/M_{2}-1/2} p_{\eta_{1}^{l}}(a) p_{\eta_{2}^{l}}(b) \times \\ &\left\{ 1 \,-\, \cos\left[2\pi \left(a \,-\, \frac{m_{1}}{M_{1}}\right) i_{1}\,+\, \left(b \,-\, \frac{m_{2}}{M_{2}} \right) i_{2} \right]\right\} da db\,. \end{aligned} $$

(42)

This MSE provides a guideline for the choice of M_d, as we must have at least γ^(q)>σ², so that quantization does not introduce more errors (in terms of its power) than noise already present.