Massive multiple-input multiple-output (MIMO) antenna systems potentially allow base stations (BSs) to operate with huge improvements in spectral and radiated energy efficiency, using relatively low-complexity linear processing. The higher spectral efficiency is attained by serving several terminals in the same time-frequency resource through spatial multiplexing, and the increase in energy efficiency is mostly due to the array gain provided by the large set of antennas [1].

The expected massive MIMO improvements assume that accurate channel estimations are available at both the receiver and transmitter for detection and precoding, respectively. Additionally, the reuse of frequencies and pilot reference sequences in cellular communication systems causes interferences in channel estimation, degrading its performance. Since both the time-frequency resources allocated for pilot transmission and the channel coherence time are limited, the number of possible orthogonal pilot sequences is also limited, and as a consequence, the pilot sequences have to be reused in neighbor cells of cellular systems. Therefore, channel estimates obtained in a given cell get contaminated by the pilots transmitted by the users in other cells [2]. This coherent interference is known in the literature as pilot contamination, i.e., the channel estimate at the base station in one cell becomes contaminated by the pilots of the users from other cells [3]. The contamination not only reduces the quality of the channel estimates, i.e., increases the MSE, but also makes the channel estimates statistically dependent, even though the true channels are statistically independent. Moreover, pilot contamination does not disappear with the addition of more antennas [4].

Massive MIMO systems operating in TDD assume channel reciprocity between uplink and downlink in order to minimize pilot overhead, transmitting pilot reference signals only in the uplink. In this scenario, pilot overhead cost is proportional to the number of terminals and improved estimation quality can be achieved due to the large number of antennas [5, 6]. Base stations estimate channels usually based on least squares (LS) [3] or minimum mean square error (MMSE) [7–9] methods. Besides, inter and intra-cell large-scale fading coefficients are assumed to be perfectly known when applying the MMSE method in the great majority of works [5, 9–13].

In a real-world network deployment, although changing slowly, the large-scale fading coefficients must be estimated and updated from time to time. Additionally, the estimation error of the large-scale fading coefficients impacts significantly on the performance of uplink data decoding and downlink transmission (e.g., precoding and beamforming) [14–16]. Approaches on how to estimate the large-scale fading coefficients are presented in the following pieces of work [10, 14, 17].

The most commonly used analytical massive MIMO channel is the spatially i.i.d. frequency non-selective (flat) fading channel model. Flat fading channels are also known as *amplitude varying channels* and *narrowband channels* as the signal’s bandwidth is narrow compared to channel’s bandwidth [18]. In this narrowband channel model, the channel gain between any pair of transmit-receive antennas is modeled as a complex Gaussian random variable. This model relies on two assumptions: (i) the antenna elements in the transmitter and receiver being spatially well separated once the more widely spaced (in wavelengths) the antenna elements, the smaller the spatial channel correlation [19, 20], and (ii) the presence of a large number of temporally but narrowly separated multipaths (common in a *rich-scattering* environment), whose combined gain, by the central-limit theorem, can be approximated by a Gaussian random variable [20].

Flat fading channels present a channel response that exhibits flat gain and linear phase over a bandwidth (coherence bandwidth) that is greater than the signal’s bandwidth. Therefore, all frequency components of the signal will experience the same magnitude of fading, resulting in a scalar channel response. The gain applied to the signal varies over time according to a fading distribution. In this work, we consider that the gain applied to the signal passing through this channel will vary randomly, according to a Rayleigh distribution. We additionally assume that the antenna spacing is sufficiently large so that the antennas are uncorrelated.

In this paper, we deal with the channel estimation and pilot contamination problems associated with uplink training in flat Rayleigh fading channels and understand its impact on the operation of multi-cell MU massive MIMO TDD cellular systems. We propose and evaluate an efficient and practical channel estimator that does not require previous knowledge of inter/intra-cell large-scale fading coefficients (i.e., interference) and noise power. Differently from [21], we employ the maximum likelihood (ML) method to find an estimator for the interference plus noise power term in the MMSE channel estimator. We show that this estimator is not only unbiased but also achieves the Crámer-Rao lower bound. We replace this estimator back into the MMSE estimator and prove that the performance of the new channel estimator asymptotically approaches that of the MMSE estimator. Simulation results confirm that the performance of the proposed channel estimator approaches that of the ideal MMSE estimator asymptotically with the number *M* of antennas, i.e., *M*→*∞*. Additionally, in contrast with [21], we derive an approximate analytical MSE expression for the proposed channel estimator that is more mathematically tractable and not susceptible to numerical issues.

### Related work

In this section, we survey previous work on channel estimation and pilot contamination mitigation.

A TDD cellular system employing BSs equipped with large numbers of antennas that communicate simultaneously with smaller numbers of cheap, single-antenna terminals through MU MIMO techniques is proposed in [3]. The author employs LS channel estimation in order to study and evaluate the problems caused by pilot contamination to such systems. He concludes that even when different sets of orthogonal pilots are used in different cells, it makes little difference to the resulting signal-to-interference ratio (SIR). This work is the first one to present the massive MIMO concept and identify its intrinsic issues, however, it fails to suggest ways to mitigate the pilot contamination problem.

The impact of pilot contamination on multi-cell systems is studied in [5]. The authors adopt MMSE channel estimation for the analysis of pilot contamination and the achievable rates in a massive MIMO system suffering from such problem. They propose a multi-cell MMSE-based precoding method that mitigates the pilot contamination problem by considering the set of training sequences assigned to the users in the solution of an optimization problem that minimizes the error seen by users in the serving cell and the interference seen by the users in all other cells. Simulation results show that the proposed approach has significant gains over certain single-cell precoding methods such as zero-forcing. In summary, the authors address the pilot contamination problem through a precoding technique and assume that the large-scale fading coefficients are known to all BSs.

MMSE channel estimation is used in [7] to derive approximations of the achievable uplink and downlink rates with several linear precoders and detectors for realistic system dimensions, i.e., systems where the number of antennas is not extremely large compared to the number of users. Simulation results show that the approximations are asymptotically tight, but accurate for realistic systems. The authors do not propose any approach to mitigate the pilot contamination problem, however, they study and evaluate its impact on the achievable rates.

The impact of pilot contamination effect on the achievable uplink ergodic rate when using linear detection in multi-cell MU massive MIMO systems under a more realistic physical channel model is assessed in [8]. The authors assume that the channel vectors for different users are correlated, or not asymptotically orthogonal due to the antennas not being sufficiently well separated and/or the propagation environment not offering rich enough scattering. Moreover, they assume that the BS performs MMSE channel estimation based on training sequences received on the uplink and a priori knowledge of the large-scale fading coefficients.

In [9], the polynomial expansion (PE) technique is applied to channel estimation of massive MIMO systems in order to approximate the MMSE estimator and thereby obtain a new set of low-complexity channel estimators. Conventional MMSE estimators present cubic complexity due to an inversion operation while the estimator proposed in [9] reduces this to square complexity by approximating the inverse by a *L*-degree matrix polynomial. The proposed estimator achieves near-optimal MSE with low polynomial degrees. However, statistical knowledge of channel and disturbance parameters at the receiver is assumed in this paper.

Outer multi-cellular precoding is employed in [10] to devise a method used to eliminate pilot contamination in massive MIMO systems. Each BS performs two levels of precoding, firstly it estimates and shares only the large-scale fading coefficients with a central entity (network controller) which computes the precoding matrices and sends them back to the BSs, i.e, outer precoding. Next, each BS performs local precoding using estimates of the fast-fading vectors, i.e., inner precoding. The proposed approach is shown to completely mitigate the pilot contamination problem, making it possible to construct interference and noise free multi-cell massive MIMO systems with frequency reuse one and infinite downlink and uplink signal-to-interference-plus-noise ratios (SINRs). The proposed method employs MMSE channel estimation, however, the effectiveness of this method lies in the estimation accuracy of the shared large-scale fading coefficients from each BS. The authors also propose a method to estimate the large-scale fading coefficients. As this approach needs to share the large-scale coefficients with the network controller for outer precoding computation, it presents a higher computational complexity than non-cooperative approaches.

The authors in [11], adopt a massive MIMO system model that is based on spatially correlated channels. They devise a covariance aided channel estimation method which exploits the covariance information of both desired and interfering user channels. The Bayesian method is used to derive two different channel estimators (it is also shown that the Bayesian estimators coincide with the MMSE estimators), one for all channels from users in all cells to the target cell and the other one for the channels from users within the target cell. Results show that in the ideal case, where the desired and the interference covariance matrices span distinct subspaces, the pilot contamination effect tends to vanish in the large antenna array case. As a consequence, users with mutually non-overlapping angle of arrival (AoA) hardly contaminate each other. Based on the results, the authors propose a coordinated pilot assignment strategy which assigns carefully selected groups of users to identical pilot sequences.

A semi-blind iterative space-alternating generalized expectation maximization (SAGE) based channel estimation algorithm for massive MIMO systems with pilot contamination is proposed in [13]. The proposed method does not assume a priori knowledge on the large-scale fading coefficients of the interfering cells, employing an estimate obtained from the received signal. The method updates the pilot based MMSE channel estimates iteratively with the help of the SAGE algorithm, which improves the initial estimate with the help of pilot symbols and soft information of the transmitted data. However, as it refines the channel estimates over some iterations starting from an initial MMSE channel estimation, it presents a computational complexity that is higher than the one presented by pure blind and linear estimators.

After surveying the literature on channel estimation and pilot contamination mentioned above, it is clear that, for clarity, in the great majority of studies the authors always assume complete knowledge on large-scale fading coefficients, i.e., path-loss and shadow fading, of the interfering cells, which is not the case in practical deployments of MU Massive MIMO systems. Furthermore, several studies propose solutions that present additional computational complexity in order to mitigate the pilot contamination problem.

The main contribution of our work is the proposal and assessment of a simple and practical channel estimator used to mitigate the pilot contamination problem. The proposed estimator does not assume a priori knowledge of the large-scale fading coefficients of the interfering cells. Moreover, it does not require the heavy overhead created by their estimation once it obtains them from the received signal.

### Organization

The remainder of this work is divided into four parts: First, we present the problem structure, signal model adopted for this study and briefly discuss two well-known channel estimators, namely, LS and MMSE linear estimators. Then, we introduce the proposed channel estimator for flat Rayleigh fading channels. Later, some numerical results are presented in order to support the effectiveness of the proposed estimator against the well-known linear estimators. Finally, we present our conclusions.