Open Access

3GPP 3D MIMO channel model: a holistic implementation guideline for open source simulation tools

EURASIP Journal on Wireless Communications and Networking20162016:55

Received: 14 August 2015

Accepted: 4 February 2016

Published: 19 February 2016


Massive MIMO and 3D beamforming have been identified as key technologies for future mobile cellular networks. Their investigation requires channel models that consider not only the azimuth- but also the elevation direction. Recently, the 3rd Generation Partnership Project (3GPP) has released a new 3D spatial channel model. It supports planar antenna arrays and enables to scrutinize concepts such as elevation beamforming and full dimension MIMO. A particular challenge is the practical implementation of the model. Dealing with enormous computational complexity requires to design a highly efficient approach. This paper provides a guideline for the practical implementation of the 3GPP 3D model into existing link- and system-level simulation tools. Considering the complexity of the model itself, our main focus is on computational efficiency. We present simulation examples using the proposed procedure with the Vienna LTE-A Downlink System Level Simulator. We measure simulation run times with respect to various network parameters. Our results allow to quantify the increase in complexity, when accounting for the elevation dimension. Moreover, they exhibit general trends when considering a large number of antenna elements per antenna array. We also draw a comparison with the WINNER channel model, which represents the most closely related channel model in 2D.


3GPP 3D channel model System level simulations Link level simulation Open source Interference channel Elevation beamforming Full-dimension MIMO Vertical sectorization Channel coefficient generation

1 Introduction

In the last decades, simulations have become a substantial tool for analyzing and designing wireless cellular communication systems. As the systems themselves are growing in complexity, the effort of simulations becomes tremendous. Thus, the challenge is to keep the computational costs at a minimum while preserving accuracy. A commonly employed solution is to divide the simulations into two stages or levels of abstraction, known as link-level and system-level [1]. Link-level simulations are used to assess the performance of the physical layer and those higher layer aspects directly related to the radio interface. Mostly, only a single radio link is evaluated, rarely some few users. System level simulations, on the other hand, aim to evaluate the performance of a whole network comprising a substantial number of Evolved Node B (eNodeB) sectors and user equipments (UEs) [2, 3]. At the UEs, both the signals received from the serving as well as the interfering eNodeB sectors are modeled, taking into account large- as well as small-scale fading effects. Realistic models for the small-scale effects, also known as channel models, impose a major challenge in describing wireless communications. Broadly speaking, channel models can be divided into two categories, deterministic and stochastic [4]. Deterministic models describe the channel for a specific propagation environment between eNodeB sector and UE. This method can be tedious to evaluate and does not allow for general statements in an ensemble of environments. In stochastic models, the channel characteristics are condensed to a statistical description, e.g., the power delay profile (PDP) [57].

In order to close the gap between the two approaches, the 3rd Generation Partnership Project (3GPP) has introduced the Spatial Channel Model SCM [8]. Unlike traditional channel models, it incorporates not only a random PDP but also a random angular profile (AP). The model represents scatterers through statistical parameters without having a real physical location. The SCM belongs to the class of geometric stochastic model and separately defines large-scale parameters (e.g., shadow fading, delay spread, and angular spreads) and small-scale parameters (e.g., delays, cluster powers, and arrival- and departure angles). Both parameter sets are randomly drawn from tabulated distributions. The large-scale parameters encompass the geometric positions of the eNodeB sectors and the UEs, respectively. Moreover, they are used to parameterize the statistics of the small scale parameters. The SCM model in [8] includes six different scenarios, each of them representing a unique environment. Initially, it was targeted for a bandwidth of 5 MHz and a carrier frequency of 2 GHz. Later, it was extended to the Spatial Channel Model Extended (SCME). The SCME follows the same procedure as the SCM, but supports bandwidths of up to 100 MHz and a frequency range of 2−6 GHz. In the course of the Wireless World Initiative New Radio (WINNER) projects, the model was extended for 15 different scenarios [9, 10], including urban-, rural-, and moving environments. The WINNER model is recommended as a baseline for evaluating radio interface technologies in the International Telecommunication Union - Radiocommunication Sector (ITU-R) [11].

The interest in 3-dimensional (3D) beamforming is greatly increasing in industry and standardization consortia, enabling concepts such as full dimension (FD)-multiple input multiple output (MIMO) and vertical sectorization [12, 13]. Consequently, describing channel characteristics in three dimensions, including both azimuth- and elevation angles is becoming indispensable. Recently, 3GPP introduced a new 3D SCM for Long Term Evolution-Advanced (LTE-A) in their recommendation TR 36.873 [14].

As of this writing, only a few simulation studies, including reports from the 3GPP TSG RAN WG1 meetings, have been published, claiming the practical implementation of the model [15, 16]. In [17], we have introduced the implementation and validation of the 3GPP 3D channel model in open source simulation tools, considering only the desired signal channels, while the interfering channels are modeled by Rayleigh fading. In this contribution, we extend our approach considering also the interfering links to fade according to the 3GPP 3D channel model. We provide a complete guideline for the practical implementation of the model. The MATLAB source code is openly available for download on our webpage under an academic, non commercial use license [18]. It is provided as a stand-alone package that is directly applicable for system level simulation tools and can straightforwardly be ported to link level. We strongly believe that open access is a key prerequisite for reproducible simulation studies. Moreover, we have made an iteration of the model openly available, with over 100 beta testers and an active online forum. We see it as the only way to ensure the quality of implementation. In this paper, we focus on downlink. According to [14], the channel can be applied for both up- and downlink. For this reason, the notions transmitter and receiver interchangeably refer to the antenna elements of the eNodeB sectors and the UEs, respectively. In practice, eNodeB sectors and UEs will be equipped with antenna arrays, each consisting of one or more antenna elements (conf. Fig. 2). Note that an eNodeB comprises a base band processing unit and can serve multiple sectors or cells.

This contribution outlines as follows. A brief description of the 3GPP 3D channel model is provided in the beginning of Section 2, followed by a guideline for its computationally efficient implementation. In Section 3, the implementation is validated against results from the 3GPP standard with the Vienna LTE-A System Level Simulator. Moreover, simulation run time measurements are provided with respect to various parameters that determine the network complexity. We also compare the 3GPP 3D model with the WINNER channel model. Section 4 outlines challenges and new opportunities for investigations. Section 5 concludes the work.

2 3GPP 3D channel model in system level simulator

2.1 3GPP 3D channel model

The 3GPP 3D channel model characterizes wireless communication channels of typical European cities. It is a 3D geometric stochastic model, describing the scattering environment between eNodeB sector and UE in both azimuth and elevation dimensions. The scatterers are represented by statistical parameters without having a real physical location. In 3GPP TR 36.873 [14], three scenarios, urban macro cell (UMa), urban micro cell (UMi), and UMa-high rise (UMa-H) are specified. They represent typical urban macro-cell and micro-cell environments. Both UMa and UMa-H scenarios, consider an sector antenna height of 25 m, thus surpassing the surrounding buildings. UMa-H also specifies such environments with one high-rise building per eNodeB sector. UMi, considers a sector antenna height of 10 m, lying below the rooftop level. All three environments are assumed to be densely populated with buildings and take into account both indoor- and outdoor UEs.

The 3GPP 3D channel model specifies three propagation conditions, line-of-sight (LOS), non line-of-sight (NLOS) and outdoor-to-indoor (O-to-I). For each of these conditions, it defines different parameters for mean propagation path loss, macroscopic fading, and microscopic fading. All three scenarios in [14], UMa, UMi, and UMa-H, consider 80 % of the UEs to be located indoors. The probability of being in LOS is determined separately for indoor and outdoor UEs and depends on the height of the UE as well as the break point distance. The break point distance characterizes the gap between transmitter and receiver at which the Fresnel zone is barely broken for the first time [19]. For an indoor UE, LOS refers to the signal propagation outside the building in which the UE is located. For each UE location, large-scale parameters are generated according to its geographic position as well as the propagation conditions at this location. The large scale parameters incorporate shadow fading, the Ricean K-factor (only in the LOS case), delay spread, azimuth angle spread of departure- and arrival, as well as zenith angle spread of departure- and arrival.

The small-scale parameters incorporate delays, cluster powers as well as angles of departure and -arrival in azimuth, and elevation direction, respectively. The model considers N clusters of scatterers, where each cluster is resolvable to M paths. A simplified sketch of the model is given in Fig. 1. The channel coefficients are defined per cluster n, sector antenna element s and UE antenna element u as
$$ \begin{aligned}{} H_{u,s,n}(t) &= \sqrt{\frac{P_{n}}{M}} \sum\limits_{m=1}^{M} {\left[ \begin{array}{c} F_{rx,u,\theta}\left(\theta_{n,\, m,\, ZOA},\phi_{n,\, m,\, AOA}\right) \\ F_{rx,u,\phi}\left(\theta_{n,\, m,\, ZOA},\phi_{n,\, m,\,AOA}\right) \\ \end{array}\right]}^{T} \\ & \quad \times {\left[ \begin{array}{cc} \exp \left(j\Psi_{n,m}^{\theta\theta}\right) & \sqrt{K_{n,m}^{-1}}~\exp \left(j\Psi_{n,m}^{\theta\phi}\right) \\ \sqrt{K_{n,m}^{-1}}~\exp \left(j\Psi_{n,m}^{\phi\theta} \right) & \exp \left(j\Psi_{n,m}^{\phi\phi} \right) \end{array}\!\right]} \\ & \quad \times {\left[ \begin{array}{c} F_{tx,s,\theta} \left(\theta_{n,\, m,\, ZOD},\phi_{n,\, m,\, AOD}\right) \\ F_{tx,s,\phi} \left(\theta_{n,\, m,\, ZOD},\phi_{n,\, m,\, AOD}\right) \\ \end{array}\right]} \\ & \quad \times \exp \left(j2\pi\lambda_{0}^{-1} \left(\hat{r}_{rx,\, n,\, m}^{T} \bar{d}_{rx,\, u}\right)\right)\\ & \quad \times \exp \left(j2\pi\lambda_{0}^{-1} \left(\hat{r}_{tx,\, n,\, m}^{T} \bar{d}_{tx,\, s}\right)\right)\\ & \quad \times \exp \left(j2\pi~v_{n,\, m}t\right),\\ \end{aligned} $$
Fig. 1

Scattering concept in the 3D model. Figure demonstrates a link that is resolvable to M paths. Elevation- and azimuth angles at eNodeB sector and UE are denoted as θ and ϕ, respectively

where P n is the power of path n, F rx,u,θ and F rx,u,ϕ are field patterns of the receive antenna element u in the direction of the spherical basis vectors, \(\hat {\theta }\) in the zenith direction, \(\hat {\phi }\) in the azimuth direction. The expressions F tx,s,θ and F tx,s,ϕ are field patterns of the transmit antenna element s in the direction of \(\hat {\theta }\) and \(\hat {\phi }\), respectively. The departure- and arrival angels in zenith and azimuth direction are denoted with θ and ϕ, respectively. The term K n,m represents cross polarization power ratios for each path m and cluster n, and Ψ n,m A B are random initial phases for four different polarization combinations A B={θ θ,θ ϕ,ϕ θ,ϕ ϕ}. The terms \(\hat {r}_{\text {\textit {rx,n,m}}}\) and \(\hat {r}_{\text {\textit {tx,n,m}}}\) are the receiver and transmitter spherical unit vectors expressed in Cartesian coordinates. They are defined as
$$ \hat{r}_{n,m} = \left(\begin{array}{c} \sin\theta\cos\phi \\ \sin\theta\sin\phi \\ \cos\theta \\ \end{array} \right). $$
The parameters \(\bar {d}_{\text {\textit {rx,u}}}\) and \(\bar {d}_{\text {\textit {tx,s}}}\) are the location vectors of receive and transmit antenna elements, respectively. Considering an eNodeB sector with coordinates (s x ,s y ,s z ), and a planar antenna array, the location vector per antenna element is
$$ \bar{d}_{tx,s} = \left(\begin{array}{c} s_{x} \\ s_{y} \\ s_{z} \\ \end{array}\right) + \left(\begin{array}{c} 0 \\ (p-1)d_{H} \\ (q-1)d_{V} \\ \end{array} \right), $$

where p ε P and q ε Q. The terms P and d H denote the number of antenna elements and the element spacing in the horizontal direction, while Q and d V are the number of antenna elements and the element spacing in the vertical direction, respectively. The last component of (1), v n,m , represents the Doppler frequency component of the UE moving at velocity \(\bar {v}\). Further details on the calculation of the variables in (1) can be referred from [14].

2.2 Antenna modeling

The 3GPP 3D channel model enables to scrutinize 2-dimensional (2D) planar antenna arrays, also known as rectangular arrays. The antenna elements can either be linearly polarized (co-pol) or cross prolarized (cross-pol), as shown in Fig. 2. In this regard, the model represents a compromise between practicality and precision as it does not include the mutual coupling effect as well as different propagation effects of horizontally and vertically polarized waves. Our well-structured implementation will substantially facilitate the implementation of further techniques for modeling different polarization modes such as the one proposed in [20].

The antenna elements are equidistantly spaced in the y- and the z-direction. For static electrical beam steering, also known as electrical tilting, a complex weight is applied to each antenna element in the vertical direction. For an antenna element in the q-th row, it is given as
$$ w_{q} = \frac{1}{\sqrt{Q}} \exp \left(-j \frac{2\pi}{\lambda} (q-1)d_{V} \cos\theta_{\text{etilt}} \right), $$
Fig. 2

Geometry and polarization modes of a planar antenna array. The antenna elements in the horizontal and vertical directions are indexed by p and q, respectively

where Q represents the total number of antenna elements in the vertical direction and θ etilt is the steering angle in the vertical plane. Unlike the conventional approach of applying an array factor to the field pattern of a single element of a uniform antenna array, in the 3D model, the beamforming weights are applied to the channel coefficients for each antenna element
$$ \left[\mathbf{H}_{i,n}^{\mathrm{c}}(t)\right]_{a,b} = \sum\limits_{u\epsilon P_{a}} w_{u} \sum\limits_{s\epsilon P_{b}} w_{s} H_{i,u,s,n}(t), $$

where \([\mathbf {H}^{\mathrm {c}}_{\textit {i,n}}(t)]_{\textit {a,b}}\) represents the weighted and combined channel coefficients. The index i indicates the eNodeB sectors, where i=0 denotes the serving sectors, while the indices i={1,2,…} refer to the interfering sectors. The terms \({\mathcal {P}}_{a}\) and \({\mathcal {P}}_{b}\) denote the sets of antenna elements that belong to receive antenna port a with a{1,…,N Rx} and transmit antenna port b with b{1,…,N Tx}, respectively. The terms ω u and ω s are complex weights that account for phase shifts as applied for static beamforming (e.g., electrical downtilting), respectively. The relative position of each element in the array is incorporated in the channel coefficients H u,s,n (t), where n denotes the cluster index, s and u are the eNodeB sector and UE antenna elements, respectively.

In the following, a detailed procedure on the implementation of the model for simulations is provided.

2.3 Implementation in system-level

In this section, we describe the necessary steps to integrate the 3D channel model into an existing simulation tool. The target is to compute a N Rx×N Tx MIMO-channel matrix H(t,f) for each sampling point on the time-frequency grid, where N Tx and N Rx refer to the number of transmit- and receive antenna ports, respectively. On link level, channel realizations are typically calculated per OFDM symbol and LTE-A subcarrier [21]. On system level, they are commonly generated per physical resource block (RB) and transmission time interval (TTI) [22].

In the 3GPP 3D channel model, the channel coefficients depend on the UE location in the 3D space and, thus, have to be calculated at runtime. The calculation complexity directly scales with the number of interfering channels. In view of this, a complexity reduction can be achieved by neglecting the contribution of those interferers that have a received power below a certain threshold (e.g., the noise power). Consider a network scenario with hexagonally arranged macro-sites as illustrated in Fig. 3. The network comprises seven sites, each employing three eNodeB sectors. In each eNodeB sector, 50 UE are randomly distributed according to a uniform distribution. For a 4×4 MIMO configuration, the number of channel coefficients that needs to be generated in each time instant of the simulation is 1.41×108. Hence, in order to reduce complexity, the challenge is to perform computationally intensive tasks off-line or on demand, whenever possible.
Fig. 3

Hexagonal-grid macro-cell scenario with seven macro sites and 21 eNodeB sectors. This scenario is used in simulations throughout the paper

We will follow the stepwise procedure as specified in ([14] Sec. 7.3) and implemented in [17] for the desired channel. This consists of 12 steps, which we will subsequently denote as ’Step N’ with N {1,..,12}, as illustrated in Fig. 4. We will explain its expedient partition for implementation, and emphasize the additional steps for the interfering channels.
Fig. 4

Procedure for generating channel coefficients in 3GPP 3D channel model

[GP] The first step is to generate the general parameters. It starts with specifying the network layout, the scenario environment, and the antenna array parameters (Step 1). Currently, the standard specifies three scenarios, 3D-UMa, 3D-UMi, and 3D-UMa-H, and various planar antenna array structures, defining the location and polarization of each antenna element, as well as the element-to-port mapping. Step 2 is to assign the propagation condition, i.e., either LOS or NLOS, separately for indoor and outdoor UEs. The decision is based on the LOS probability as specified in ([14] Tab. 7.2-2), which is dependent on the UE’s height and its distance to the macro-site. Different path loss models are applied for LOS, NLOS and, O-to-I, as defined in ([14] Tab. 7.2-1). The experienced path loss is calculated in Step 3. In Step 4, the large-scale parameters are generated. The detailed procedure is described in ([10] Sec. 3.3.1). For each UE, a matrix of large scale parameters is generated as
$$ LSP = \left(\begin{array}{cccccc} \delta_{0,\text{SK}} & \delta_{0,\mathrm{K}} & \delta_{0,\text{DS}} & \delta_{0,\text{ASD}} & \cdots & \delta_{0,\text{ZSA}} \\ \delta_{1,\text{SK}} & \delta_{1,\mathrm{K}} & \delta_{1,\text{DS}} & \delta_{1,\text{ASD}} & \cdots & \delta_{1,\text{ZSA}} \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ \delta_{i,\text{SK}} & \delta_{i,\mathrm{K}} & \delta_{i,\text{DS}} & \delta_{i,\text{ASD}} & \cdots & \delta_{i,\text{ZSA}} \\ \end{array} \right), $$

where the serving eNodeB sector is denoted by index 0 and the interfering eNodeB sectors are denoted by the indices i={1,…,I}. In case the UE is not in LOS of BS i, δ i,K=0. These tasks can be performed off-line, i.e., before entering the actual simulation loop. Moreover, they can be carried out simultaneously for serving- and interfering eNodeB tors, allowing to employ, e.g., MATLAB’s parallel computing toolbox. Similar to the generation of the shadow fading, they have to be performed only once per site.

[SSP] i The next step is to generate small-scale parameters for desired and interfering signals. In the 3GPP 3D channel model, channel coefficients H i,u,s,n (t) are determined individually for each eNodeB sector i, each cluster n, and each receiver- and transmitter antenna element pair {u,s}, respectively. Similar to the implementation in [17], the calculation of H i,u,s,n (t) requires to generate delays (Step 5), cluster powers (Step 6) as well as arrival- and departure angles for both azimuth and elevation (Step 7). After coupling the rays within a cluster (Step 8), cross polarization power ratios (XPRs), and random initial phases are drawn (Step 9 and 10). Together with the calculation of the spherical unit vectors and the Doppler frequency component (both Step 11), all parameters mentioned above are commonly applied to each antenna element pair {u,s} and thus have to be determined only once per antenna array and eNodeB sector. The Doppler component accounts for the time variance of the channel. The frequency selectivity is determined by the channel impulse response H i,u,s,n (t) and the sampling frequency, which is directly related to the system bandwidth.

[CG] i After generating the channel coefficients for each antenna element pair {u,s}, the channel coefficients for an antenna array are combined according to the antenna element-to-port mapping given in Sec. 2.2. Then, the combined channel H i,nc(t), for each cluster n is sampled based on the delay taps m defined as
$$ m = \left\lfloor \frac{\tau_{n}}{T_{s}} \right\rfloor, $$
where m{1,…,N}, T s is the sample interval in the time domain and τ n denotes the actual delay of the n-th cluster. The sampled N Rx×N Tx channel matrix is denoted as \(\hat {\mathbf {H}}_{\text {\textit {i,m}}}(t)\), with an element \(\left [\hat {\mathbf {H}}_{\text {\textit {i,m}}}(t)\right ]_{\text {\textit {a,b}}}\), referring to the sampled channel coefficient for receive antenna port a and transmit antenna port b, respectively. It is important to note that the model is designed such that the channel impulse response before sampling has unit sum power on average over t, i.e., \(\mathbb {E}_{t}\left [\sum _{n}|\left [\mathbf {H}^{\mathrm {c}}_{\text {\textit {i,n}}}(t)\right ]_{\text {\textit {a,b}}}|^{2}\right ]=1\), when assuming antenna elements with omni-directional gain pattern and 0 dB gain, as well as an XPR of one. In order not to change the sum power after the sampling, we multiply the sampled channel coefficients \(\left [\hat {\mathbf {H}}_{\text {\textit {i,m}}}(t)\right ]_{\text {\textit {a,b}}}\) by a factor of \(\sqrt {{\sum _{n} |\left [\mathbf {H}^{\mathrm {c}}_{\text {\textit {i,n}}}(t)\right ]_{\text {\textit {a,b}}}|^{2}}/{\sum _{m} |\left [\hat {\mathbf {H}}_{\text {\textit {i,m}}}(t)\right ]_{\text {\textit {a,b}}}|^{2}}}\). The channel transfer function is obtained by performing a fast Fourier transform (FFT) over the sampled and normalized channel impulse response
$$ \tilde{\mathbf{H}}_{i}[k,t] = \sum\limits_{m=0}^{\text{N}-1} \hat{\mathbf{H}}_{i,m}(t) e^{-j2\pi k\frac{m}{\mathrm{N}}}, $$

where, k=0,1,…,N−1. The term N represents the number of FFT samples which is the maximum number of delay taps m. For example, assuming a transmission bandwidth of 10 MHz, according to [23], the sampling interval is T s =65 ns and the number of FFT samples is N=1024.

Considering a UE with a fixed location in the 3D space, [SSP] i and [CG] i have to be carried out only in the first time instant of the simulation. Afterwards, the channel will remain static over time (no Doppler effect). If the UE moves at a certain speed, represented by the vector \(\mathbf {v}\in \mathbb {R}^{3}\), in principle, [SSP] i would have to be performed at runtime in each time instant of the simulation. This also implies the generation of new clusters and random initial phases, i.e., a complete change of the multi-path propagation environment. Thus, it is considered reasonable from a physical perspective (see, e.g., [10]) as well as in view of computational complexity to partition the scenario into equally sized cubes. As long as the UE resides within the same cube, it is assumed to experience the same path loss, shadow fading, propagation conditions (LOS/NLOS/O-to-I), and large scale parameters, as generated in [GP]. Then, [SSP] i has to be carried out only once at the beginning of the simulation and each time the UE transfers to another cube. Assuming a spatial resolution of 1 m and a temporal resolution of 1 ms, referring to the length of one LTE sub-frame, also denoted as TTI, a UE moving at v=[27.78,0,0] m/s requires 36 ms to travel from one face of the cube to the other, as indicated in Fig. 5. In this case, [SSP] i is performed every 36 sub-frames. Within a cube, channel variations are caused by the slightly changing angles of arrival and departure (and thus the antenna element field patterns) as well as the phase shift due to the Doppler effect. They can be incorporated into [CG] i thus yielding the only variable components that have to be recalculated in each time instant of the simulation. If the UE trace is known, e.g., in train and car scenarios, the simulation complexity can be reduced even further. In such scenarios, the Doppler frequency component in (1) has to be calculated only in the first time instant of the simulation, and can be reused in subsequent time instances as long as the user stays within the same cube.
Fig. 5

UE travels through cube with an edge length of 1 m

3 Simulation run times and throughput performance evaluation

We now demonstrate the application of the proposed procedure in an existing system level simulation tool. In [17], we have demonstrated the implementation of the 3GPP 3D model for the desired channel. We now extend this approach by also taking into account the interfering links. We integrate our code into the Vienna LTEA Downlink System Level Simulator (current version v1.8r1375) [22]. The simulator is implemented in object-oriented MATLAB and is made openly available for download under an academic, non-commercial use license. It is built according to the commonly employed structure for system level simulation tools (see, e.g., in [1, 24]), as illustrated in Fig. 6 and, thus, serves as a representative example. Its centerpiece is the link abstraction model that specifies the interaction between link- and system level simulations [1, 22]. This structure is expected to persist in simulation tools for the fifth generation of mobile cellular networks (5G) [24]. The enhancements that were necessary to enable the 3GPP 3D channel model for desired and interfering signals, are depicted by the boxes shaded in gray at the top of the figure.
Fig. 6

Enhanced link abstraction model for enabling 3D channel modeling

3.1 Calibration

For calibration purposes, we carry out simulations with the setup as specified in ([14] Table 8.2-2) and summarized in Table 1. Two scenarios, 3D-UMa and 3D-UMi, are investigated. In [17], the calibration results for large-scale parameter statistics are based on the angular parameters, which are generated in Step 4. In this contribution, we provide the calibration results for large scale parameter statistics, using the circular angle spread method, as recommended in [25]. This method is used to evaluate the angular statistics from the angular parameters a posteriori, i.e., after generating the channel coefficients for each antenna element pair (Step 11). We thus consider this procedure to provide a more reliable verification of our implementation than the previously used method in [17], as it also validates Steps 5-11. Figure 7 depicts the obtained statistics for zenith spread of departure- and arrival. In accordance with the results in [26], the distributions show similar characteristics for 3D-UMa and 3D-UMi scenarios. Furthermore, they exhibit a good agreement with results from [14] (dash-dotted curves), which were obtained by averaging over 21 sources as reported in [27]. In Fig. 8, we provide the calibration results for the largest and smallest singular values as referred from ([14] Table 8.2-2). The singular values are generated on a RB basis at t=0 by considering channel matrices where path loss and shadowing are not yet applied to the channel coefficients. The results show a good agreement with the results from [14] (dash-dotted curves), which were obtained by averaging over 21 sources as reported in [27].
Fig. 7

Large-scale parameter statistics. Gray lines refer to results reported by 21 sources from [27]. Dashed curves denote the 3GPP reference results from ([14] Figure 8.2-11, Figure 8.2-13). a UMa- Zenith spread of departure. b UMi- Zenith spread of departure. c UMa- Zenith spread of arrival. d UMi- Zenith spread of arrival

Fig. 8

Largest and smallest singular value empirical cumulative distribution function (ECDF) in logarithmic scale. Gray lines refer to results reported by 21 sources from [27]. Dashed curves denote the 3GPP reference results from ([14] Figure 8.2-17, Figure 8.2-19). a UMa- Largest (1st) singular value. b UMi- Largest (1st) singular value. c UMa- Smallest (2nd) singular value. d UMi- Smallest (2nd) singular value

Table 1

Simulation parameters for calibration as referred from [14]



Carrier frequency

2 GHz

LTE bandwidth

10 MHz

Macro-site deployment

Hexagonal grid


3D-UMa, 3D-UMi

Sector antenna height (UMa)

25 m

Sector antenna height (UMi)

10 m

Sector antenna configuration

N Tx=4

UE antenna configuration

N Rx=2

Polarized antenna modeling

Model 2 [14]

Sector antenna polarization

X-pol (+/−45°)

UE antenna polarization

X-pol (0/+90°)

Antenna elements per port


Vertical antenna element spacing


Horizontal antenna element spacing


Maximum antenna element gain

8 dBi

UE antenna pattern

Isotropic antenna gain

Electrical downtilt


UE distribution

Uniform in cell ([14] Tab. 6-1)

3.2 Simulation run times

In this section, we measure the simulation run times of the 3GPP 3D channel model in the Vienna LTE-A Downlink System Level Simulator. The goal is to observe how the 3GPP 3D channel model affects the simulation run time. Note that applying this model does not alter the signal processing part of the simulation tool. For a fair comparison, all simulations were carried out on the same hardware, an Intel(R) Core(TM) i7-3930K CPU@3.20 GHz, equipped with 32 GB of DDR3 1333 quad-channel RAM.

The network comprises seven hexagonally arranged macro-sites, each employing three eNodeB sectors spaced out 120°. Hence, a UE will experience a maximum of N sector=20 interfering eNodeB sectors. We carry out simulations with the antenna port configuration N Tx×N Rx=4×2 and M={8,24,40,80} antenna elements at the transmitter, yielding Q={2,6,10,20} antenna elements per port (see Fig. 12). We evaluate various simulation lengths N TTI={10,50,100}, where N TTI denotes the number of simulated 1 ms subframes (also termed TTI), and consider K={2,20,50} UEs per eNodeB sector. The results are averaged over five simulation runs per individual configuration. The simulation parameters are summarized in Table 2. Figure 9 provides the results in terms of simulation run time measured in [ s]. In Fig. 9 a, it is observed that the results scale approximately linear with the number K of UEs and with the simulation length N TTI. Such behavior was already observed in [22] together with a more detailed description of its origins. In this paper, our main interest is on the impact of the elevation direction and the application of the 3GPP 3D channel model for interfering channels on the simulation run time. We further observe that the simulation run time increases with a high number M of antenna elements. In order to make that trend more clear, we depict the simulation run time over M for various numbers K of UEs. We note that the run times grow approximately linearly with the number of antenna elements per antenna array. This confirms the expected result that the price to pay for modeling more than one antenna element per antenna port is a linear increase in complexity for each additional antenna element per antenna port.
Fig. 9

Simulation run times for various antenna array geometries, simulation lengths N TTI, number K of UEs, and number N sector of interfering eNodeB sectors. a Fixed number of interfering eNodeB sectors (one tier). b Varying number of interfering eNodeB sectors for N TTI=20

Table 2

Simulation setup



Carrier frequency

2 GHz

LTE bandwidth

10 MHz

Macro-site deployment

hexagonal grid, one tier

Interfering eNodeB sectors

N sector={2,8,14,20}

Number of UEs per cell


Antenna elements per antenna array


Antenna elements per antenna port


3GPP scenario


Inter-site distance

500 m

eNodeB transmit power

46 dBm

Antenna element gain pattern

3D pattern ([14] Tab. 7.1-1)

Antenna polarization


Polarized antenna modeling

model 2 ([14] Sec. 7.1.1)

Maximum antenna element gain

8 dBi

Vertical antenna element spacing


Horizontal antenna element spacing


UE distribution


UE speed

3 km/h

UE antenna gain pattern


UE antenna array polarization


Wrapping method

center UEs evaluated

Receiver type

zero forcing

Channel knowledge


Feedback delay


Noise power density

−174 dBm/Hz

LTE transmission mode



proportional fair

Traffic model

full buffer

Simulation length

N TTI={10,50,100}

Since we model both the desired as well as the interfering channel by the 3GPP 3D channel model, in the next step, we investigate how simulation run times scale by successively increasing N sector, starting with two interfering eNodeB sectors. The selection of interfering sectors is carried out randomly. For example, N sector=2 means that two out of the 20 interfering eNodeB sectors were taken into account but might not necessarily be the two strongest ones. Note that the strength of the interferers is of no relevance for our evaluation as we are mainly interested in simulation run times, i.e., the complexity of generating the interfering channels by means of the 3GPP 3D channel model. Assuming K={2,20,50} UEs per cell at a simulation length of 20 TTI, we consider N sector={2,8,14,20} interfering eNodeB sectors and M={8,24,40} antenna elements per antenna array. The simulation results are provided in Fig. 9 b. It is found that the run time scales linearly with the number of interfering eNodeB sectors. On the other hand, for a large number of interfering eNodeB sectors, there is a non-linearity with the number of UEs. This results from the fact that we neglect the contribution of those interferes that have a received power below a certain threshold (e.g., the noise power). Hence, the UEs will in general experience a different amount of interfering eNodeB sectors. Furthermore, we investigate simulation run times over the number M of antenna elements assuming two simulation lengths N TTI={50,100} and including all the interfering eNodeB sectors. The simulation results are provided in Fig. 10. Again, the results exhibit an approximately linear growth in the simulation run time over M antenna elements for various number K of UEs and various simulation lengths. Hence, in scenarios with large antenna arrays, the simulation run time will scale roughly proportional with the number of antenna elements.
Fig. 10

Simulation run time [s] over number of antenna elements M. Dashed lines denote a simulation length of N TTI=50 TTI, solid lines refer to a simulation length of N TTI=100 TTI

In the next step, we compare two types of spatial channel models, the 3GPP 3D model and the WINNER model. In these two models, the generation of the channel coefficients follows a similar procedure. The main difference is that the WINNER model is a 2D model, i.e., it does not incorporate the elevation dimension. Consequently, it only allows to apply linear antenna arrays with Q=1. In contrast, the 3GPP 3D model enables to scrutinize 2D antenna arrays. We carry out simulations with an antenna port configuration of N Tx×N Rx=4×2. In order to unveil the impact of including the elevation information onto the simulation run time, we consider Q={1,2,10} antenna elements in vertical dimension, while we are restricted to Q=1 in the WINNER model. We evaluate various simulations lengths N TTI={10,50,100} and consider K={2,20,50} UEs per eNodeB sector. The results in terms of simulation run times are provided in Fig. 11. When comparing the simulation run times of the 3GPP 3D channel model and the WINNER model using a linear antenna array, it is observed that for K=50 UEs and N TTI=100, the run time increases by a factor of 3.1. Thus, by taking into account the elevation dimension, the simulation run time more than triples. For planar antenna arrays with Q=10, the run time is increased by a factor of 27.49, when comparing with the WINNER model. Hence, the complexity grows roughly proportional with the number of antenna elements, which will become a prominent factor for simulations of future scenarios with large antenna arrays.
Fig. 11

Simulation run times [s] of two channel models: 3GPP 3D model and WINNER model for various simulation lengths N TTI and number K of UEs. For observing the elevation information added in the 3GPP 3D model, a linear array with Q=1 and a planar array with Q=2 and Q=10 are considered, while the WINNER model employs a linear array

3.3 Throughput performance evaluation

In this section, the impact of modeling both the desired as well as the interfering channels with the 3GPP 3D channel model is scrutinized. We consider a network with seven macro-sites, each employing three eNodeB sectors, and simulate 50 randomly distributed UEs per eNodeB sector. The simulation parameters are summarized in Table 2.

3.3.1 Rayleigh versus 3GPP 3D model

We carry out a performance comparison in terms of average UE throughput considering: (1) a noise-limited network as a baseline, (2) interference channel coefficients modeled by Rayleigh fading as used in [17], and, (3) interference channel coefficients generated by the 3GPP 3D channel model. We consider four antenna ports at the transmitter, i.e., N Tx=4 and a planar antenna array with Q=10 linearly polarized antenna elements per antenna port. The antenna elements Q of an individual port are stacked in vertical direction with a spacing of 0.5λ, as indicated in Fig. 12. This configuration is known as sub-array partition model since the same weight vector ω q is applied for each port ([28] Sec. 5.2.2). The UEs are equipped with a linear antenna array consisting of two antenna elements, each being associated to an individual antenna port. Figure 13 depicts the results in terms of average UE throughput statistics. It is observed that interference channel coefficients modeled by the 3GPP 3D model provide a more optimistic view on the performance compared to Rayleigh fading, as applied in [17]. The noise-limited scenario serves as a best-case reference. A major difference between channels modeled by the 3GPP 3D model and Rayleigh fading stems from the fact that in the 3GPP 3D model, the antenna element field patterns are already incorporated in the channel coefficients, while in the Rayleigh fading case, they only affect the path loss. Consequently, the 3GPP 3D channel model has a stronger tendency to result in quasi-orthogonal signal spaces of desired and interfering signals. A systematic proof of this statement goes beyond the scope of this paper and is left for further work. Practically, our results indicate that simple channel models may underestimate the throughput performance.
Fig. 12

Antenna array port virtualization applied on system-level for co-pol and cross-pol polarization modes. The antenna ports are denoted as p i with i{1,…,4}. The parameters ω q , q{1,…,Q} are the phase shifts for static beamforming (e.g., electrical downtilting)

Fig. 13

Average UE throughput [Mbit/s] ECDF curves considering three different channel models for interfering links

3.3.2 2D- versus 3D channel modeling

We evaluate system level performance in terms of average UE throughput and average UE spectral efficiency applying the 3GPP 3D channel model and the WINNER channel model. The MATLAB implementation of the WINNER channel model is provided in [29]. For the simulation of both models, we consider the same network layout as shown in Fig. 3 and simulation parameters as given in Table 2. Our network comprises 50 UEs per cell and an antenna port configuration of N Tx×N Rx=4×2. For a fair comparison, we employ a UMa scenario. In the WINNER model, this corresponds to scenario C2 and is equivalent to 3GPP 3D-UMa scenario. The WINNER model only allows to apply linear antenna arrays with Q=1, whereas for the 3GPP 3D channel model, we consider a linear antenna array with Q=1 as well as a planar antenna array with Q=10 antenna elements in vertical direction. In the planar antenna array case, we scrutinize two electrical downtilt settings θ etilt={0°,10°}. In the 3GPP 3D-UMa scenario, both desired and interfering channels are modeled by the 3GPP 3D model, whereas in the WINNER C2 scenario, all channels are abstracted by the WINNER model. Figure 14 depicts the simulation results in terms of average UE throughput. It is observed that there is a distinct gap between the 3GPP 3D model and the WINNER model. This mainly stems from the fact that the UMa scenarios of the two models employ slightly different parameter values, e.g., for cross-correlation of the large scale parameters and the mean and standard deviation of delay- and angular spreads. When increasing the size of the antenna array to Q=10, the elevation dimension considered in the 3GPP 3D model remarkably results in similar throughput values. The results indicate that antenna arrays with Q=10 antenna elements per column may not achieve such effective confinement of energy as largely envisioned. Moreover, the interference suppression is highly sensitive to small errors in the weights applied to each of the antenna elements [30]. These statements have to be substantiated by systematic investigations, yielding an interesting topic for further work. Moreover, in the 3D-UMa scenario, it is observed that when increasing the electrical downtilt angle from θ etilt=0° to θ etilt=10°, a slight performance improvement is achieved. The effect of electrical downtilting is paled by the varying UE heights in this scenario. Figure 15 depicts the simulation results in terms of average UE spectral efficiency [bit/s/Hz] statistics. It is observed that for the 3GPP 3D channel model both Q=1 and Q=10 show a similar performance for high spectral efficiency values. Similar to the throughput case, tilting slightly impacts the performance.
Fig. 14

Average UE throughput [Mbit/s] ECDF curves for WINNER and 3GPP 3D channel models. Solid and dashed lines show the performance as achieved by setting the electrical tilting at 10° and 0°, respectively. For the 3GPP 3D model a linear and a planar array denoted by Q=1 and Q=10 are considered

Fig. 15

Average UE spectral efficiency [bit/s/Hz] ECDF curves for WINNER and 3GPP 3D channel models. Solid and dashed lines show the performance as achieved by setting the electrical tilting at 10° and 0°, respectively. For the 3GPP 3D model a linear and a planar array denoted by Q=1 and Q=10 are considered

4 New opportunities and challenges

The integration of the 3D channel model into existing link- and system-level simulation tools paves the way for more advanced studies on the performance of a mobile cellular system in realistic environments. Existing channel models only support linear antenna arrays in the azimuth. With the introduction of the third dimension, not only higher-order MIMO schemes, but also a higher number of antenna elements per antenna array can be investigated. Currently, the 3GPP LTE-A standard supports up to eight antenna ports. However, recent trends aim at 100 and more antenna ports per eNodeB sector [31]. A main enabler for this so called massive MIMO approach will be the adoption of higher carrier frequencies, also termed millimeter-wave communication, as it enables to considerably decrease the size of the antenna arrays. On the one hand, this may lead to higher complexity of the hardware, larger energy consumption and a greater demand for signal processing capabilities. On the other hand, it will enable a much more accurate bundling of energy towards the intended receiver, which is a key prerequisite for aggressive frequency reuse. In dense urban environments, where UEs move in three dimension (consider, e.g., shopping malls, skyscrapers, and more), it is conceivable that the spectral efficiency per unit sphere might replace the area spectral efficiency as a figure of merit. Other important use cases are scenarios with high user mobility, as the number of commuters is expected to increase substantially. People have become used to services following them wherever they travel. Mobile cellular access has even become a key argument to choose the means of transportation. Sharp, steerable beams might be an expedient solution to this issue, as they could follow a vehicle along its path.

Improvements targeting planar antenna arrays are to be further investigated. New virtualization models of antenna arrays, considering a full-connection between antenna elements, weighted in both horizontal-and vertical directions will lead to a better understanding of the 3D beamforming. Moreover, new two-dimensional codebook designs are necessary for the evaluation of FD-MIMO.

5 Conclusions

This work presented a guideline for the practical implementation of the 3GPP 3D channel model into existing link-and system level simulation tools. In comparison to previous work in [17], we faced the challenge of calculating the channel coefficients at simulation runtime for both desired- and interfering channels by carefully partitioning the step-wise procedure as proposed by 3GPP. We demonstrated the behaviour of the 3GPP 3D model in terms of simulation run time. A comparison against the WINNER channel model indicates that the incorporation of the elevation dimension increases the computational complexity by more than three times. Furthermore, we observed that the complexity grows roughly linearly with the number of antenna elements per antenna array. Hence, it may become one of the dominant factors that affects the simulation run time in future massive MIMO scenarios. We showed that, compared to Rayleigh fading interference channels, as applied in [17], when the interfering channels are abstracted by the 3GPP 3D channel model, a more optimistic view on the performance is obtained. This result indicates that more simple channel models may underestimate the achievable performance. Next, we compared linear against planar antenna arrays. Remarkably, taking into account a planar antenna array results in performance degradation. This result indicates, that the beams might not be as sharp as largely envisioned. A systematic evaluation of this behavior is left for further work. The paper is completed by an elaboration on new opportunities that became possible with the 3D channel model, and the challenges ahead to realize the full potential of the upcoming new technologies. Our implementation approach is openly accessible, and our hope is to inspire researches and developers of link- and system level simulation tools to further elaborate and develop these topics by applying it.



This work has been funded by A1 Telekom Austria AG, and the KATHREIN-Werke KG. The financial support by the Federal Ministry of Economy, Family and Youth and the National Foundation for Research, Technology and Development is gratefully acknowledged.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

Christian Doppler Laboratory for Dependable Wireless Connectivity for the Society in Motion, TU Wien, Institute of Telecommunications


  1. S Ahmadi, LTE-Advanced: a practical systems approach to understanding 3GPP LTE releases 10 and 11 radio access technologies (ITPro collection, Elsevier Science, 2013).Google Scholar
  2. J Ikuno, M Wrulich, M Rupp, in Vehicular Technology Conference (VTC 2010-Spring). System level simulation of LTE networks, (2010), pp. 1–5. doi:n10.1109/VETECS.2010.5494007.View ArticleGoogle Scholar
  3. A Rodriguez-Herrera, S Mcbeath, D Pinckley, D Reed, in IEEE 62nd Vehicular Technology Conference, VTC-2005-Fall, 3. Link-to-system mapping techniques using a spatial channel model, (2005), pp. 1868–1871. doi:10.1109/VETECF.2005.1558430.Google Scholar
  4. P Matthias, Mobile fading channels (John Wiley and Sons, Inc., New York, NY, USA, 2002).Google Scholar
  5. J Parsons, The mobile radio propagation channel, 2nd edn. (John Wiley and Sons, Inc., New York, NY, USA, 2000).View ArticleGoogle Scholar
  6. P Almers, E Bonek, A Burr, N Czink, M Debbah, V Degli-Esposti, H Hofstetter, P Kyosti, D Laurenson, G Matz, A Molisch, C Oesteges, H Ozcelik, Survey of channel and radio propagation models for wireless MIMO systems. EURASIP J. Wireless Commun. Networks (2007). doi:10.1155/2007/19070.
  7. A Paulraj, R Nabar, D Gore, Introduction to space-time wireless communications (Cambridge University Press, 2003).Google Scholar
  8. 3GPP TR36.996, Spatial channel model for multiple input multiple output (MIMO) simulations (3rd Generation Partnership Project (3GPP), 2003).Google Scholar
  9. WINNER I WP5, Final report on link level and system level channel models. IST-2003-507581 WINNER I Deliverable D5.4 (2005).
  10. WINNER II WP1, WINNER II channel models. IST-4-027756 WINNER II Deliverable D1.1.2 (2007)
  11. ITU-R and M.2135, Guidelines for evaluation of radio interface technologies for IMT-Advanced. Report (2009).
  12. E Larsson, O Edfors, F Tufvesson, T Marzetta, Massive MIMO for next generation wireless systems. IEEE Commun. Mag. 52(2), 186–195 (2014). doi:10.1109/MCOM.2014.6736761.View ArticleGoogle Scholar
  13. Y Song, X Yun, S Nagata, L Chen, in IEEE International Conference on Communications Workshops (ICC). Investigation on elevation beamforming for future LTE-Advanced, (2013), pp. 106–110. doi:10.1109/ICCW.2013.6649210.Google Scholar
  14. 3GPP TR36.873, Study on 3D channel model for LTE, v12.2.0 (3rd Generation Partnership Project (3GPP), 2014).Google Scholar
  15. A Kammoun, H Khanfir, Z Altman, M Debbah, M Kamoun, Preliminary results on 3D channel modeling: From theory to standardization. IEEE J. Selected Areas Commun.32(6), 1219–1229 (2014). doi:10.1109/JSAC.2014.2328152.View ArticleGoogle Scholar
  16. Z Hu, R Liu, S Kang, X Su, J Xu, in International Conference on Communications and Networking in China (CHINACOM). Work in progress: 3D beamforming methods with user-specific elevation beamfoming, (2014), pp. 383–386. doi:10.1109/CHINACOM.2014.7054323.Google Scholar
  17. F Ademaj, M Taranetz, M Rupp, Implementation, Validation and Application of the 3GPP 3D MIMO Channel Model in Open Source Simulation Tools. Paper presented at the Twelfth International Symposium on Wireless Communication Systems, ISWCS, 25-28 August 2015.Google Scholar
  18. Vienna LTE-A Simulators. Accessed 14 Aug 2015.
  19. H Masui, T Kobayashi, M Akaike, Microwave path-loss modeling in urban line-of-sight environments. IEEE J. Selected Areas Commun.20(6), 1151–1155 (2002). doi:10.1109/JSAC.2002.801215.View ArticleGoogle Scholar
  20. A Weber, A Bestard, in 8th International Symposium on Wireless Communication Systems (ISWCS), 2011. Modeling of X-pol antennas for LTE system simulation, (2011), pp. 221–225. doi:10.1109/ISWCS.2011.6125342.View ArticleGoogle Scholar
  21. S Schwarz, J Ikuno, M Simko, M Taranetz, Q Wang, M Rupp, Pushing the limits of LTE: A survey on research enhancing the standard. IEEE Access. 1:, 51–62 (2013). doi:10.1109/ACCESS.2013.2260371.View ArticleGoogle Scholar
  22. M Taranetz, T Blazek, T Kropfreiter, MK Müller, S Schwarz, M Rupp, Runtime precoding: enabling multipoint transmission in LTE-Advanced system-level simulations. IEEE Access. 3:, 725–736 (2015). doi:10.1109/ACCESS.2015.2437903.View ArticleGoogle Scholar
  23. 3GPP TS 36.201, LTE Physical Layer - General Description, v9.0.0 (3rd Generation Partnership Project (3GPP), 2009).Google Scholar
  24. Y Wang, J Xu, L Jiang, Challenges of system-level simulations and performance evaluation for 5G wireless networks. IEEE Access. 2:, 1553–1561 (2014). doi:10.1109/ACCESS.2014.2383833.View ArticleGoogle Scholar
  25. 3GPP 325.996, Spatial channel model for Multiple Input Multiple Output (MIMO) simulations, v12.0.0 (3rd Generation Partnership Project (3GPP), 2012).Google Scholar
  26. 3GPP TSG RAN WG-1 R1-140048, Phase 2 calibration results for 3D channel model. Meeting report, (2014).Google Scholar
  27. 3GPP TSG RAN WG-1 R1-143469, Summary of 3D-channel model calibration results. Meeting report, (2014).Google Scholar
  28. 3GPP TR36.897, Elevation Beamforming/Full-Dimension MIMO for LTE, v0.3.1 (3rd Generation Partnership Project (3GPP), 2015).Google Scholar
  29. L Hentila, P Kyosti, M Kaske, M Narandzic, M Alatossava, Matlab implementation of the winner phase II channel model ver1.1 (2007). Accessed Sep. 2008.
  30. O Bakr, M Johnson, R Mudumbai, U Madhow, in 47th Annual Allerton Conference on Communication, Control, and Computing Allerton. Interference suppression in the presence of quantization errors, (2009), pp. 1161–1168.Google Scholar
  31. F Rusek, D Persson, BK Lau, E Larsson, T Marzetta, O Edfors, F Tufvesson, Scaling up MIMO: Opportunities and challenges with very large arrays. IEEE Signal Process. Mag. 30(1), 40–60 (2013). doi:10.1109/MSP.2011.2178495.View ArticleGoogle Scholar


© Ademaj et al. 2016