The evaluation of the proposed iCRRM entity involved evaluating the performance of several simulation scenarios:
-
1.
Two LTE systems operating separately at 800 MHz and 2.6 GHz, i.e. without CA, considering the M-LWDF scheduler and either 5-MHz or 20-MHz bandwidths;
-
2.
One LTE-A scenario with both frequency bands managed with basic CRRM functionalities (basic multi-band scheduling), only considering a bandwidth of 5 MHz;
-
3.
One LTE-A scenario with both frequency bands managed with the proposed iCRRM entity:
-
(a)
One set performed with the general multi-band scheduler, only considering a bandwidth of 5 MHz;
-
(b)
One set performed with the enhanced multi-band scheduler, bandwidths of either 5 MHz or 20 MHz.
Each scenario was simulated 20 times, these results have been averaged and the confidence intervals have been determined. In the case of 5-MHz bandwidth, the analysis of average cell PLR and delay are performed by averaging the results from scenario 1 while comparing them with the ones from scenarios 2 and 3. In terms of average cell-supported goodput analyses, the system capacity obtained in scenario 1 is summed and compared with the results from scenarios 2 and 3. Furthermore, it is worthwhile to note that the following results obtained without CA have been compared and are well within the range of ones obtained in the packet scheduling algorithms studies performed in [23,26] and [24]. Innovative results considering a bandwidth of 20 MHz are compared between scenario 1, without CA, and scenario 3(a), which only considers the EMBS proposal.
Packet loss ratio
Figure 3 shows the average cell PLR as a function of the number of UEs for a cell with R = 1,000 m. A bandwidth of 5 MHz and the H.264 128-kbps video flow have been initially considered. As expected, both CRRM (BMBS) and iCRRM outperform the scenario without CA.
However, according to the ITU-T G.1010 [27] and 3GPP TS 22.105 [28] recommendations, the PLR should not exceed 1%. In this context, the 1% PLR threshold is only exceeded above approximately 58 UEs with iCRRM general (GMBS) and enhanced (EMBS) multi-band scheduler (1.03% and 0.88%, respectively), whereas CRRM only supports up to 54 UEs (0.99%). Without CA, the minimum obtained PLR is approximately 2%. Besides, it is also verified that overall, the EMBS enables to obtain values for the PLR lower than the ones with the GMBS. Figure 4 shows again the obtained results by magnifying Figure 3. The average PLR without CA (Average 2.6 GHz and 800 MHz) is not shown, since the minimum obtained value is superior to 2%, which is the maximum value shown for PLR.
Additionally, the 20-MHz bandwidth scenario has been assumed together with the use of the H.264 3.1-Mbps video flow. Figure 5 shows the average cell PLR as a function of the number of UEs for cell radius R = 1,000 m. In this case, the 1% PLR threshold is only exceeded above approximately 40 UEs (PLR = 0.65%). This clearly shows that, with EMBS, PLR considerably decreases.
Delay
The average cell delay for R = 1,000 m is shown in Figure 6 for the scenario with 5-MHz bandwidth. As in the previous case, both CRRM (BMBS) and iCRRM present better results than without CA, i.e. lower delay. Moreover, iCRRM’s EMBS outperforms the GMBS. Similarly to the PLR, ITU-T G.1010 [27] and 3GPP TS 22.105 [28] also define delay performance targets, i.e. 150-ms preferred and 400-ms limit delay. For the considered number of UE, neither of these targets is exceeded. Nonetheless, when the previous 1% performance target is exceeded, i.e. 54 and 58 UEs, with CRRM and iCRRM (EMBS and GMBS), respectively, the achieved delay for the 5-MHz bandwidth case is approximately 11.22, 11.44 and 7.68 ms with CRRM and iCRRM’s GMBS and EMBS scheduler, respectively. Without CA, the average cell delay is always superior to the ones from the above cases.
It is worthwhile to compare the results from the 5-MHz bandwidth scenario with the ones from the 20-MHz bandwidth one. Figure 7 shows the average cell delay for R = 1,000 m while comparing the cases without CA and the use of EMBS for both values of the bandwidth.
With EMBS, when the 1% PLR performance target is exceeded, i.e. 54 and 40 UEs using 5 MHz and 20 MHz CCs, respectively, the achieved delay is approximately 8 and 2.4 ms (128 kbps and 3.1 Mbps video clips, respectively). Without CA, the average cell delay is always considerably superior to the ones from the previous cases.
Quality of experience
The permanent evolution of wireless network technologies allows for improved data rates and coverage areas while facilitating new multimedia and mobile services. Considering this evolution of services and applications, operator’s success does not only depend on their QoS but also if it meets the end user’s expectations. With the increasing competition, improving the quality of the offered services, as perceived by the users (QoE), becomes important as well as a significant challenge to service providers with a goal to minimize the customer churn yet maintaining their competitive edge [29]. However, QoS is generally defined in terms of network delivery capacity and resource availability but not in terms of the satisfaction to the end-user. QoE is very subjective in nature, the most accurate approach to evaluate it is the subjective quality assessment, since there is no better indicator of personal quality than the one given by a human being.
The existing works cited in [8,19,30,31] mainly focus on the quality of service (QoS) performance of CA system. However, these days, service providers are switching the focus from network QoS to user quality of experience (QoE) to provide their services in the most cost- and resource-efficient manner with ensured user satisfaction. Therefore, the resource allocation algorithm based on various QoE contributing factors such as throughput, jitter, cost and reliability should be required in the process of resource allocation for the CA system.
It is important that QoE is expressed as a function of the network and equipment that influence user behaviour and result in a certain level of QoE. Therefore, QoE data should succeed whenever possible in combining both user experience and technical measures; for example, to provide an equation for the user experience when using a particular service with known levels of QoS [32]. As such, QoS metrics gathered from various parts of the network must be mapped onto QoE targets, facilitating the inclusion of the end-user perception into the QoE model. In this context, besides assessing the network service level parameters, the QoE can also be evaluated by employing the model for the mapping between QoS and QoE proposed in [33]. This model addresses multimedia applications, gaming, video, web-browsing and audio. The video sub-model is based on the mean opinion score (MOS) measurements which are mathematically fitted to obtain the following equation:
$$\begin{array}{*{20}l} {}\text{QoE} =& 3.2147 -0.00266916 \times b_{\text{rate}}-10.4811 \\ &\times d-20.9894 \times \rho -5.8875 \times 10^{-6} \\ &\times b^{2}_{\text{rate}} +40.3305 \times d^{2} +166.121 \times \rho^{2} +1.449\\ &\times 10^{-8} \times b^{3}_{\text{rate}} -42.493 \times d^{3} -730.016 \\ &\times \rho^{-3} -4.2939 \times 10^{-12} \times b^{4}_{\text{rate}} + 18.3884 \times d^{4} \\ &+1764.47 \times \rho^{4} -2.29851 \times 10^{-15} \times b^{5}_{\text{rate}} \\ & -3.48213 \times d^{5} -2069.09\! \times\! \rho^{5} + 8.08679 \times \!10^{-19}\\ &\times b^{6}_{\text{rate}} +0.237418 \times d^{6} +903.102 \times \rho^{6} \end{array} $$
((6))
where d is the delay, in ms, ρ is the percentage of loss and b
rate is the video bitrate, in kilobit per second. The goodness of fit is confirmed by the coefficient of determination, R
2=0.84, and the mean square error (MSE), MSE=0.197.
Considering this model and previous average cell PLR and delay simulation results (with b
rate=128 kbps), Figure 8 shows the predicted average cell QoE as a function of the number of active UEs in cells with R = 1,000 m.
From Figure 8, it is clear that employing CA improves the average cell QoE. Without CA, an average QoE of 2.7 is obtained below 28 UEs; beyond this values, the quality substantially decreases and reaches its lower value with the maximum considered UEs. With CA, as expected by the obtained PLR and delay, the EMBS provides the better results followed by the GMBS and CRRM (BMBS). Moreover, it is interesting to note that, as expected by ITU-T G.1010 [27] and 3GPP TS 22.105 [28], the higher decline of the estimated QoE value occurs approximately with the same number of UEs from which the 1% PLR is exceeded, i.e. 54 and 58 UEs with CRRM and iCRRM, respectively, corresponding to a value of the QoE of circa 2.81 and 2.86, respectively.
In terms of QoE, it is also worthwhile to compare the results between the former 5-MHz bandwidth scenario and the 20-MHz bandwidth one (video with b
rate=3.1 Mbps). Figure 9 shows the predicted average cell QoE as a function of the number of active UEs, in cells with cell radius R = 1,000 m.
It is observable that for the 20-MHz bandwidth without CA, the behaviour of QoE is not as regular as it was for the 5-MHz bandwidth. However, when EMBS is employed, the QoE curve recovers its regular behaviour. It is also clear that the values of QoE raise from around 2.86 up to circa 3.96 (the value of QoE that corresponds to PLR = 1% for the 5-MHz and 20-MHz bandwidths, respectively).
Goodput
The variation of the supported average goodput with the number of UEs is shown in Figure 10 considering R = 1,000 m and CCs with bandwidth of 5 MHz. In this case, the performance gap between iCRRM, CRRM and BMBS is less apparent. With the exception of the case without CA, all the remaining scenarios can support the cell traffic requirement for PLR = 1% up to approximately 54, 58 and 58 UEs with CRRM (BMBS), iCRRM (GMBS) and iCRRM (EMBS), respectively. However, it is clear that as the number of UEs within the cell increases so does the iCRRM performance gain, in comparison with the results both widths CRRM and without CA results. In the context of the iCRRM, it has also been shown that above 58 UEs, the goodput obtained with the EMBS is higher than the one obtained with the GMBS.
Additionally, it is also important to consider the supported goodput within ITU-T G.1010 [27] and 3GPP TS 22.105 [28] performance target. In this context, considering the number of UE supported within the 1% PLR margin, i.e. 58 and 54 UEs, with iCRRM and CRRM (BMBS), respectively, the supported goodput improvement between both RRM is evident, as shown in Figure 11. With iCRRM, an average of 7,480 and 7,400 kbps are supported, with the EMBS and GMBS, respectively, whereas only 6,900 kbps is supported with CRRM (BMBS). The case without CA is not considered, since the lowest obtained PLR is approximately 2%.
Similarly to the above ITU-T and 3GPP target performance considerations, one should bear in mind the ITU-T ACR scale [34] whose lower QoE value is 1 and highest is 5. When a bandwidth of 5 MHz is considered, as the delay threshold is not reached, the parameter that limits the definition of the QoE threshold is the PLR (which should be less than 1%). A value of 2.86 was identified for the EMBS (58 UEs), while CRRM (BMBS) and GMBS correspond to a value of 2.81 for QoE (54 and 58 UEs, respectively). A value equal to 2.86, achieved for 58, 54 and 52 users (for EMBS, GMBS and BMBS, respectively), is henceforward considered as a threshold below which the QoE is not acceptable. In this context, considering the results from Figure 8, without CA, the average cell QoE is never considered sufficient. With CA, the 2.86 threshold is no longer achieved above the same number of UEs as for the previous PLR analysis. Finally, the number of UEs supported by the cell below this QoE threshold can also be reflected in terms of average supported cell goodput, as shown in Figure 12. Given these considerations, the average supported goodput is approximately 7,480, 7,010, and 6,740 kbps with iCRRM (EMBS), iCRRM (GMBS) and CRRM (BMBS), respectively.
The analysis of the behaviour from the goodput while considering the 20-MHz bandwidth leads us to the curves of the cell average goodput as a function of the number of UEs from Figure 13, where R = 1,000 m is assumed. With EMBS and 5-MHz bandwidth, a maximum value of 9.2 Mbps is obtained for the average cell goodput (for 80 UEs, here PLR is circa 7%), whereas for the 20-MHz bandwidth the goodput is linearly increasing until it reaches a value of 71.53 to 75.30 Mbps (for 40 to 44 UEs); then it takes slightly lower values. Without CA, values of only 6.5 and 41.2 Mbps are achieved (for 76 and 80 UEs, respectively).
It is worthwhile to analyse the value for the goodput which corresponds to number of users supported under the 1% PLR performance target (since the 150-ms threshold has not been reached). In this context, with EMBS, the corresponding values for the average supported cell goodput are approximately 7.48 and 71.53 Mbps (9.56 times increase), for the 5 and 20 MHz (58 and 40 UEs), respectively. Results without CA are not considered since the PLR performance target is always exceeded.
Figure 14 shows the average supported cell goodput as a function of the cell radius, R, for the number of users supported under the 1% PLR threshold. In this case, the formulation to compute the transmitter power required to guarantee a similar average SINR for different values of the cell radius is the one from [11]. The transmitter power has been normalized so that comparable results between CCs are assured and eNBs from lower cell radius have reduced energy consumption.
The average cell spectral efficiency has been computed as the ratio between the goodput and the CCs bandwidth, i.e. 5 and 20 MHz (for 128-kbps and 3.1-Mbps video clips). Figure 15 shows the average cell spectral efficiency and corresponding percentage of gain (between EMBS and without CA). Considering the number of UEs, supported under the PLR threshold, with EMBS, the value for the spectral efficiency is 1.788 b/Hz/cell (for 40 UEs) and 0.889 b/Hz/cell without CA, in the 20-MHz case. In turn, for bandwidth of 5 MHz, the spectral efficiency is only 0.75 b/Hz/cell (for 58 UEs) and 0.62 b/Hz/cell without CA. Compared to the case without CA, the corresponding percentage of gain is 101.2% (for 40 UEs) and circa 24% (for 58 UEs), respectively.