QoS Modeling for End-to-End Performance Evaluation over Networks with Wireless Access
© Gerardo Gómez et al. 2010
Received: 31 July 2009
Accepted: 31 January 2010
Published: 22 March 2010
This paper presents an end-to-end Quality of Service (QoS) model for assessing the performance of data services over networks with wireless access. The proposed model deals with performance degradation across protocol layers using a bottom-up strategy, starting with the physical layer and moving on up to the application layer. This approach makes it possible to analytically assess performance at different layers, thereby facilitating a possible end-to-end optimization process. As a representative case, a scenario where a set of mobile terminals connected to a streaming server through an IP access node has been studied. UDP, TCP, and the new TCP-Friendly Rate Control (TFRC) protocols were analyzed at the transport layer. The radio interface consisted of a variable-rate multiuser and multichannel subsystem, including retransmissions and adaptive modulation and coding. The proposed analytical QoS model was validated on a real-time emulator of an end-to-end network with wireless access and proved to be very useful for the purposes of service performance estimation and optimization.
Quality of Service (QoS) over networks with wireless access is a common research topic and is often studied in relation to end-to-end QoS or cross-layer architectures. Most authors focus on particular network elements or domains (e.g., terminals, radio interfaces, or core networks) or on specific protocol layers, such as congestion control schemes for wireless multimedia at the transport layer (TCP-friendly)  or QoS-scheduling techniques at the radio interface .
However, the QoS perceived by end users is an end-to-end issue and is therefore affected by every part of the network, the protocol layers, and the way they all interact. Moreover, seamless connectivity requires wireless and wired networks to operate in a coordinated manner in order to support packet data services with different QoS requirements. In such scenarios, data service performance assessment is usually addressed through active terminal monitoring over real networks . However, such a method proves to be costly if the operator wants to collect statistics from a reasonable number of terminals, applications, and locations. It may also prove to be a highly time-consuming process due to the variety of potential scenarios, both in terms of the type of service being offered and their spatial location.
Only a small number of works in the literature describe a general framework for end-to-end QoS control. One such end-to-end QoS framework for streaming services in 3G mobile networks is considered in , analyzing the interaction between UMTS and IETF's protocols and mechanisms. In , several key elements in the end-to-end QoS support for video delivery are addressed, including network QoS provisioning and scalable video representation. A small number of works have begun to include proposals involving end-to-end QoS management over wireless networks. In , a theoretical model for integrated cross-layer control and optimization in wireless multimedia communications is introduced. The work presented in  proposes an adaptive protocol suite for optimizing service performance over wireless networks, including rate adaptation, congestion control, mobility support, and coding. An overview of the current cross-layer solutions for QoS support in multihop wireless networks including cooperative communication and networking or opportunistic transmission can be found in . However, none of the previous works presents a method or tool for assessing and/or optimizing end-to-end QoS in a simple manner.
In this paper, the problem of providing accurate end-to-end performance estimations over networks with wireless access is addressed through a QoS model. The quality of packet data services is analyzed by calculating the performance degradation that occurs at each protocol layer. The overall degradation is analyzed starting from the physical layer up to the application layer. The performance assessment model described herein can be used to estimate the end-to-end performance of services in this type of networks before deployment. In addition, the proposed model is a useful tool for achieving end-to-end optimization, as it helps to find an appropriate configuration for each layer, thereby optimizing the end-to-end performance.
The proposed model was validated using a set of mobile terminals which were connected to a streaming server through an IP network with wireless access. We paid special attention to the impact of different radio interface mechanisms and transport layer protocols on streaming service performance.
The remainder of this paper is organized as follows. The general system model for multimedia streaming services over the wired-wireless network is outlined in Section 2. The QoS modeling process of the streaming protocol stack is presented in Section 3. Section 4 presents the end-to-end model results, whereas their validation results from a real-time emulator are shown in Section 5. Section 6 discusses the applicability of the proposed architecture for assessing the Quality of Experience (QoE) for data service users. Finally, Section 7 states the main conclusions of this work.
2. System Model
(L1)A variable-rate multiuser and multichannel subsystem is considered for the radio interface. Channel multiplexing is performed at the PHYsical (PHY) layer, where the radio channel is divided into resources independently allocated to users. Also, the PHY layer performs adaptive modulation and channel coding .
(L2)The link layer is responsible for performing user multiplexing; that is, resources are temporarily assigned to users following a specific scheduling algorithm. Moreover, selective retransmissions of erroneous (if so configured) and the compression of upper layer headers are also performed at this layer. Traffic shaping is performed at the upper interface of the network side L2; when the network load is high, data may be lost due to overflow in the queue.
(L3)An IP-based radio access node is considered at the network layer (L3), through which mobile terminals connect to the streaming server.
(L4)At the transport layer (L4), several options were analyzed at the user plane (UDP, TCP, and TFRC ).
(L5)At the user plane, the Real-time Transport Protocol (RTP) carries delay-sensitive data while the Real-time Transport Control Protocol (RTCP) conveys information on the participants and monitors the quality of the RTP session. Performance analysis of streaming signaling protocols during session setup is out of the scope of this paper; however, further details can be found in [1, 5].
In this work, throughput, delay, and loss rate indicators at each layer are modeled analytically, except the delay associated to scheduling algorithms at the radio and IP domains, which is still an open issue and has been obtained from simulations.
For the traffic model, variable rate information sources are considered at the application layer. A sufficiently large application buffer is assumed; thus network jitter is compensated at this layer. A summary of the numerical parameters used in this work at all layers is given in Table 12 at the end of the paper.
3. Protocol Layer Modeling
3.1. Physical Layer Model
Adaptive modulation is used to follow the fading behavior of the channels represented by its instantaneous Signal to Noise Ratio (SNR); such behavior is different for each user and PRB . Let be a matrix representing the received instantaneous SNR for user and channel at frame , and let be the number of bits/symbol of a QAM constellation that should ideally fulfill a certain target bit error rate . Channel coding (with coding rate ) is used to obtain a certain coding gain that generally ranges from 2 to 10 dB.
The same constellation is used for all QAM symbols within a PRB, making it possible to transmit a total of bits. The term can be seen as the potential rate (in bits/frame) of channel k if it is assigned to user i (see MAC layer model in the following section). The actual rate of a channel will be , where represents the user who is actually allocated to channel k.
Regarding the radio channel's behavior represented by the random process , its temporal variation (fading) is assumed to follow the usual Jakes' model ; an exponential decay model with factor is assumed for correlation between channels ; independence is assumed between users .
Parameters associated to the physical layer model.
Target Bit Error Rate (BER)
Constellation set (modulation levels)
Transmission Time Interval
Number of channels
Correlation between consecutive channels
Number of QAM symbols multiplexed on a channel
Number of QAM symbols per TTI
Channel Coding Gain
Average Signal to Noise Ratio (SNR)
3.2. Link Layer Model
The link layer includes the Medium Access Control (MAC), Radio Link Control (RLC), and Packet Data Convergence Protocol (PDCP) sublayers (as shown in Figure 1).
3.2.1. MAC Layer Model
Two scheduling algorithms were assessed: Round Robin (RR) and Modified Largest Weighted Delay First (M-LWDF) . RR is fair among users, although it fails to achieve any multiuser or multichannel diversity gain. On the other hand, M-LWDF considers both channel quality and QoS indicators in its scheduling criteria by allocating the resources to the user with the highest potential rate and delay product. According to , the M-LWDF algorithm is throughput optimal; that is, it gets the maximum possible diversity gain for stable queues. Other scheduling algorithms such as Best Channel (BC) or Proportional Fair (PF) algorithms achieve better throughput for some users, but this comes at the expense of others, who experience throughput starvation . As mentioned earlier, the delay associated to the scheduling process was obtained from simulations.
The error rate at the MAC layer ( ) depends on the BER achieved at the physical layer ( ) and the size of an -PDU ( ). In order to provide an expression for the Block Error Rate (BLER) at the MAC layer, , instantaneous BER is assumed to be equally distributed along bits, which is reasonably true if proper interleaving is performed.
3.2.2. RLC Layer Model
While some streaming applications are error-tolerant, others may require reliable data delivery. In this case, the network can optionally retransmit erroneous -PDUs (i.e., RLC blocks). Thus, the error rate can be lowered at the expense of decreasing throughput and increasing mean delay and jitter.
The retransmission mechanism analyzed in this paper considers a generic link layer retransmission scheme (based on the ARQ protocol) . ARQ protocol behavior is described as follows. Incoming upper layer PDUs are segmented into - PDUs and buffered. The transmitter sends all -PDUs and polls the receiver in the last -PDU of a higher layer PDU ( -PDU). A status report request is issued if no response is received to the polling upon expiration of . Selective acknowledgement is used to report which -PDUs have been incorrectly received. Nonacknowledged -PDUs are retransmitted if the maximum number of retransmissions has not been reached; we call cycle the th (re)transmission attempt. Further details can be found in .
where represents the required mean number of retransmission cycles to send one -PDU; and represent the mean size of a data and control -PDU, respectively; represents the number of -PDUs per -PDU (including retransmissions).
where the terms and can be computed as a function of and , whose details can be found in .
3.2.3. PDCP Layer Model
The PDCP layer is in charge of adapting the data to achieve efficient transport through the radio interface. This layer performs header compression, which reduces network and transport headers (e.g., TCP/IP or RTP/UDP/IP). The most advanced header compression technique is known as RObust Header Compression (ROHC) , which has been adopted by cellular standardization bodies such as 3 GPP. Using ROHC, the RTP/UDP/IPv4 header is compressed from 40 bytes to approximately 1 to 4 bytes, providing a compression gain .
where is the probability of requiring i (re)transmission cycles, as defined in (13), whereas where the terms and can be computed as a function of , and , whose details can be found in .
3.3. Network Layer Model
The network layer is based on an end-to-end IP connection from the mobile terminal to the streaming server. IP links are assumed to be over-dimensioned compared to radio links. The well-known Weighted Fair Queuing (WFQ) multiplexing algorithm was assessed in the IP routers by means of simulations.
3.4. Transport Layer Model
This section aims to model the performance of three different transport protocols (UDP, TCP, and TFRC) based on performance indicators of the lower layers.
3.4.1. UDP Model
3.4.2. TCP Model
TCP includes a congestion control mechanism to react against network congestion. When TCP is used as transport protocol, application throughput behavior depends on the specific TCP implementation. An analytic characterization of the steady-state throughput for TCP-Reno protocol has been applied in this work. This model characterizes TCP throughput as a function of loss rate in the network , Round-Trip-Time (RTT), Retransmission Time-Out duration ( ), maximum TCP window size (W) for a bulk transfer TCP flow, and the number of packets (b) acknowledged by each received ACK. The complete characterization of the TCP source rate, assuming that the maximum TCP window size has been reached, is computed in .
TCP performance is highly sensitive to packet losses because of its inherent congestion control mechanism, which decreases the window transmission, even if such losses are not due to congestion. Besides, the higher the RTT, the lower the throughput at the transport layer, because the congestion window is increased at a rate of RTT.
3.4.3. TFRC Model
Since TFRC only includes the congestion control mechanism (and not retransmissions), losses remaining at the transport layer come from noncorrected errors at the radio link, and TFRC delay is similar to the network delay ( ).
3.5. Application Layer Model
The application layer is responsible for establishing the streaming session, and thereafter, for transferring the multimedia content (at the server side) and reproducing the content (at the client side).
On the receiver side, the application layer adds an additional delay because of the application buffer of the streaming player. A sufficiently large application buffer size that hides network jitter to application performance has been assumed. Then, considering that the application throughput is not interrupted by buffer starvation, the following expressions can be obtained.
Parameters associated to the application layer model.
From the end user perspective, the delay introduced by the application buffer, , can be considered as part of the session establishment, since the application does not start reproducing media until the buffer is full. The buffer usually spans from 1 to 10 (depending on the technology). However, in two-way streaming services (like Push-to-Talk over Cellular, PoC) the lower limit is generally small (not higher than 500 ms) since the interactivity requirements are much stricter than they are in one-way streaming services.
4.1. Performance Estimation
( ) MAC layer throughput, , is rapidly degraded above a certain critical load point, which corresponds to the maximum achievable system throughput for a particular multiplexing algorithm. As expected, the M-LWDF algorithm achieves a higher system throughput (about 12 Mbps with scenario settings) than RR, since M-LWDF takes Channel State Information (CSI) into account, thus providing a higher diversity gain .
( ) The use of ROHC makes it possible to decrease the required amount of resources below the PDCP layer while achieving the same application level throughput. Specifically, ROHC achieves a capacity gain of 7% in our scenario. Due to compression, the PDCP layer may even compute a higher throughput (after decompression) than the lower layers, as illustrated in Figures 4(b) and 4(d).
Throughput curves in Figure 4 also provide very valuable information about the required resources at each layer in order to fulfill the desired QoS at the application level. For instance, the proposed model is able to map application level QoS requirements onto lower layer requirements; for example, a 384 kbps coding rate requires performing a resource reservation of 400 kbps at the IP level or assigning 450 kbps at MAC layer scheduling.
4.2. End-to-End Design
In this section, an end-to-end design example for TCP-based applications is described. The analysis is focused on those parameters having a higher influence on the overall performance: TCP window size (W), maximum number of RLC retransmissions ( ), and number of users in the system ( ). The following parameter values were used: packets and .
In terms of TCP throughput results, which are depicted in Figures 5(a) and 5(b), it is shown how high values require a higher number of RLC retransmissions to minimize data losses, and consequently, maximize throughput. For low load conditions ( users), potential TCP throughput is higher than the video codec rate (384 kbps) as long as a proper value is configured. However, for high load conditions ( users), TCP is not able to achieved the desired throughput.
Low load ( ): in general, high loss rates at MAC sublayer ( ) must be reduced by RLC retransmissions (configuring a high value of parameter). As the radio interface delay is very low in low load conditions, the impact of RLC retransmissions on TCP delay is almost negligible. Otherwise, if is set to a low value, TCP will be responsible for performing end-to-end retransmissions, thus increasing delay.
High load ( ): in addition to the previous effect, high load conditions increase the radio interface delay, and thus consecutive RLC retransmissions will increase the end-to-end RTT. As the TCP delay depends on the average RTT, a high will leads to high TCP delays. Besides, as the TCP throughput (per user) increases for high values, the overall load in the network is higher, thereby further increasing the TCP delay.
According to the results shown in Figure 5, for a given there is an optimum value that maximizes throughput while keeping delay as low as possible. This value depends on the loss rate in the network. For instance, for (obtained from a at the physical layer), the optimum value of is 6.
In sum, the values of , and W parameters must be jointly decided upon, making trade-offs between throughput and delay. For a given , a trade-off value for was 6 in order to limit the end-to-end delay. For these values of and , the maximum TCP window that maximizes throughput was W= 32 kB.
5. Model Validation
The objective of this section is to validate the theoretical model proposed in this work. The validation process is divided in two phases: ( ) validation of the radio interface model, and ( ) validation of the upper layer model.
5.1. Radio Interface Model Validation
Since the radio technology under study is not yet available, the validation process of the radio subsystem is based on link level simulations. Such simulations have been performed for a frequency-selective Rayleigh fading channel using adaptive modulation with a . The feedback channel is assumed to be ideal (with no delay or losses).
5.2. Upper Layer Model Validation
(i) Streaming Server.
Darwin Streaming Server v5.5.5 was used on the server side. This server allows one to select UDP or TCP as the transport protocol. Streaming content is based on a single video flow whose parameters are listed in Table 10. A packet sniffer (Wireshark v0.99.7) is used on both sides (server and client) to capture and analyze the traffic between peers.
(ii) Real-Time Emulator.
Between the server and the client, a real-time emulator models the behavior of the whole network, so that the client-server connection experiences (in real-time) the quality degradation introduced over the end-to-end path. This emulator uses the packet filtering framework included in the Linux 2.4.x and 2.6.x kernel series together with the iptables utility: iptables allows one to configure the packet filtering rule set. Certain quality degradation (in terms of delay or packet loss) is applied to the filtered packets. Such degradation is set according to the quality indicators obtained at the IP layer: loss rate P L3 and delay D L3 . In this way, the emulator offers a real-time data flow that experiences the degradation introduced by the network with wireless access.
(iii) Streaming Client.
A VLC Media Player 0.8.6d is responsible for establishing the streaming session with the server and reproducing content. For the TCP-based solution, TweakMaster v2.50 was also used to align TCP settings on the client side with the parameters assumed in the theoretical model.
It is shown that UDP delivers data to the network at a source rate determined by the encoding process, independently of the network status (loss rate and delay). Average UDP source rate can be computed from the average application source rate ( kbps) and taking into account UDP headers, yielding kbps. On the other hand, the TCP source rate at the server is highly influenced by network conditions as a consequence of the TCP congestion control mechanism, which tries to react against congestion. This mechanism leads to an important reduction in the average TCP source rate ( ), as the network load increases.
Delay validation results are shown at the transport and application layers. TCP delays were measured by tracing the received ACKs from the terminal (using Wireshark and tcptrace software), taking into account that RTT. Validation of RTP delay is more complex, as there is no feedback information from the receiver to measure the RTP RTT. The solution involves using an RTCP time stamp to measure the delay from sender to receiver; this solution requires the sender and receiver to be synchronized via Network Time Protocol (NTP).
6. Use of the Model for QoE Assessment
The proposed end-to-end emulator delivers a detailed real-time analysis and understanding of the service quality for any application and technology by applying a proper configuration. This approach provides a simple mapping from network-level performance indicators to service-level performance indicators.
From a mobile operator's point of view, knowing how subscribers perceive the performance of the services they are offered is a key issue. Quality of Experience (QoE) is the term used to describe this end user perception.
As the complexity of the lower layers in the end-to-end connection is simplified by means of network performance indicators from the QoS model, our proposed emulator is able to run in real-time. This real-time emulator provides certain quality degradation (in terms of delay or packet loss) to the filtered packets, offering a data flow experiencing the degradation that a real end-to-end network would add. In this manner, the user QoE can be assessed for different network types, configurations, and topologies.
where MaxErr represents the maximum possible absolute value of colour components difference, w is the video width, and h is the video height.
PSNR evaluation of video quality.
Numerical Parameters at different layers.
2 (TCP), 1 (TFRC)
8 bytes (UDP), 20 bytes (TCP), 16 bytes (TFRC)
3 (UDP), 6 (TCP & TFRC)
0, 2 (QPSK), 4 (16QAM), 6 (64QAM)
In this work, a detailed analysis of the end-to-end QoS assessment over networks with wireless access has been presented. This paper proposes a new modeling methodology based on QoS models for each protocol layer, providing a set of performance indicators across the protocol stack.
Based on this methodology, a QoS model for streaming services has been developed. This model can be used to estimate the performance at any protocol layer. In addition, the model makes it possible to identify the main factors affecting the quality of service, which is very useful for end-to-end parameter optimization. Finally, the model can also be used to map QoS needs at different layers from application requirements (e.g., to reserve appropriate resources at each layer). The framework applied in this work for streaming can be extended to other services (e.g., VoIP) and radio technologies (e.g., WiMax).
In terms of performance results, it was shown that multiplexing algorithms which take into account both channel state information and QoS indicators (such as M-LWDF) provide the best performance (in terms of capacity and fairness). The values of BER T , and W parameters must be jointly decided upon, making trade-offs between throughput and delay; for example, for a given of , the maximum number of RLC retransmissions should be set to 6 in order to limit the end-to-end delay. With these values, the maximum TCP window that maximizes throughput is W = 32 kB.
In order to validate the proposed QoS model, a real-time emulation platform was developed. Additionally, this emulator makes it possible to experience the end-to-end quality of service and facilitates QoE assessment using appropriate measurement tools.
This work is partially supported by the Spanish Government under project TEC-2007-67289 and by the Junta de Andalucia under Proyecto de Excelencia P07-TIC-03226.
- Floyd S, Handley M, Padhye J, Widmer J: TCP friendly rate control (TFRC): protocol specification. RFC 3448, January 2003Google Scholar
- M. Andrews H, K. Kumaran G, K. Ramanan R, Stolyar A, Vijayakumar R, Whiting P: CDMA data QoS scheduling on the forward link with variable channel conditions. Bell Labs Technical Memorandum 2002.Google Scholar
- Gómez G, Sanchez R: End-to-end quality of service over cellular networks: data services performance optimization in 2G/3G. John Wiley & Sons, New York, NY, USA; 2005.View ArticleGoogle Scholar
- Montes H, Gómez G, Cuny R, Paris JF: Deployment of IP multimedia streaming services in third-generation mobile networks. IEEE Wireless Communications 2002, 9(5):84-92. 10.1109/MWC.2002.1043858View ArticleGoogle Scholar
- Zhang Q, Zhu W, Zhang Y-Q: End-to-end QoS for video delivery over wireless Internet. Proceedings of the IEEE 2005, 93(1):123-133.View ArticleGoogle Scholar
- Ci S, Wang H, Wu D: A theoretical framework for quality-aware cross-layer optimized wireless multimedia communications. Advances in Multimedia 2008, 2008:-10.Google Scholar
- Akyilduz I, Altunbasak Y, Fekri F, Sivakumar R: AdaptNet: an adaptive protocol suite for the next-generation wireless internet. IEEE Communications Magazine 2004, 42(3):128-136.View ArticleGoogle Scholar
- Zhang Q, Zheng Y-Q: Cross-layer design for qos support in multihop wireless networks. Proceedings of the IEEE 2008, 96(1):64-76.View ArticleGoogle Scholar
- Goldsmith AJ: Wireless Communications. Cambridge University Press, Cambridge, UK; 2005.View ArticleGoogle Scholar
- Gómez G, Morales-Jiménez D, López-Martínez FJ, Sánchez JJ, Entrambasaguas JT: Radio-interface physical layer. In Long Term Evolution: 3GPP LTE Radio and Cellular Technology. CRC Press, Boca Raton, Fla, USA; 2009.Google Scholar
- 3GPP 36.201 : Long Term Evolution (LTE) physical layer; general description. V8.3.0, March 2009Google Scholar
- Entrambasaguas JT, Aguayo-Torres MC, Gómez G, Paris JF: Multiuser capacity and fairness evaluation of channel/QoS-aware multiplexing algorithms. IEEE Network 2007, 21(3):24-30.View ArticleGoogle Scholar
- Peisa J, Meyer M: Analytical model for TCP file transfers over UMTS. Proceedings of International Conference on Third Generation Wireless and Beyond, June 2001, San Francisco, Calif, USA 42-47.Google Scholar
- Gómez G: QoS modeling for end-to-end streaming performance evaluation over wireless access networks, Ph.D. thesis. Departamento de Ingeniería de Comunicaciones, Universidad de Málaga, Malaga, Spain; 2009.Google Scholar
- Bormann C, Burmeister C, Degermark M, et al.: Robust Header Compression (ROHC). RFC 3095, July 2001Google Scholar
- Padhye J, Firoiu V, Towsley D, Kurose J: Modeling TCP throughput: a simple model and its empirical validation. Computer Communication Review 1998, 28(4):303-314. 10.1145/285243.285291View ArticleGoogle Scholar
- Xu L, Helzer J: Media streaming via TFRC: an analytical study of the impact of TFRC on user-perceived media quality. Proceedings of the 25th IEEE International Conference on Computer Communications (INFOCOM '06), April 2006, Barcelona, SpainGoogle Scholar
- Gómez G, Poncela-González J, Aguayo-Torres MC, Entrambasaguas JT: A real-time end-to-end testbed for evaluating the performance of multimedia services. Proceedings og the 2nd International Workshop on Future Multimedia Networking (FMN '09), 2009, Lecture Notes in Computer Science 5630: 212-217.View ArticleGoogle Scholar
- Jacobson V, Braden R, Borman D: TCP Extensions for High Performance. RFC 1323, May 1992Google Scholar
- Huynh-Thu Q, Ghanbari M: Scope of validity of PSNR in image/video quality assessment. Electronics Letters 2008, 44(13):800-801. 10.1049/el:20080522View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.