On energy-efficient TCP traffic over wireless cooperative relaying networks

Abstract

Cooperative relay is a promising technology to provide better end-to-end performance for wireless communication through spatial diversity. Most of previous study on relay networks concentrates on lower layer performance. However, how to design the relay network for standard transport control protocol (TCP), which is very important for application layer service, is rarely investigated. We focus on energy efficiency of TCP traffic which is determined by relay selection and transmission parameters in the lower layer, e.g., the length of frame, the modulation and the maximum retransmission time. The relay selection in the cooperative network is formulated as a restless bandits with switching cost (RBSC) system, in which the problem of optimal selection of cooperative relay is solved by using a linear programming relaxation and primal-dual index based heuristic algorithm. By this means, the relay selection is designed with "one step looka-head" policy so that the energy efficiency performance can be achieved in a time horizon. The simulations verify that the lower layer parameters affect the energy efficiency of TCP traffic in wireless cooperative networks environment, and demonstrate that the proposed relay selection scheme outperforms the memoryless schemes, e.g., greedy scheme that takes no consideration of the statistical characteristics of the channels.

1 Introduction

Multi-path fading is one major impediment in designing of wireless networks. A lot of frequency, time, and spatial diversity techniques can be used to alleviate the fading in wireless environment. Cooperative relaying technique has gained much attention due to its ability to mitigate the fading through spatial diversity, hence it could provide better end-to-end performance [1, 2]. In cooperative relaying networks, a number of relay nodes are assigned to use their antennas to help the source forward data to the destination via constructing a virtual antenna array. Typical cooperative relay communication consists of two phases: (1) the source broadcasts data to relay(s) and destination, (2) relay(s) transmit data received before to the destination using simply amplify forward (AF) or decode forward (DF) schemes. The destination combines the data received from independently channels, by which the spatial diversity gain can be obtained, the system capacity can be expanded, and the networks quality and system performance can be improved.

Because the cooperation gain acquired by destination is tightly related to the channel state information (CSI) between the source to relay, source to destination ad relay to destination, the relay selection is regarded as the key to achieve optimal performance, e.g., throughput, and outage probability, etc., [3, 4]. Moreover, the energy-efficient transmission in relay system has been paid more attention on, since most nodes in cooperative relay net-work scenarios are mobile devices which are battery-powered, and the energy-efficiency is crucial to the life time of the network and the user perceived quality. Bletsas et al. [5] devise a novel scheme that selects the best relay between source and destination based on instantaneous channel measurements to reduce the outage probability. However, the analysis of upper layer protocol's performance is not included. Madan [6] analyze the energy efficiency issue subject to the overall outage constraint in physical layer.

On the other hand, cross-layer design philosophy encourages us to explore the performance optimization for upper layers, such as transportation control protocol (TCP). TCP is dominant in most internet applications, e.g., the interactive session like Telnet and HTTP, and bulk data transfer such as FTP, and some of streaming applications. Initially, TCP is designed for wired networks where it has a lower bit error rate. However, in the wireless environment, TCP may suffer from severe performance degradation due to the unstable wire-less channel which has been depicted in [7, 8]. In addition, the performance of TCP directly affects the users' perceived quality of service (QoS). Since TCP is so important for upper layer service provisioning, the wireless networks, including cooperative relay network, should be designed or optimized for standard TCP, rather than modifying standard TCP to adapt to wireless networks [9]. Some efforts are carried out while focusing on TCP throughput optimization from cross-layer perspective, such as [10, 11]. Wei et al. [12] analyze the distributed relay selection problem considering the CSI, the residue energy of relay nodes and the statistical parameters of each channel. However, most of previous study concentrates on physical layer and link layer. Even [13] investigates cross-layer design for TCP over relay networks, the optimized design for optimal energy-efficient performance in TCP traffic in relay networks has not been well studied in above works.

In this article, we analyze the energy efficiency of TCP traffic over relay network. The channel fading states of differently selected relay and lower layer control parameters, e.g., the modulation, coding and frame size, are proved to be the determinants of the performance of the energy efficiency of the upper layer TCP. The wireless cooperative network with reley selection decision problem is formulated as a restless bandits with switching cost (RBSC) system [14]. Based on the instantaneous channel measurement and the experiential channel state transition probability anticipation, the relay selection problem is solved by using the primal-dual index heuristic algorithm with switching costs of relay reselection being considered. Finally, the energy-efficient performance for TCP traffic could be achieved in time horizon while using a "one step lookahead" policy.

The rest of this article is organized as follows. The description of system model and the energy efficiency issue of TCP traffic in wireless cooperative networks are analyzed in Section 2. In Section 3, the relaying network is formulated as a RBSC system and the relay selection problem is investigated. Simulations with their analysis are shown in Section 4. Finally, we conclude the article in Section 5.

2 System model and performance description

2.1 System model

First, we consider a wireless network scenario where the source node S, the destination D and a set of N relay nodes, $\mathcal{N}=\left\{1,\dots ,N\right\}$ coexist. The cooperative network is depicted in Figure 1. The source node S is expected to transmit bulk TCP data to destination via one relay R that is selected out from N relay nodes. The single relay Decode and Forward (DF) [15] is considered as relaying transmission strategy.

Figure 1

The wireless cooperative network model.

Upon the relay network, the TCP transmission is carried out based on IP services to provide the application layer protocols, e.g., Telnet, HTTP, and FTP, with an end-to-end and connection-oriented packet transport mechanism that ensures the reliable and ordered delivery of data. In this article, considering the standard TCP protocol, we choose TCP Reno [16] as reference, which is widely used for TCP implementation. Moreover, we assume that the bulk data service is carried in the TCP transmission, e.g., a big le over FTP, and there are several TCP services are concurrently deployed upon the relay networks. Let L_seg represents the length of each TCP segment (in bits). The segment is input into link layer and encapsulated as N _fr frames with length L _fr . The frame is composed of the payload L _frp and frame header L _frh . Obviously, L _frp equals to L_seg/N _fr .

In the link layer, the automatic repeat request (ARQ) mechanism is used to guarantee certain level of reliability in current layer. An acknowledgment (ACK) frame is fed back if the destination received correctly. Otherwise, a NACK frame is fed back to the source node to indicate the frame is not received correctly. A frame is transmitted repeatedly for N_arq times at the most if it is not received by the destination successfully. The whole TCP segment is correctly received if all the frames belonging to it are successfully transmitted.

For the MAC implementation for relay network, CoopMAC [17] is used as reference protocol for future expansion. According to CoopMAC, the source node S determines whether the cooperation is needed. If not, the source node transmits data directly using the traditional MAC mechanism, e.g., 802.11. If the cooperation is needed, the source node selects certain node out of relay or helper candidates, which will be discussed later in Section 3. If the relay R is selected, the CoopMAC RTS-CTS mode interaction is then used, which is followed by two phases relay transmission of data frame [17]. In this article, we consider that each relay is of different relay state which is characterized by the three models of Rayleigh fading channels, the source to the destination, the source to the nth relay and the nth relay to destination, and they are denoted as h _SD , h _SRn , and h _RnD . The state of each channel is independent distributed. And for Rayleigh fading channel, the received SNR from node i to node j, which is denoted as γ _ij = γ _i |h _ij |², is exponentially distributed with parameter $1/{\bar{\gamma }}_{ij}.{\gamma }_i$ is the transmit SNR at node i. Each receiver side in the network can have exact CSI, including SNR. Moreover, we assume that all the rayleigh fading channels can be modeled as finite-state Markov channel (FSMC) according to the received SNR [18]. The initial state and the transition probability can be calculated by the network beforehand, and can be disseminated to all the nodes. To simply the analysis, the initial states and transition probabilities are assumed to be consistent for a period, e.g., the whole duration of data transmission.

The transmission of bulk TCP data is designed to be processed across T time slots that $t\in \mathcal{J}=\left\{0,\dots ,T-1\right\}$. We assume that the channels of all relays are block fading which means the states keep constant for a period of time and vary after the time. Based on different relay states, the source selects a specific relay and cooperates with it with elaborated setting of lower layer parameters in each time slot, by which the performance of TCP traffic can be optimized. For each time slot, we assume that only one relay node can be selected to provide cooperation for a source in the network. $\mathcal{A}\left(\mathsf{\text{t}}\right)=\left\{a_1\left(t\right),a_2\left(t\right),\dots ,a_n\left(t\right),\dots ,a_N\left(t\right)\right\}$ is used to describe the actions of all relay in time slot t, where a _n (t) ={0, 1} and $n\in \mathcal{N}$. a _n (t) = 1 means in time slot t the n th relay is selected and set ACTIVE, otherwise it is not selected and set NOT ACTIVE.

The energy-efficiency performance of TCP transmission is defined as the successfully transmitted TCP data per energy unit during a time period, e.g., a relay frame time. The object of our article is to achieve the maximum energy efficiency of TCP traffic by finding the appropriate helpers to provide cooperative communication across the time slots and controlling the lower layer parameters in each slot. In the next subsection, we will discuss the model on energy efficiency of TCP traffic over relay network if the bulk date is transmitted.

2.2 The model of energy-efficiency of TCP traffic

TCP provides end-to-end reliable transmission based on IP service. In our scenario, IP layer mechanism is not considered since the routing problem in cooperative relay network in not involved. As mentioned above, TCP throughput which is the successfully transmitted TCP data per unit time, and the steady-state performance of TCP [19] should be considered as the base of energy-efficiency of TCP traffic. We de ne the energy efficiency of TCP traffic using n th relay for cooperation in time slot t as follows:

$$\eta \left(n\right)=\frac{B\left(n\right)L_{\mathsf{\text{seg}}}{T}_{fr}}{E_f}.$$

(1)

where T _fr is the duration of one frame in time slot t, B(n) is the throughput of TCP traffic (in segments) when the n th relay is selected in time slot t. The state of n th relay in time slot t is denoted as x _n (t). E _f is the energy consumed for one frame transmission in time slot t.

For the standard TCP protocol, the throughput of TCP adapts to the network condition by the congestion avoidance mechanism [20]. Upon receiving the ACKs, the window enlarges to increase its sending rate. If the ACK is not received within a timeout interval, the retransmission of packet will be processed. In Reno, transmission includes two main phases: slow start and congestion avoidance. At the slow start phase, the TCP sender's congestion window size (cwnd) grows exponentially. When cwnd reaches the slow start threshold (ssthresh), the TCP source enters the congestion avoidance phase and the cwnd increases linearly by round trip times (RTTs). If packet is received incorrectly or lost, the sender will receive duplicate ACK. When the number of the duplicated ACK reaches a threshold, fast retransmit process is entered at the TCP sender. The sender retransmits the lost packet, reduces the cwnd to half and linearly increases cwnd as in the congestion avoidance phase, therefore, the transmission rate is reduced. The Reno sender retransmits at most one dropped packet per round-trip time. In summary, the throughput of TCP is tightly related to the cwnd, the RTT and segment loss event probability. Based on Padhye's model [19], the throughput (segments per second) of TCP Reno is described as follows:

$$B\left(W_m,RTT,P_{\mathsf{\text{seg}}}\right)\approx min\left(\frac{W_m}{RTT},\frac{1}{RTT\sqrt{\frac{2b{P}_{\mathsf{\text{seg}}}}{3}}+T_{0}min\left(1,3\sqrt{\frac{3b{P}_{\mathsf{\text{seg}}}}{8}}\right)P_{\mathsf{\text{seg}}}(1+32P_{\mathsf{\text{seg}}}^2}\right),$$

(2)

where W _m is the maximum cwnd, RTT is the round trip time and P_seq is the segment loss probability. b is the number of packets acknowledged by a received ACK segment. Most implementations send a cumulative ACK for two consecutive segments. T₀ is the initial timeout, i.e., when segment is lost the sender waits for T₀ before retransmitting the non-acknowledged segment.

Since TCP transmission, especially in wireless network, will suffer from frequent packets loss due to the unstable wireless channel, the TCP throughput is regarded to be related to lower layer mechanisms and parameters.

The segment loss probability P_seg is determined by the bit error rate (BER) caused by additive white Gaussian noise (AWGN) in the wireless environment. The closed-form expressions of BER for the three nodes decode-and-forward (DF) relaying system using binary phase shift keying (BPSK) modulation is presented as [21]:

$$\begin{array}{c}{P}_b=\frac{1}{4}\left(1-\sqrt{\frac{{\bar{\gamma }}_{S{R}_n}}{1+{\bar{\gamma }}_{S{R}_n}}}\right)\left(1-\sqrt{\frac{{\bar{\gamma }}_{SD}}{1+{\bar{\gamma }}_{SD}}}\right)\hfill \\ +\frac{1}{4}\frac{{\bar{\gamma }}_{SD}{\gamma }_{R_n}}{{\bar{\gamma }}_{SD}{\gamma }_{R_n}-{\bar{\gamma }}_{R_{n}D}{\gamma }_{R_s}}\left(1+\sqrt{\frac{{\bar{\gamma }}_{S{R}_n}}{1+{\bar{\gamma }}_{S{R}_n}}}\right)\left(1-\sqrt{\frac{{\bar{\gamma }}_{SD}}{1+{\bar{\gamma }}_{SD}}}\right)\hfill \\ +\frac{1}{4}\frac{{\bar{\gamma }}_{R_{n}D}{\gamma }_S}{{\bar{\gamma }}_{R_{n}D}{\gamma }_S-{\bar{\gamma }}_{SD}{\gamma }_{R_n}}\left(1+\sqrt{\frac{{\bar{\gamma }}_{R_{n}D}}{1+{\bar{\gamma }}_{R_{n}D}}}\right)\left(1-\sqrt{\frac{{\bar{\gamma }}_{SD}}{1+{\bar{\gamma }}_{SD}}}\right)\hfill \end{array}$$

(3)

The frame error rate is denoted as P _fr and it is expressed as:

$$P_{fr}=1-{\left(1-P_b\right)}^{L_{fr}}.$$

(4)

The result in (4) is applicable to other modulation and coding schemes, and the BER can be acquired according to [21]. Using ARQ, the frame error probability is P _fr ^Narq+1. One TCP segment is transmitted successfully if all frames are transmitted correctly. Hence the segment loss probability is

$$P_{\mathsf{\text{seg}}}=1-{\left(1-P_{fr}^{N_{arq}+1}\right)}^{N_{fr}}.$$

(5)

In the link layer, the carrier sense multiple access with collision avoidance (CSMA/CA) mechanism is used as the medium access protocol for distributed coordination function (DCF). This means each terminal can initiate a data transmission and the channel sensing before data transmission is crucial to avoid the collisions. As mentioned above, CoopMAC is used as a reference of the link layer protocol. During each time slot, the procedure of frame transmission is shown in Figure 2. According to CoopMAC, when the source decides to send the data using relay mode and he relay node is selected, it first senses the channel to make sure it is clear. The source listens to the channel for DCF interframe space (DIFS) before data transmission. A random backo is executed to avoid collision. Then the source sends Request-to-Send (RTS) and sets Network allocation Vector (NAV) in RTS frame to let all nodes receiving the control frame know the time for which the channel is going to be free. After receiving RTS from S, the selected relay or helper node sends Help-ready-To-Send (HTS) frame after a short interframe space (SIFS) to declare the conformation and preparation of the relay cooperation. The destination D will send Clear-To-Send (CTS) to complete the control interaction among S, R and D after it correctly receives the RTS and HTS. Then, the cooperative relay data transmission will begin with two phases mentioned above.

Figure 2

The frame transmission procedure of CoopMAC protocol.

At the link layer, the automatic repeat request (ARQ) mechanism is also employed to support certain level of reliability in lower layer transmission. An ACK frame is fed back if the destination received the data frame correctly. Otherwise, a NACK frame is fed back to the source node to indicate the frame is not received correctly. A frame can be transmitted repeatedly for at most N _arq times if it is not received by the destination successfully. When the retransmission time exceeds N _arq , the frame will be discarded.

Therefore, the RTT of TCP segment can be expressed as:

$$RTT=N_{fr}\left(N_{ar}+1\right)\left(T_{\mathsf{\text{overhead}}}+\frac{L_{fr}}{R_{S{R}_n}}+\frac{L_{fr}}{R_{R_{n}D}}\right),$$

(6)

where T_overhead = 5T_SIFS +T_DIFS +T_RTS +T_HTS +T_CTS is the overhead cost, which is the summation of interframe space, control frame and random backo time. ${R_S}_{R_n}$ and $R_{R_{n}D}$ are the transmit rate from source to the n th relay and the n th relay to destination respectively. N _ar is the average number of frame retransmission that equals to $\left(1-P_{fr}\right){\sum }_{i=1}^{N_{arg}}i{P}_{fr}^i.$

The energy consumed by TCP segments is determined by the consumption of each frame.

The total energy consumed during the frame transmission can be expressed as:

$$E_f=\left(P_{T_S}+2P_R\right)\frac{L_{fr}}{R_{S{R}_n}}+\left(P_{T_{R_n}}+P_R\right)\frac{L_{fr}}{R_{R_{n}D}},$$

(7)

where $P_{T_S}$ and $P_{T_{Rn}}$ stand for the transmit power of the source node and the nth relay node. P _R is the power consumption for receiving.

Therefore, based on Equation (2), (3), (5), (6), and (7), the equation(1) can be expressed as:

$$\begin{array}{cc}\hfill \eta \left(n\right)& =\frac{B\left\{W_{\mathsf{\text{m}}}\left(t\right),RTT\left[L_{fr}\left(t\right),R_{S{R}_n}\left(t\right),R_{R_{n}D}\left(t\right),N_{arq}\left(t\right)\right],P_{\mathsf{\text{seg}}}\left[x_n\left(t\right),L_{fr}\left(t\right),N_{arq}\left(t\right)\right]\right\}}{E_f\left[R_{S{R}_n}\left(t\right),R_{R_{n}D}\left(t\right),L_{fr}\left(t\right)\right]}\hfill \\ \times L_{\mathsf{\text{seg}}}\times T_{fr}\left[R_{S{R}_n}\left(t\right),R_{R_{n}D}\left(t\right),L_{fr}\left(t\right)\right]\hfill \end{array}$$

(8)

where $R_{S{R}_n}\left(t\right)$ denotes the transmit rate from source to relay and $R_{R_{n}D}\left(t\right)$ the rate from relay to destination, which is determined by the modulation and the bandwidth of relay network. We also assume $R_{S{R}_n}\left(t\right)$, $R_{R_{n}D}\left(t\right)$ to be constant for each time slot t in the article. In addition, we use binary phase shift keying (BPSK) modulation. T _fr (t) = T_overhead + L _fr (t)/R_SR(t)+L _fr (t)/R _RD (t).

We could see that the channel fading states which represent relay selection result in lower layer relay network contributes to the segment loss probability and finally the energy efficiency of TCP traffic in the wireless cooperative relay networks. In addition, the modulation, the bandwidth, the N _arq and the frame length L _fr also have impacts to the TCP energy-efficiency performances. To be more specific on relay cooperative network, we pay more attention to the factors related to relay selection mechanism with its decision algorithm, therefore, W_m, timeout T₀, b in formula (2), bandwidth and modulation (determining the $R_{S{R}_n}\left(t\right)$ and $R_{R_{n}D}\left(t\right)$) are considered to be static in this article. Equation (8) can be expressed as:

$$\eta \left(n\left(t\right)\right)=\frac{B\left\{RTT\left[L_{fr}\left(t\right),N_{arq}\left(t\right)\right],P_{\mathsf{\text{seg}}}\left[x_n\left(t\right),L_{fr}\left(t\right),N_{arq}\left(t\right)\right]\right\}{L}_{\mathsf{\text{seg}}}{T}_{fr}\left[L_{fr}\left(t\right)\right]}{E_f\left[L_{fr}\left(t\right)\right]}.$$

(9)

3 Achieving energy-efficient TCP traffic

3.1 Optimal decision problem

Under the prescribed model, the optimal relay decision problem of a source node to transmit energy efficient TCP traffic is straightforward: it periodically updates its best choices of relay nodes for cooperative communication and sets the appropriate control parameters for link layer frame transmission. Equivalently, in each time slot t, S calculates the optimal η(n) for each relay candidate R _n with relay state x _n (t) by finding the optimal control parameter setting of L _fr (t) and N _arq (t), and then finds the best relay $R_n^{*}\left(t\right)$ with its control parameters being $L_{fr}^{*}\left(t\right)$ and and $N_{arq}^{*}\left(t\right)$.

$$\begin{array}{c}\left\{R_n^{*}\left(t\right),L_{fr}^{*}\left(t\right),N_{arq}^{*}\left(t\right)\right\}\hfill \\ =\underset{Rn\left(t\right)}{arg\; max}\left\{\frac{B\left(W_{\mathsf{\text{m}}},RTT\left[L_{fr}\left(t\right),N_{arq}\left(t\right)\right],P_{\mathsf{\text{seg}}}\left[x_n\left(t\right),L_{fr}\left(t\right),N_{arq}\left(t\right)\right]\right)L_{\mathsf{\text{seg}}}{T}_{fr}\left[L_{fr}\left(t\right)\right]}{E_f\left[L_{fr}\left(t\right)\right]}\right\}\hfill \end{array}$$

(10)

We may transform the original decision problem (10) into an optimal control problem based on the system view: source S decides $\left\{R_n^{*}\left(t\right),L_{fr}^{*}\left(t\right),N_{arq}^{*}\left(t\right)\right\}$ as an optimal control signal into system consisting of S, $\mathcal{N}$ and D, so that the control objective, i.e., energy efficiency for TCP relay transmission for S - D pair in the t th time slot, is maximized.

The decision in formula (10) is just made for one time slot. However, when the overall energy efficiency across the time horizon is considered, the greedy solution in which the decision is made just in one time slot scope may be unable to achieve the optimizing goal in a long run. The channel fading states of a relay n, x _n (t), may have high probability to change to another x _n (t+1) in the next time slot t+1, which may cause non-ideal performance in energy efficiency. By this way, the overall TCP traffic may have poor energy efficiency performance in a long run. Therefore, we prefer to making the decision with a "one-step lookahead" policy [14].

Based on the assumption that the rayleigh fading channels can be modeled using FSMC and the states of channels for a relay is independent distributed, the decision model for optimal control problem with "one-step lookahead" policy match the Restless Bandit system model in control theory [14]. In addition, we also consider the switching cost in relay selection procedure where the reselection and handover process may consume the energy while spending time for TCP data transmission. Therefore, we adopt Restless Bandit system with Switching Cost (RBSC) [14] based approach in this article.

3.2 Approach based on restless bandit system with switching cost

The RBSC approach is used to solve the stochastic control problems with the following characteristics: there are N projects and one is selected and set active at each time slot t = 0;1, . . . , T - 1. If the system switches from project i to j, it bring an additional costs denoted by c _ij . Each project is in a finite state space x _n (t)∈S _n , n = 1, . . . , N. We denote the Cartesian product of the state spaces as $\mathcal{J}=S_1\times \dots \times S_N$. For project n, a reward $r_n^1\left(xn\left(t\right)\right)$ is earned if it is selected, otherwise the reward is $r_n^0\left(xn\left(t\right)\right)$. Moreover, the reward is time-discounted by factor α∈(0,1). The projects' states change in a Markovian fashion according to the transition probability matrix. Project is selected under a policy u, which is in the set of all available policies $\mathcal{U}$. The purpose is to find the optimal u that maximize the expected discounted reward.

In following subsections, the relay network with relay selection decision will be formulated as RBSC system.

3.2.1 Relay state description and system reward

A FSMC is used to model the Rayleigh fading channel according to the received SNR [18]. The received SNR at j from i at time t is denoted as γ _ij (t) and divided into K discrete levels with increasing order. Each level is associated with a state of the Markov chain. Let ϒ = {ϒ₁, ϒ₂, . . . , ϒ _K } denotes the finite state space. The K × K transition probability matrix is

$$\Psi \left(t\right)=\left[{\psi }_{uv}\left(t\right)\right],$$

(11)

where ${\psi }_{uv}\left(t\right)=\mathsf{\text{Pr}}\left(\gamma \left(t+1\right)=v|\gamma \left(t\right)=u\right)$ and v, u ∈ϒ

In each time slot t ∈ {0, 1, . . . T-1},the state of the n th relay is characterized by the three Rayleigh fading channels, including the source to the n th relay and the destination, and the n th relay to the destination. We assume the state of each channel is independent distributed.

Therefore, the state of the n th relay is described as the three channels' state at time t:

$$x_n\left(t\right)=\left[{\gamma }_{SD}\left(t\right),{\gamma }_{S{R}_n}\left(t\right),{\gamma }_{R_{n}D}\left(t\right)\right].$$

(12)

The relay state changes in a Markovian fashion and the finite state space of the n th relay is denoted as S _n , that x _n (t)∈S _n . The transition probability matrix is

$$P_n\left(t\right)={\left[{\psi }_{f_n{j}_n}\left(t\right),{\psi }_{h_n{y}_n}\left(t\right),{\psi }_{u_n{v}_n}\left(t\right)\right]}_{H_n\times H_n},$$

(13)

where H_n = K × K × K and f _n , j _n , h _n , y _n , u _n , v _n ∈ ϒ. The matrixes can be obtained and updated according to the history observation, and in simulation, they keep constant.

The purpose of the proposed scheme is to maximize the energy efficiency described in equation (9) by selecting the optimal relay and adjusting the lower layer parameters according to the current relay state. The reward of the n th relay in each time slot t is

$$r_n^{a_n\left(t\right)}\left(x_n\left(t\right)\right)=a_n\left(t\right)\eta \left(x_n\left(t\right),L_{fr}\left(t\right),N_{arq}\left(t\right)\right).$$

(14)

where x _n (t) is the n th relay state at time t, $a_n\left(t\right)\in \mathcal{A}=\left\{0,1\right\}$, L _fr (t) is the frame size and N _arq (t) is the maximum retransmission time.

3.2.2 Solving the RBSC

The RBSC problem is determined by several factors: $\mathcal{N}$ is the set of available projects, S _n is the state space of the n th project, the Cartesian product of the state spaces is $\mathcal{J}=S_1\times \dots \times S_N,A_n$ is the action space, P _n is the transition probability, ν is the initial distribution, c _ij is denoted as the switching cost from i th relay to j th relay, α∈(0,1) is the discounted factor and r is the reward.

The complete state of the network is (x;s) = (x₁,. . . , x _N ;s), where vector × is the current states of N relay nodes, s denotes the current selected relay that $s\in \mathcal{N}$. To solve RBSC problem with linear programming, we denote the state-action pairs as $\mathcal{K}=\left\{\left(\mathsf{\text{x}};s\right),a:\mathsf{\text{x}}\in \mathcal{J},s\in \mathcal{N},a\in \mathcal{N}\right\}$. The occupation measures ρ_(x;s),aare the expected discounted time of different state-action pairs. The distribution of initial system state is described by ν, e.g., ν (x₁, . . . ,x _N ;s) = ν₁(x ₁ ) . . . ν _N (x _N ) δ₁ (s) if the 1 project is selected and set active initially. Let c _ij denote the switching cost from i to j. α∈(0,1) is the discounter factor. The RBSC problem can be cast into the MDP framework and the linear program can be written:

$$\begin{array}{c}\mathsf{\text{maximize}}\hfill \\ \sum _{s=1}^N\sum _{x\in \mathcal{J}}\sum _{a=1}^N\left(r_a^1\left(x_a\right)+\sum _{j\ne a}{r}_j^0\left(x_j\right)-c_{sa}\right){\rho }_{\left(x_1,...,x_N;s\right),a}\hfill \\ \mathsf{\text{subject to}}\hfill \\ \sum _{a=1}^N{\rho }_{\left(\mathsf{\text{x}};s\right),a}-\alpha \sum _{s^{\prime }=1}^N\sum _{x^{\prime }\in \mathcal{J}}{\rho }_{\left(\mathsf{\text{x'}};s^{\prime }\right),s}{p}_{x_1^{\prime }{x}_1}^0...p_{x_s^{\prime }{x}_s}^1...p_{x_N^{\prime }{x}_N}^0=\left(1-\alpha \right)v\left(\mathsf{\text{x}};s\right),\forall \left(\mathsf{\text{x}},s\right)\in \mathcal{J}\times \mathcal{N}\hfill \\ {\rho }_{\left(\mathsf{\text{x}};s\right),a}\ge 0,\forall \left(\left(\mathsf{\text{x}};s\right),a\right)\in \mathcal{J}\times {\mathcal{N}}^2,\hfill \end{array}$$

(15)

In the above formula, the decision variables ${\rho }_{\left(x_1,\dots ,x_N;s\right),a}$, which is exponential in size of the input data. Therefore, this problem can not be solved efficiently in the real world. By relaxing the constraint in (15), we get the final LP relaxation for RBSC:

$$\begin{array}{c}\mathsf{\text{maximize}}\hfill \\ \sum _{s=1}^N\sum _{a=1}^N\left[\sum _{x_a\in S_a}\left(r_a^1\left(x_a\right)-c_{sa}\right){\rho }_{\left(x_a;s\right)}^a+\sum _{j\ne a}\sum _{x_j\in S_j}{r}_j^0\left(x_j\right){\rho }_{\left(x_j;s\right),a}^j\right]\hfill \\ \mathsf{\text{subject to}}\hfill \\ {\rho }_{\left(x_i;s\right),a}^i\in Q_i^{\alpha }\left(v_i\right),\forall i\in \mathcal{N}\hfill \\ \sum _{x_i\in S_i}{\rho }_{\left(x_i;s\right),a}^i=\sum _{x_1\in S_1}{\rho }_{\left(x_1;s\right),a}^1,\forall i,s,a\in \mathcal{N}.\hfill \end{array}$$

(16)

where $Q_i^{\alpha }\left(v_i\right)$ is the polytope of the state-action pair variable defined as:

$$\begin{array}{c}{Q}_i^{\alpha }\left(v_i\right)=\left\{\begin{array}{c}{\rho }_{\left(x_i;s\right),a}^i\in {ℝ}_{+}^{\left|s_i\right|\times N^2}:\hfill \\ \sum _{a=1}^N{\rho }_{\left(x_i;i\right),a}^i-\alpha \sum _{x_i^{\prime }\in S_i}\sum _{s^{\prime }=1}^N{\rho }_{\left(x_i^{\prime };s\right),i}^i{p}_{x_i^{\prime }{x}_i}^1=\left(1-\alpha \right)v_i\left(x_i\right){\delta }_1\left(i\right)\forall x_i\in S_i,\hfill \\ \sum _{a=1}^N{\rho }_{\left(x_i;i\right),a}^i-\alpha \sum _{x_i^{\prime }\in S_i}\sum _{s^{\prime }=1}^N{\rho }_{\left(x_i^{\prime };s\right),i}^i{p}_{x_i^{\prime }{x}_i}^0=\left(1-\alpha \right)v_i\left(x_i\right){\delta }_1\left(s\right)\forall x_i\in S_i,s\ne i.\hfill \end{array}\right.\hfill \end{array}$$

(17)

For N projects, there are $\mathcal{O}\left(N^3\times {max}_i\left(\left|S_i\right|\right)\right)$ variables and constraints. So the variables in (16) is in polynomial size of input data. In addition, the dual of the above LP is:

$$\begin{array}{c}\mathsf{\text{minimize}}\hfill \\ \left(1-\alpha \right)\sum _{i=1}^N\sum _{s=1}^N\sum _{x_i\in S_i}{v}_i\left(x_i\right){\delta }_1\left(s\right){\lambda }_{x_i,s}^i\hfill \\ \mathsf{\text{subject to}}\hfill \\ {\lambda }_{x_1,s}^1-\alpha \sum _{x_1^{\prime }\in S_1}{p}_{x_1{x}_1^{\prime }}^1{\lambda }_{x_1^{\prime },1}^1+\sum _{i=2}^N{\mu }_{s,1}^i\ge r_1^1\left(x_1\right)-c_{s1},s\in \mathcal{N},x_1\in S_1\hfill \\ {\lambda }_{x_1,s}^1-\alpha \sum _{x_1^{\prime }\in S_1}{p}_{x_1{x}_1^{\prime }}^0{\lambda }_{x_1^{\prime },a}^1+\sum _{i=2}^N{\mu }_{s,a}^i\ge r_1^0\left(x_1\right),s\in \mathcal{N},x_1\in S_1,a\ne 1\hfill \\ {\lambda }_{x_j,s}^j-\alpha \sum _{x_j^{\prime }\in S_j}{p}_{x_j{x}_j^{\prime }}^1{\lambda }_{x_j^{\prime },j}^j-{\mu }_{s,j}^j\ge r_j^1\left(x_j\right)-c_{sj},j\ge 2,s\in \mathcal{N},x_j\in S_j\hfill \\ {\lambda }_{x_j,s}^j-\alpha \sum _{x_j^{\prime }\in S_j}{p}_{x_j{x}_j^{\prime }}^0{\lambda }_{x_j^{\prime },j}^j-{\mu }_{s,j}^j\ge r_j^0\left(x_j\right),j\ge 2,s\in \mathcal{N},x_j\in S_j,\alpha \ne j\mathsf{\text{.}}\hfill \end{array}$$

(18)

The author in [14] provided a primal-dual index heuristic approach to solve the selection problem. For (16) and (18), the solution is denoted as $\left\{{\bar{\rho }}_{\left(x_i;s\right),a}^{\iota }\right\}$ and $\left\{{\bar{\lambda }}_{x_i,s}^i,{\bar{\mu }}_{s,a}^i\right\}$. The reduced costs $\left\{{\bar{\xi }}_{\left(x_i;s\right),a}^i\right\}$ are the difference between the left and right of the constraints in (18). The reduced costs are nonnegative and ${\bar{\xi }}_{\left(x_i;s\right),a}^i$ equals to 0 if the corresponding ${\bar{\rho }}_{\left(x_i;s\right),a}^{\iota }>0$. It is also interpreted as the rate of decrease in the objective value of the primal linear program (16) per unit increase of variable ${\rho }_{\left(x_i;s\right),a}^i$. If the current state is (x₁, . . . , x _N ; s), the candidate index (CI) of action a is as:

$$I\left(\left(x_1,\dots ,x_N;s\right),a\right)=\sum _{i=1}^N{\bar{\xi }}_{\left(x_i;s\right),a}^i$$

(19)

which is the sum of all projectsŕeduced cost to action a. Therefore, the optimal action is to select the project with the minimal CI:

$$â=arg\underset{a\in \mathcal{N}}{min}\left\{I\left(\left(x_1,\dots ,x_N;s\right),a\right)\right\}.$$

(20)

We can store the $\mathcal{O}\left(N^3{max}_i\left(\left|S_i\right|\right)\right)$ optimal reduced costs and compute the indices described in (19) in time $\mathcal{O}\left(N^2\right),$ which is polynomial in the size of input data.

3.3 Relay selection procedure

Based on the approach depicted above, the relay decision algorithm is summarized as Algorithm 1.

For each node, a CoopTable similar to [17] is maintained. Based on the CoopTable, the source node can decide which relay candidate to cooperate for energy-efficient TCP transmission. The revised CoopTable consists of seven fields: the address of destination node (S), the address of potential relay for source node S (R _n ), the CSI (SNR) from S to D (h_SD), the CSI (SNR) from S to R _n $\left({h_{SR}}_n\right)$, the CSI (SNR) from R _n to D $\left({hRn}_D\right)$, the count of failure about this cooperative relay $\left(N{F_R}_n\right)$, and the update time. The initial distribution of state and the transition probability, which can be obtained from the network × node (long run node or base station) in advance, is also stored in each node.

The channel states information in CoopTable record can be updated through control frames and overheard transmissions: The channel between any two nodes is assumed to be symmetric, because they all use the same frequency band for transmission. When S receives HTS frame sent by R _n , it gets the estimated ${\gamma }_{S{R}_n}$ D receives HTS frame and estimates ${\gamma }_{R_{n}D}$ and piggyback it in its CTS frames. When D sends CTS frames, it encapsulates SNR from S to D (γ_SD) which is estimated at D when receiving RTS. After receiving CTS, S extracts the channel state information γ _SD and ${\gamma }_{S{R}_n}$. S and D can also overhear frame from other candidate relays to estimate the channel state information between S and R _x , R _x , and D, and D will also piggyback the ${\gamma }_{R_{x}D}$ in its CTS frames. By this way, each S can maintain the CSI for all relays based on which the optimized result for relay selection to achieve energy efficiency of TCP traffic in every time slot can be calculated.

Moreover, when S decision produces a new relay R _n other than currently selected relay R _s el, it invokes the relay selection procedure expanded based on CoopMAC, as shown in Figure 3. A new message IDLE is introduced to facilitate the re-selection. S sends a IDLE frame which inform the currently selected relay R_sel to suspend its cooperation and set the state from ACTIVE to NOT ACTIVE after sending back an ACK. IDLE frame also contains NAV to preserve the channel for selection query interactions. S sends RTS to request $R_{n1}^{*}$ to provide the cooperation while setting NAV in its frame. The target relay $R_{n1}^{*}$ may be available and willing to provide the cooperation so that it sends back the HTS. On the other hand, the target relay may also be reserved by other blind source or unwilling to provide the cooperation. In this case, it keeps silent and replies nothing. If no HTS received at S after SIFS interval, which means the newly selected relay is unavailable at this time, S begins another round of relay decision using Algorithm 1 excluding the unavailable relay. It sends RTS to the newly selected $R_{nj}^{*}$ and waits for HTS. The procedure iterates until one HTS is received. Finally, D sends CTS to complete the selection and control interactions. After that, the data transmission begins for t th time slot with the selected relay $R_{nj}^{*}$ with the length of link layer frame being $L_{fr}^j\left(t\right)$ and the maximum retransmission time being $N_{arq}^j\left(t\right)$. Obviously, if S does not decide to change relay in time slot t, the CoopMAC RTS-CTS mode as well as two phases of data relay transmission follows as regular design shown in Figure 2.

Figure 3

The relay selection procedure extended based on CoopMAC.

Algorithm 1 Select the optimal relay â

Input:

• S _n , $n\in \mathcal{N}${State Space}

• v(x₁, . . ., x _N ;s) {Distribution of Initial State}

• P _n (t) {State Transition Probability of Each Relay}

• c _ij {Switching Cost From Relay i to j}

Output:

• â {The Optimal Relay}

1: Obtain the current state of the system: (x;s).

2: Get the optimal solution of primal and dual problems:

$$\left\{{\widehat{\rho }}_{\left(x_i;s\right),a}^i\right\}\left\{{\widehat{\lambda }}_{x_i,s}^i,{\widehat{\mu }}_{s,a}^i\right\}.$$

3: Get the reduced cost:

$$\left\{{\widehat{\gamma }}_{\left(x_i;s\right),a}^i\right\}.$$

4: Calculate the CI of the N relays and select the relay with smallest CI:

$$â=arg\underset{a\in \mathcal{N}}{min}\left\{I\left(\left(x_1,\dots ,x_N;s\right),a\right)\right\}.$$

In the procedure mentioned above, the switching cost is incurred by the control message interactions for reselection, e.g., IDLE and ACK, RTS and its retransmission. To keep the measurement of switching cost consistent with the energy efficiency in Equation (1), we define the switching cost as the product of the average system reward $\bar{\eta }$ and the expected extra retransmission time of control frames, where $\bar{\eta }=\frac{{\Sigma }_{n=1}^N{\Sigma }_{x_n\in S_n}\eta \left(x_n\right)}{{\Sigma }_{n=1}^N\left|S_n\right|}$. The probability that the target relay node is idle can be denoted as P_idl, and the expected extra retransmission number is {1/P_idl - 1}. Therefore, the switching cost is $\bar{\eta }\left(1/P_{\mathsf{\text{id}}1}-1\right)$, i.e., if the target relay node is idle with probability 1 there is no switching cost.

4 Performance evaluation

In this section, we depict the in influence of the frame length and the maximum retransmission time N _arq to the energy efficiency of TCP traffic. Also, we present the simulations to evaluate the performance of the proposed relay scheduling scheme. Then, we illustrate the effect of switching cost to the relay scheduling process. Finally, we compare the pro-posed scheme with a random selection method and a greedy method which maximize the instantaneous system reward without considering the transition probability of relays' state, i.e., $a_{\mathsf{\text{greedy}}}=arg{max}_{a\in \mathcal{N}}\left\{r_a^1\left(x_a\right)-r_a^0\left(x_a\right)-c_{sa}\right\}$. The greedy policy is optimal in the case of multi-armed bandits (MAB) problem that the passive projects do not change their states, with no switching cost and deteriorating rewards.

We use the TCP Reno in which a large le is transmitted by le transfer protocol (FTP) to keep the data transmission buffer full. Consider there are 5 relays in the network, which could incur switching cost when choosing a different relay for the next time slot transmission, and the bandwidth of each channel is 1 MHz. The data rates ${R_S}_{R_n}$ and $R_{R_{n}D}$ are set 1Mbps as BPSK modulation mode is used. The length of segment L_seg is 1500 bytes, the timeout T₀ is 3 s and the maximum window size is 64 K bytes. In the data link layer, the length of frame header L_frh is 20 bytes. The discount factor is set to be 0.95. The RTS is 20 bytes, HTS is 14 bytes, CTS is 14 bytes, T_SIFS = 16 μs and T_DIFS = 35 μs.

For simplicity, we assume the AWGN has unit variance and the transmit power of source and relays is the same. The power consumption for transmitting is set to be 1400 mW, and the power consumption for receiving is 900 mW [22]. Rayleigh fading channel models is used to describe the channels between nodes. The received SNR from source to relays and relays to destination is categorized into three states: 15 dB, 20 dB, and 25 dB. Similarly, the received SNR at destination from sources is divided into three states: 14 dB, 16 dB, and 18 dB. We set the transition probability of each channel as:

$${\Psi }_n\left(t\right)=\left(\begin{array}{c}\hfill 0.60.30.1\hfill \\ \hfill 0.20.60.2\hfill \\ \hfill 0.10.30.6\hfill \end{array}\right).$$

Figures 4, 5, and 6 illustrate the segment loss probability with different frame size and different N _arq . First, when the frame size is short, i.e., N _fr is large, the segment loss probability gets small. Because if the frame size is short, the frame loss probability will decrease. Second, with different maximum frame retransmission time N _arq , the segment loss probability is not the same. As the N _arq gets larger, the segments loss probability will be reduced since the frame loss probability is in inverse proportion to the N _arq .

Figure 4

The segment loss probability with different frame size under certain relay states ( N _arq = 2).

Figure 5

The segment loss probability with different frame size under certain relay states ( N _arq = 3).

Figure 6

The segment loss probability with different frame size under certain relay states ( N _arq = 4).

Figure 6, 7, 8, and 9 illustrate the effect of frame size L _fr and N _arq to the energy efficiency of TCP traffic. For a specific relay state, by dividing the TCP segment into a proper number of frames the TCP traffic could reach the optimal energy efficiency. It can be seen when the frame size is short, the energy efficiency is degraded since the overhead cost is dominant which causes a waste of energy. As the frame gets longer, the energy efficiency reaches a peak and then decreases dramatically. Because as the frame gets longer the segment loss probability gets higher, which decreases the TCP throughput. In addition, as the N _arq gets larger, the energy efficiency is increased under different relay state, because large N _arq could reduce the segment loss probability. Moreover, the results get more irrelevant to the channel states as N _arq gets larger. Therefore, to maximize the energy efficiency of TCP traffic in transport layer, a proper frame size should be associated to each relay state.

Figure 7

The energy efficiency with different frame size under certain relay states ( N _arq = 2).

Figure 8

The energy efficiency with different frame size under certain relay states ( N _arq = 3).

Figure 9

The energy efficiency with different frame size under certain relay states ( N _arq = 4).

Figure 10 gives an idea of the importance of switching costs in the decision process of the source node. The probability that the target relay node is idle P_idl is set with different values (0.4, 0.5, 0.625), which determines $c/\bar{\eta }$, the result of the average switching cost divided by the average active reward. When the P_idl is 0.625, the source node switches its relay frequently since the average switching cost c is small comparing to the reward of the target relay. As the P_idl gets smaller (e.g., 0.4), which means the re-selection spends more interactions, the switching costs get larger ($c/\bar{\eta }$, equals to 1.5), the switching events happens occasionally. Because the source node switch only if the reward of the target relay exceeds the current relays reward after compensating for the cost.

Figure 10

The relay selection process with different P _idl .

Figure 11 shows the cumulative discounted rewards of three policies (the proposed solution for RBSC problem, greedy policy and random selection policy) in three P_idl preconditions (0.4, 0.5, 0.625) which determine different ratios of the average switching cost to average active reward $c/\bar{\eta }$. The greedy policy considers only the instantaneous system reward and ignores the state transition probability of each relay node. The one-step lookahead policy of RBSC outperforms the other two policies since it not only take the current reward into account but also analyzes the state transition probability of each relay.

Figure 11

The cumulative discounted reward with different P _idl .

5 Conclusion and future work

In this article, we analyze the energy efficiency of TCP traffic in wireless cooperative relaying networks from a cross-layer perspective. The state of relay node is modeled as a Markov chain and we formulate the relay selection process as a RBSC system which comes from control theory and can be solved by linear programming relaxation. A primal-dual index heuristic algorithm is presented to solve the selection problem. Simulations show that the energy efficiency of TCP can be improved by adjusting the lower layer parameter, e.g., frame size and maximum retransmission time. The one-step lookahead heuristic algorithm could improve the system performance in time horizon according to the current system state and the transition probability.