 Research
 Open Access
 Published:
Adaptive access and rate control of CSMA for energy, rate, and delay optimization
EURASIP Journal on Wireless Communications and Networking volume 2012, Article number: 27 (2012)
Abstract
In this article, we present a crosslayer adaptive algorithm that dynamically maximizes the average utility function. A per stage utility function is defined for each link of a carrier sense multiple accessbased wireless network as a weighted concave function of energy consumption, smoothed rate, and smoothed queue size. Hence, by selecting weights we can control the tradeoff among them. Using dynamic programming, the utility function is maximized by dynamically adapting channel access, modulation, and coding according to the queue size and quality of the timevarying channel. We show that the optimal transmission policy has a threshold structure versus the channel state where the optimal decision is to transmit when the wireless channel state is better than a threshold. We also provide a queue management scheme where arrival rate is controlled based on the link state. Numerical results show characteristics of the proposed adaptation scheme and highlight the tradeoff among energy consumption, smoothed data rate, and link delay.
1. Introduction
In wireless networks, mobile devices are usually battery powered with a limited amount of energy. Therefore, minimization of energy consumption while maintaining the quality of service in the network is crucial. This must be accomplished by adapting the transmission parameters to the system dynamics and to the timevarying channel of the links. In this article, we present a crosslayer adaptive algorithm that dynamically maximizes the average utility function of a carrier sense multiple access (CSMA)based wireless link.
Benefits of such adaptation schemes are shown in some prior works in terms of energy efficiency [1–8]. In such works various control algorithms have been proposed that tradeoff among different goals such as energy consumption, average delay, packet dropping probability and bit error rate, and dynamically adapt the transmission parameters to the channel and system state. The aforementioned works assume pointtopoint links with dedicated channels. However, in data transmission networks, where data are generated at random time instances, random access schemes are used to efficiently exploit channel resources. In such systems, there are more users than available channels, and at any given time only a subset of users can access the channels. Therefore, the optimality of channel access decision is crucial in random access networks. Random access is widely used in ad hoc networks as it can be implemented in a distributed manner. Wireless local area networks (WLAN) and practical personal or sensor networks usually use random access control in their ad hoc operation mode [9, 10]. On the other hand, it is shown recently that CSMA protocols can achieve maximum stable throughput [11] while keeping bounded queuing delay [12], and it can achieve a collision free WLAN [13].
Optimization of random access networks was first proposed in order to achieve single hop proportional fairness for slotted ALOHA networks [14]. Different types of fairness are also considered and random access control is modeled as a utility maximization problem in [15]. In addition, the crosslayer optimization problem of random access control and transmission control protocol is solved as a network utility maximization problem [16]. Newtonlike algorithms are also provided for energy and throughput optimization with endtoend delay constraint in multi hop random access network [17]. However, in the aforementioned articles static transmission probability was used and opportunity of time varying and adaptive control was ignored.
On the other hand, queuebased random access algorithms were studied in [18], where access probabilities are assumed to be adapted based on queue sizes. Stability of the proposed algorithms was verified and their delay performance was shown to surpass fixed optimization algorithms. Also a heuristic differential queuebased scheduling algorithm is proposed in [19] which shows superior performance compared to 802.11 through experimental results. However, such queuebased algorithms are inappropriate for fading channels and prioritize links with low channel quality, which results in low energy efficiency [20].
In this article, we propose crosslayer adaptive algorithms; derived from dynamic programming, for distributed optimization of the links in CSMAbased wireless networks operating in mobile environments. As a performance metric, we define the per stage utility of the link as a weighted concave function of energy consumption, smoothed data rate, and smoothed queue size in the link, where the weights are assigned based on the desired tradeoff among them. The algorithms maximize the average utility by dynamically adapting the channel access decision and transmit data rate (by selecting different modulation and coding schemes) according to the queue size of the link and the availability and quality of the timevarying channel (channel state is assumed to be known at the transmitter). Both, finitetime horizon (FTH) and infinitetime horizon (ITH) problems are considered. In the first case, the utility sum is maximized for a finite time period, whereas in the second case, the longterm average utility is maximized.
We consider a mobile environment with frequencyflat timevarying channel response. This requires suitable models of the wireless channel dynamics. Here, we use finitestate Markov chains (FSMC) to model channel dynamics, such that channel timecorrelation at network links is partially exploited by the proposed algorithms. Although the physical wireless channel is inherently nonMarkovian, it has been shown that stationary Markov chains can capture the essence of the channel dynamics [21]. Many transmission adaptation algorithms are based on firstorder Markov channel models [1, 2]. Here, we consider first and secondorder Markov chains to model characteristics of network links.
The numerical simulations show the benefits of the proposed adaptation algorithms in terms of energy efficiency, and highlight the tradeoff among energy consumption, smoothed data rate, and delay in links of a CSMA network. They also show that the use of suitable Markov model for the wireless channel improves performance of the adaptation algorithm, mainly for slow fading channels. Algorithms based on uncorrelated, first and secondorder Markov models are considered and their performance is compared through simulations.
The rest of the article is organized as follows. Section 2 presents the system model and in particular it describes the model of the network links as well as wireless channel models. In Section 3, per stage utility of the links is defined. Consequently, the utility sum maximization for a finite time period is formulated as an optimal finitehorizon control problem. Similarly, the longterm average utility maximization is formulated as an optimal infinitehorizon control problem. Section 4 uses dynamic programming to compute the optimal adaptation policies for the problems formulated in Section 3. We have investigated structural properties of the optimal solution in Section 5. Numerical results and comparisons are described in Section 6. Finally, Section 7 concludes the article.
2. System model
In this section, we describe the model of the random access links as well as wireless channel models.
2.1. Link model
We consider an ad hoc network where links use CSMA protocol similar to the one provided in [22] which prevents collision among links and also resolves hidden and exposed node problems which exist in wireless networks [23]. As shown in Figure 1, we assume a slotted transmission model where each timeslot, of duration T_{s}, contains both a data slot and a number of control mini slots. When the link has a packet to transmit, it should wait for a random value of W control minislots, and if no other link has reserved the channel earlier, it will send a short request to send packet to reserve the channel. Then, the potential receiver which also perceives that the channel is idle will response with a clear to send (CTS) packet that allows the transmitter to transmit and informs possible interfering nodes that the channel will be used. Once the transmitter receives the CTS, it sends its packet in the data slot.
Timeslot k is defined as the time interval [(k  1)T_{s}, kT_{s}). We use I_{ k } to denote the channel access, where I_{ k } = 1 indicates that the link has decided to access the channel at the k th timeslot. The control policy adapts I_{ k } in each slot based on the system and channel state. Also B_{ k } = 1 indicates that the link should delay its transmission because the channel is already occupied by another link. We model B_{ k } as a Bernoulli process where P_{ B } Pr{B_{ k } = 1} is the channel occupancy probability. The Bernoulli distribution is widely used to model the statistics of B_{ k } in CSMA networks [24].
The link has a queue of maximum size L. Let q_{ k } denote the number of packets in the queue at the k th timeslot, which is assumed to be known at the transmitter. Obviously, I_{ k } = 0 when q_{ k } = 0. r_{ k } denotes the controlled number of packets that arrive the queue in slot k, which we will call arrival rate hereafter. The value of r_{ k } should be chosen both to provide suitable rate for source data and to prevent delay due to backlog through adapting source rate to the link state [25]. To avoid buffer overflow the arrival rate is constrained by r_{ k } ≤ (L  q_{ k } ). The queue update equation is
Where C_{ k } indicates the maximum number of packets that can be transmitted during the k th data slot. C_{ k } depends on the channel state, and it is assumed to be known at the transmitter at the beginning of each timeslot. We call the data that the physical layer transmits in one time slot a frame and the link consumes a constant energy e for transmission of frame in the data slot. Thus, the energy consumed in the k th timeslot will be E_{ k } = eI_{ k } (1  B_{ k } ).
We also consider the exponentially weighted moving average (EWMA) of the queue occupancy ${\stackrel{\u0304}{q}}_{k}$ and of the arrival rate ${\stackrel{\u0304}{r}}_{k}$ as the link state variables which are defined as follows
Note that ${\stackrel{\u0304}{q}}_{k}$ and ${\stackrel{\u0304}{r}}_{k}$ can be viewed as "smoothed" measures of the delay and data rate in the link. The parameters θ_{ q } and θ_{ r } determines the time scale over which the smoothing is performed. The smaller the value of θ_{ r } or θ_{ q } , the shorter the time period of moving average (smoothing). Values of θ_{ r } and θ_{ q } are determined based on the tolerance of the applications to the delay and data rate variations in the link. Random early detection protocol has used the EWMA of the delay (${\stackrel{\u0304}{q}}_{k}$) as a criterion for congestion control [26]. In addition, the EWMA of the rate (or smoothed rate), ${\stackrel{\u0304}{r}}_{k}$, has been used in [27, 28] as a measure of the quality of service. EWMA is also used as a metric in statistical quality control [29].
2.2. Channel model
We consider a frequencyflat blockfading channel, where the channel remains constant during each timeslot, and can change for consecutive timeslots. Therefore, we assume that the duration of each timeslot (T_{ s } ) is less than the coherence time of the channel. Hence, channel responses at different timeslots can be correlated. The channel power gain at the k th timeslot is denoted by γ_{ k } . Since we assume constant transmit power, the received signaltonoise ratio (SNR) in the link for the k th timeslot will be proportional to γ_{ k } . The fading range 0 ≤ γ is partitioned into M disjoint regions so that the j th region is defined as R_{ j } = {γ : A_{ j } ≤ γ < A_{j+1}}, where A_{1} = 0 and A_{M+1}= ∞. The channel for the k th timeslot is in state j if γ_{ k } ∈ R_{ j } . Also the values of A_{ j } are selected according to the adaptive modulation and coding as follows. Consider that transmitter has a set of modulation and coding schemes {Q_{1}, Q_{2}, ... , Q_{ M } } to select from in each time slot. We select A_{ j } ; j = 2, ... , M such that if channel is in state j, transmitter can use Q_{ j } and ensures that the frames transmitted with this scheme have error probability less than FER_{th} which is a target threshold for frame error rate (FER).
Let $\mathcal{C}=\left\{{\u0108}_{j}j=1,...,M\right\}$ denote the set of number of transmit packets associated with the set of channel states, if γ_{ k } ∈ R_{ j } then ${C}_{k}={\u0108}_{j}$ where C_{ k } is the number of packets that can be transmitted in the k th timeslot. Note that packet error rate will be below the same threshold, i.e., PER_{th} = FER_{th}, since (a) adaptive algorithm applies different Q_{ i } schemes so that transmitter ensures the same error threshold for all frames, and (b) if a frame transmission was unsuccessful all packets in the frame will be lost. Therefore, the ratio of the lost packets to the total number of packets equals the ratio of the erroneous frames to the total number of frames, regardless of the channel state.
Subsequently, we consider three models for the random process C_{ k } , with diverse degrees of complexity.
1. Uncorrelated model
In this model, the channel response at different timeslots are assumed uncorrelated so, where P_{ j } is the probability of the channel state R_{ j } . This simple model may be accurate for fading channels that exhibit high timevariability. It is also the fitting model when there is no prior information about the channel time correlation.
2. Firstorder markov model
To model the time correlation of the channel we use an M state FSMC [30] with time discretized to T_{s} and transition probabilities as
Accordingly, the random process C_{ k } will be modeled with the same M state FSMC so:
The transition probabilities depend on the normalized Doppler frequency f_{d}T_{s} which determines the rate of variation of the channel with respect to the timeslot duration, where f_{d} is the channel Doppler frequency. Although the physical wireless channel is inherently nonMarkovian, it has been shown that an FSMC can capture the essence of the channel dynamics when the number of regions/states (M) is low and the channel fades slow enough (see for example [21] and references therein). Note that the uncorrelated model can be viewed as a particular case of FSMC where P_{ i,j } = P_{ j } , ∀i.
3. Secondorder Markov model
In order to model dynamics of C_{ k } more accurately, we also consider secondorder FSMC channel models. They are more accurate than the firstorder FSMC since C_{k+1}depends on both C_{ k } and C_{k1}.
In this article, we use the socalled Cartesian product method [21] for the secondorder models. We will investigate the effect of the FSMC order on the performance of the resulting algorithm through numerical results. Note that the formulation of the firstorder Markov model can be considered as a special case of the secondorder model with P_{ i,j } = P_{ l,i,j } for any l.
3. Problem formulation
We consider a wireless link in a CSMA network which desires to optimize its transmission rate, energy consumption, and delay. We distinguish two dynamic optimization problems: FTH and ITH problems. In the FTH problem, the performance of the link is optimized over a finite number of timeslots, whereas in the ITH problem the link performance is optimized considering an infinite number of timeslots. Next, they are formulated as dynamic programming problems.
3.1. Finite time horizon
We define a utility maximization problem over N timeslots or stages as follows:
where the expectation is taken over the random process C_{ k } . The function g(s_{ k } , μ_{ k } ) is the utility per stage and is a measure of the quality of service of the link at each timeslot. It depends on the action vector μ_{ k } = (I_{ k } , r_{ k } ) and on the system state vector. We consider a secondorder Markov model for C_{ k } and include component C_{k1}in the state vector ${s}_{k}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k},{C}_{k1}\right)$. Note that the firstorder model can be considered as a special case with ${s}_{k}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k}\right)$. Considering φ(·) as the state update function we can write: s_{k+1}= φ (s_{ k } , μ_{ k } , C_{k+1}, B_{ k } ). In (6) g_{N+1}is the final stage utility which depends only on the final state of the system, s_{N+1}and it can include some limitations or penalties on the final state of the system.
Here we consider a special format for utility per stage function in order to clarify how it controls system performance:
where U(·), and V(·) are suitable continuous, concave functions, and parameters α and β control the tradeoff between rate, energy, and delay in the utility function. A similar formulation for per stage utility is used in [27, 28] for multiperiod utility maximization while queue management and thus queue sizes were not considered.
The number of packets remaining in the queue at the final stage can be penalized with a price of η as follows:
3.2. Infinite time horizon
In this case we maximize the average utility per stage which is defined by
where the action and state vectors as well as the per stage utility function are defined similar to the FTH problem. We consider both the first and secondorder models for the channel state by applying appropriate format of s_{ k } .
4. Optimal adaptive control
To maximize FTH or ITH utility functions the controller should decide optimal actions ${\mu}_{k}^{*}\left({s}_{k}\right)$ at the beginning of each timeslot as a function of the system state s_{ k } . Note that the decision must be causal since future system states are unknown due to the randomness of the channel state (C_{ k } ) and occupancy (B_{ k } ). In this section, by using the DP algorithm [31], we derive algorithms that compute the optimal control functions for the FTH and ITH problems. It is important to remark that the resulting optimal control functions are computed and stored offline. Then, they will be used online to dynamically adapt the actions to the system state. As described earlier, the system state definition can support uncorrelated, first and secondorder channel models so we do not limit the solution to any specific channel model.
4.1. Per stage adaptation to maximize FTH utility
The optimal control policy is the sequence of control functions (one for each timeslot) ${\pi}^{*}=\left\{{\mu}_{1}^{*}\left({s}_{1}\right),{\mu}_{2}^{*}\left({s}_{2}\right),...,{\mu}_{N}^{*}\left({s}_{N}\right)\right\}$that maximize (6). Note that the control functions provide the optimal action for each of the possible system states at different stages. Using the DP algorithm, the optimal policy π* is obtained from the following backward recursion for k = N, N  1, ..., 1:
The function J_{ k } (s_{ k } ) is the maximum expected accumulative utility, achieved under optimal decision, when the system is in state s_{ k } at the k th stage. Thus, J_{1} (s_{1}) is the expected total utility for N stages when the initial state is s_{1}.
The application of the DP algorithm requires computation of function J_{ k } (s_{ k } ) for all possible system states (s_{ k } ) at each stage and necessitates the system state space to be finite. Since the state components ${\stackrel{\u0304}{r}}_{k}$, and ${\stackrel{\u0304}{q}}_{k}$ can take values from continuous spaces, we discretize them using finite grids $\left\{{\stackrel{\u0304}{q}}_{m}^{d}m=1,2,...{M}_{q}\right\}$, and $\left\{{\stackrel{\u0304}{r}}_{m}^{d}m=1,2,...,{M}_{r}\right\}$. Then, we can express each nongrid value as a linear interpolation of the nearby grid values:
where ${w}_{q}^{m}$ and ${w}_{r}^{m}$ are nonnegative weights and ${\sum}_{m=1}^{{M}_{q}}{w}_{q}^{m}\left(\stackrel{\u0304}{q}\right)\phantom{\rule{2.77695pt}{0ex}}={\sum}_{m=1}^{{M}_{r}}{w}_{r}^{m}\left(\stackrel{\u0304}{r}\right)=1$. It can be shown that if Lipschitz condition holds for the functions ${w}_{q}^{m}\left(\stackrel{\u0304}{q}\right)$, ${w}_{r}^{m}\left(\stackrel{\u0304}{r}\right)$, and g(s_{ k } , μ_{ k } ), and for the state update functions (13), the DP solution of the discretized problem converges to the optimal policy for the original continuous problem, as the density of the grid increases [32]. For the problem in hand the utility and the state update functions are continuous and thus satisfy the Lipschitz condition. We select ${w}_{q}^{m}$ and ${w}_{r}^{m}$ to be suitable continuous functions of the state variables which are chosen on the basis of geometric considerations as suggested in [31] and each state will be described by two nearby discrete states:
The DP algorithm for the discretized state space ${s}_{k}^{d}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k}^{d},{\stackrel{\u0304}{r}}_{k}^{d},{C}_{k},{C}_{k1}\right)$ and k = N, N  1, ... , 1 will be
where ${\stackrel{\u0303}{J}}_{k+1}\left({s}_{k+1}\right)$ is the estimation of ${J}_{k+1}\left({s}_{k+1}\right)$ by its values at discretized states and is given by:
In the above equation, we have ${s}_{k+1}^{d}=\left({q}_{k+1},{\stackrel{\u0304}{q}}_{m}^{d},{\stackrel{\u0304}{r}}_{l}^{d},{C}_{k+1},{C}_{k}\right)$ and ${s}_{k+1}=\left({q}_{k+1},{\stackrel{\u0304}{q}}_{k+1},{\stackrel{\u0304}{r}}_{k+1},{C}_{k+1},{C}_{k}\right)$. Furthermore q_{k+1}, ${\stackrel{\u0304}{q}}_{k+1}$ and ${\stackrel{\u0304}{r}}_{k+1}$ are given by (1), (2), and (3), respectively. We consider the secondorder Markov model by using both C_{k1}and C_{ k } in the state vector. The other channel models can be considered as its special case. The solution provided in Equations (17)(19) is valid for any concave and continuous function of g.
Next we replace g(s_{ k } , μ_{ k } ) in (17) and (18) with its format provided in (7) and calculate the expectation, ${E}_{{C}_{k+1},{B}_{k}}\left[\cdot \right]$, using the channel transition probabilities, $\mathrm{Pr}\left\{{C}_{k+1}={\u0108}_{j}{C}_{k},{C}_{k1}\right\}$ and channel occupancy probability, Pr{B_{k} = 1}. The expected accumulative utility and the optimal control functions for k = N, N  1, ... , 1 will be
Since ${\stackrel{\u0304}{r}}_{k}^{d}$ and ${\stackrel{\u0304}{q}}_{k}^{d}$ are independent of the decision in the k th timeslot, $\mathsf{\text{U}}\left({\stackrel{\u0304}{r}}_{k}^{d}\right)\alpha \mathsf{\text{V}}\left({\stackrel{\u0304}{q}}_{k}^{d}\right)$ do not affect the maximization in (21). Also the summations in (20) and (21) are over all M channel states and two possible channel occupancy conditions.
The discrete DP algorithm can be executed offline and the resulting optimal policy can be stored in a lookup table available at the transmitter. Then, it will be used online to dynamically adapt the action to the system state.
4.2. Statebased adaptive control to optimize average utility per stage
To solve the ITH problem of (9) we first define the average utility per stage when using policy π and starting from the initial state s as
We denote the optimal policy as π* which produces the maximum average utility per stage J*. Both π* and J* are independent of the initial state since the influence of the utility of the early stages on the average utility reduces to 0 as N → ∞. Moreover, since the utility per stage, the transition probabilities (4), and the state update Equations (1)(3) are all stationary, the optimal policy will be stationary (does not change from stage to stage). Therefore, it is a single function, μ*(s), that maps the system states to actions regardless of the stage.
J*, together with the socalled relative value function h*(s), should satisfy the following Bellman's fixed point equation [31] for every state:
where s^{+} indicates the successor state of the current state s. Considering φ(·) as the state update function s^{+} = φ (s, μ, C^{+}, B). The expectation in Equation (23) is over the random processes {B_{ k } } and {C_{ k } }.
We use a modified relative value iteration algorithm to solve the ITH problem [31]. First, we define a variant of the Bellman operator over any function f as
where parameter τ ∈ (0, 1) is a scalar. Then, the following iterative algorithm is used in order to calculate h^{(n+1)}(s) for all states of the state space in the iteration (n + 1):
where s' is some fixed state. We initialize this algorithm with h^{(0)}(s) = g(s, I = 0, r = 0). Convergence of (25) is guaranteed since queue and channel states are recurrent [31]. The decision will also be updated and will finally converge to the optimal decision as n → ∞:
The practical application of (24) requires the state space to be discrete, so we use the same discretization procedure as in Section 4.1. This results in the following modified Bellman operator:
Therefore, we apply (27) and compute h^{(n)}(s^{d} ) for all possible discrete states. For the uncorrelated and firstorder channel models there are (L × M_{ q } × M_{ r } × M) discrete states and for the secondorder channel model this number should be multiplied by M.
5. Structural properties of the optimal solution
In the previous section, we provided DP algorithms that can be applied to find optimal decisions through numerical calculations. In this section, we investigate some structural properties of the solution. We use the following practical assumptions throughout this section.
Assumption 1: Per stage utility function has a format of (7) and (8) with U(·), and V(·) as increasing functions.
Assumption 2: Consider $\left({C}^{}={\u0108}_{{a}^{}},C={\u0108}_{a}\right)$ as the channel state in the previous and current slot, respectively, and $\left({\u0108}_{{b}^{}},{\u0108}_{b}\right)$ as other possible channel states in these two slots, where ${\u0108}_{a}\le {\u0108}_{b}$ and ${\u0108}_{{a}^{}}\le {\u0108}_{{b}^{}}$. We assume that there exists a j such that the following inequality holds for channel transition probabilities:
where ${P}_{{a}^{},a,i}$ is the probability of going from channel states $\left({\u0108}_{{a}^{}},{\u0108}_{a}\right)$ to the next state ${C}^{+}={\u0108}_{i}$ as defined for second order Markov model.
Assumption 2 is valid in practice for Markov channels since $\left({\u0108}_{{a}^{}},\phantom{\rule{0.3em}{0ex}}{\u0108}_{a}\right)$ is supposed to be lower than and $\left({\u0108}_{{b}^{}},{\u0108}_{b}\right)$and each side of the inequality calculates the probability of going to the first j states with lowest rates. For example, if ${P}_{{b}^{},b,1}\le {P}_{{a}^{},a,1}$ then the inequality will be true for j = 1 for and assumption 2 holds. If the inequality turns out to be true for any value of j then the assumption is correct. Based on this assumption we provide the following lemma:
Lemma 1: If f(C) and g(C) are two increasing functions, f(C) ≤ g(C), C^{+} is the next channel state, and similar to Assumption 2 ${\u0108}_{a}\le {\u0108}_{b}$ and $\left({\u0108}_{{a}^{}}\le {\u0108}_{{b}^{}}\right)$ then we have
Proof is provided in the Appendix.
5.1. Structural properties of FTH solution
The following theorem indicates monotonicity of J_{ k } (s_{ k } ) versus the state variables.
Theorem 1: J_{ k } (s_{ k } ) is a decreasing function of q_{ k } and ${\stackrel{\u0304}{q}}_{k}$, and an increasing function of ${\stackrel{\u0304}{r}}_{k}$ and C_{ k } for all values of k.
Proof: In order to prove the theorem we show through induction that for k = N + 1, ..., 1 we have J_{ k } (s_{ k } + Δ) ≤ J_{ k } (s_{ k } ) for any vector Δ that increase q_{ k } and ${\stackrel{\u0304}{q}}_{k}$, and decrease ${\stackrel{\u0304}{r}}_{k}$ and C_{ k } .
Based on Equation (11) for optimal decision in the k th stage, we define G_{ k } (s_{ k } , μ_{ k } ) as:
Thus, ${J}_{k}\left({s}_{k}\right)={G}_{k}\left({s}_{k},{\mu}_{k}^{*}\right)$ where ${\mu}_{k}^{*}$ is optimal decision for state s_{ k } . Also we define Δ = (δ_{1}, δ_{2},  δ_{3}, δ_{4}, δ_{5}) for any value of δ_{ i } ≥ 0, i = 1, ... , 5 such that ${s}_{k}+\mathrm{\Delta}=\left({q}_{k}+{\delta}_{1},{\stackrel{\u0304}{q}}_{k}+{\delta}_{2},{\stackrel{\u0304}{r}}_{k}{\delta}_{3},{C}_{k}{\delta}_{4},{C}_{k1}{\delta}_{5}\right)$ is an element of the state space.
For k = N + 1 we have ${J}_{N+1}\left({s}_{N+1}\right)=U\left({\stackrel{\u0304}{r}}_{N+1}\right)\gamma \mathsf{\text{V(}}{\stackrel{\u0304}{q}}_{N+1}\mathsf{\text{)}}\eta {q}_{N+1}$ and using assumption 1 it is clear that J_{N+1}(s_{N+1}+ Δ) ≤ J_{N+1}(s_{N+1}). Assuming J_{k+1}(s_{k+1}) is a monotonic function we show J_{ k } (s_{ k } ) is also monotonic for k = N, ... , 1 which completes the proof.
We define ${\mu}_{k,\mathrm{\Delta}}^{*}$ as optimal decision for state s_{ k } + Δ in stage k, so we can write ${J}_{k}\left({s}_{k}+\mathrm{\Delta}\right)={G}_{k}\left({s}_{k}+\mathrm{\Delta},{\mu}_{k,\mathrm{\Delta}}^{*}\right)$, however ${\mu}_{k,\mathrm{\Delta}}^{*}$ is not an optimal decision for state s_{ k } so we have:
Using Assumption 1 it is clear that g is a monotonic function of the state variables
We consider φ (·) as the state update function and define two possible next states ${s}_{k+1,\mathrm{\Delta}}^{*}=\phi \left({s}_{k}+\mathrm{\Delta},{\mu}_{k,\mathrm{\Delta}}^{*},{C}_{k+1},{B}_{k}\right)$ and ${s}_{k+1}^{\#}=\phi \left({s}_{k},{\mu}_{k,\mathrm{\Delta}}^{*},{C}_{k+1},{B}_{k}\right)$. For known values of C_{k+1}and B_{ k } we can use (1)(3) and easily show that ${s}_{k+1,}^{{*}_{\mathrm{\Delta}}}={s}_{k+1}^{\#}+{\mathrm{\Delta}}^{\prime}$ in which ${\mathrm{\Delta}}^{\prime}=\left({\delta}_{1}^{\prime},{\delta}_{2}^{\prime},{\delta}_{3}^{\prime},{\delta}_{4}^{\prime},{\delta}_{5}^{\prime}\right)$ for some ${\delta}_{i}^{\prime}\ge 0$. Thus ${J}_{k+1}\left({s}_{k+1,\mathrm{\Delta}}^{*}\right)\le {J}_{k+1}\left({s}_{k+1}^{\#}\right)$.
We define $f\left({C}_{k+1}\right)\triangleq {E}_{B}\left[{J}_{k+1}\left({s}_{k+1,\mathrm{\Delta}}^{*}\right)\right]$ and $g\left({C}_{k+1}\right)\triangleq {E}_{B}\left[{J}_{k+1}\left({s}_{k+1}^{\#}\right)\right]$ since B is independent of the system state thus we have f(C_{k+1}) ≤ g (C_{k+1}) and since J_{k+1}(·) is an increasing function of C, then f(·) and g(·) are increasing functions. Applying Lemma 1 with ${\u0108}_{{a}^{}}=\left({C}_{k1}{\delta}_{5}\right)$, ${\u0108}_{a}=\left({C}_{k}{\delta}_{4}\right)$, ${\u0108}_{{b}^{}}={C}_{k1}$, and ${\u0108}_{b}={C}_{k}$ we find that
Combining (31), (32) and considering definition of G_{ k } in (29) we get
Equation (30) together with (33) prove the theorem by showing: J_{ k } (s_{ k } + Δ) ≤ J_{ k } (s_{ k } ).
■
Assuming uncorrelated channel model the following theorem indicates the "threshold structure" of the optimal transmission policy versus the channel state.
Theorem 2: If the optimal access decision in state ${s}_{k}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k}\right)$ is ${I}_{k}^{*}\left({s}_{k}\right)=1$, then for another possible state ${s}_{k}^{\prime}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k}^{\prime}\right)$ in the same slot with improved channel state ${C}_{k}^{\prime}\ge {C}_{k}$ we have ${I}_{k}^{*}\left({s}_{k}^{\prime}\right)=1$.
Proof: Assume ${\mu}_{k}^{*}\left({s}_{k}\right)=\left({I}_{k}=1,{r}_{k}\right)$ but ${\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)=\left({I}_{k}^{\prime}=0,{r}_{k}^{\prime}\right)$ as the optimal decision for s_{ k } and ${s}_{k}^{\prime}$, respectively. According to the definition of G_{ k } in the proof of Theorem 1, ${\mu}_{k}^{*}\left({s}_{k}\right)$ maximizes G_{ k } (s_{ k } , μ_{ k } ) and we have
On the other hand since s_{ k } and ${s}_{k}^{\prime}$ differs only in the channel state, we have $g\left({s}_{k},{\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)\right)=g\left({s}_{k}^{\prime},{\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)\right)$ and by using $\left({I}_{k}^{\prime}=0,{r}_{k}^{\prime}\right)$ for both states, queue size will modify similarly for s_{ k } , and ${s}_{k}^{\prime}$ which results in the same next state, ${s}_{k+1}={s}_{k+1}^{\prime}$. Also for uncorrelated channel model the averaging over next channel state does not depend on the current state, thus ${E}_{{C}^{+}}\left\{{E}_{B}\left[{J}_{k+1}\left({s}_{k+1}\right)\right]\right\}={E}_{{C}^{+}}\left\{{E}_{B}\left[{J}_{k+1}\left({s}_{k+1}^{\prime}\right)\right]\right\}$ and
By applying decision ${\mu}_{k}^{*}\left({s}_{k}\right)=\left({I}_{k}=1,{r}_{k}\right)$ and since transmission with better channel state will decrease q and $\stackrel{\u0304}{q}$ which increase J_{ k } according to Theorem 1, we have
Combining (34), (35), and (36) results in
which is in contrast to optimality of the ${\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)=\left({I}_{k}^{\prime}=0,{r}_{k}^{\prime}\right)$. Thus, we should have ${\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)=\left({I}_{k}^{\prime}=1,{r}_{k}^{\prime}\right)$.
■
Note that Theorem 2 may be incorrect when channel state is time correlated. For example, consider two possible channel states C_{ k } and ${C}_{k}^{\prime}$, with ${C}_{k}<{C}_{k}^{\prime}$ and assume that optimal decision is to transmit for a state with C_{ k } . Also assume that probability of going from ${C}_{k}^{\prime}$ to a better channel state and from C_{ k } to a worse channel state is high. So, we can argue heuristically that in this condition it may be optimal to transmit data when channel is in state C_{ k } but not to transmit when it is in state ${C}_{k}^{\prime}$.
5.2. Structural properties of ITH solution
We provide structural properties of ITH solution in this section through the following theorems. First we show that relative value function, h*(s), is a monotonic function in Theorem 3 and then prove the threshold structure of access decision versus channel state in Theorem 4.
Theorem 3: h*(s), is a decreasing function of q and $\stackrel{\u0304}{q}$, and an increasing function of $\stackrel{\u0304}{r}$ and C.
Proof: We define Δ = (δ_{1}, δ_{2}, δ_{3}, δ_{4}, δ_{5}) with δ_{ i } ≥ 0 and show that h*(s + Δ) ≤ h*(s). We also define G_{ f } (s, μ(s)) on function f as
Assuming μ* as the decision that maximizes G_{ f } and according to the Bellman equation (24) we have
Taking into account h*(s) = lim_{k→∞}h^{(n)}(s), we prove through induction for every iteration n, h^{(n)}(s + Δ) ≤ h^{(n)}(s). For n = 0 we define h^{(0)}(s) = g(s, I = 0, r = 0) which according to Assumption 1 it is clear that h^{(0)} (s + Δ) ≤ h^{(0)} (s). We assume that h^{(n)}(s) is monotonic and show h^{(n+1)}(s) is also monotonic. First, we show that B _{ τ }h^{(n)}(s) is monotonic. Using (26) for states s and s + Δ and assuming μ* and ${\mu}_{\mathrm{\Delta}}^{*}$, respectively, as maximizing actions we have
From definition of g it is clear that $g\left(s+\mathrm{\Delta},{\mu}_{\mathrm{\Delta}}^{*}\right)\le g\left(s,{\mu}_{\mathrm{\Delta}}^{*}\right)$. We can use the similar approach as the proof of Theorem 1 and apply Lemma 1 to show that
which results in
Combining (38) and (40) we find that B _{ τ }h^{(n)}(s + Δ) ≤ B _{ τ }h^{(n)}(s). Using Equation (25) and taking into account that B _{ τ }h^{(n)}(s') is independent of the state vector, it can be easily shown that h^{(n + 1)}(s) is a monotonic function.
■
Assuming uncorrelated channel model the following theorem indicates existence of a threshold for channel state that the link should decide to transmit when channel state is better than or equal to that threshold.
Theorem 4: There exists a threshold, C_{th}, that for ${s}_{\mathsf{\text{th}}}=\left(q={C}_{\mathsf{\text{th}}},\stackrel{\u0304}{q},\stackrel{\u0304}{r},{C}_{\mathsf{\text{th}}}\right)$ with any $\stackrel{\u0304}{q}$ and $\stackrel{\u0304}{r}$, we have I*(s_{ th } ) = 1. Also for any s with C ≥ C_{th} and q ≥ C_{th} we have I*(s) = 1.
Proof: Assume in timeslot k we have C_{ k } = C^{max} and q_{ k } = C^{max}, transmission at this time has the energy cost of β e but it will reduce q by C^{max} which will reduce $\stackrel{\u0304}{q}$ by θ_{ q }C^{max} and also will reduce the future costs related to the queue size. However, transmission of theses C^{max} packets at any later time slot requires the same amount of energy. Thus, it is better to transmit these packets at state s_{th} to reduce the queue size as early as possible and reduce the future costs related to the queue size. We conclude that: "if C = C^{max} and q = C^{max} then I*(s) = 1" which proves existence of C_{th}.
In order to prove the second part of the theorem we assume $s=\left(q\ge {C}_{\mathsf{\text{th}}},\stackrel{\u0304}{q},\stackrel{\u0304}{r},C\ge {C}_{\mathsf{\text{th}}}\right)$ and consider optimal decisions μ*(s_{th}) = (I*(s_{th}), r*(s_{th})) and μ*(s) = (I*(s), r*(s)) for states s_{th} and , respectively. If I(s) = 0 we can show similar to the proof of Theorem 2 that it cannot be an optimal transmission policy.
■
6. Numerical results
For numerical analysis of the adaptive control algorithms provided in Section 4 we consider a lightweight sensor in a wireless network that may transmit its status using few bits. In each timeslot the sensor may send its own packet or forward packets of other sensors. We assume a Rayleigh flat fading channel, and use a set of simple Modulation and Coding schemes. Note that our adaptive algorithm only requires the FSMC model which can be found for many practical fading channels [21] and do not depend on Rayleigh fading assumption or Modulation schemes. However, in this section we consider the following types of modulations joint with ReedSolomon (RS) coding:

Q_{1}: No transmission since link is in deep fade.

Q_{2}: BPSK with RS (63,47).

Q_{3}: QPSK with RS (127,94).

Q_{4}: 16QAM with RS (255, 188).
Note that in each time slot one frame will be transmitted and the time duration of the frames is identical for different schemes. Figure 2 illustrates FER of the aforementioned schemes. Setting 0.01 as the FER threshold, we find SNR thresholds A_{ j } for the fading regions which ensure the required FER limit for (Q_{1}, Q_{2}, Q_{3}, Q_{4}) as {0, 3.8, 7.77, 33.1, ∞}. For example, we will use Q_{3} while SNR is between 7.77 (8.9 dB) and 33.1 (15.2 dB). Assuming a Rayleigh fading channel, we use the Markov model proposed in [21, 30] to obtain the transition probabilities for a given normalized Doppler frequency. Unless otherwise indicated we assume the average SNR at the receiver is $\overline{\mathsf{\text{SNR}}}=10\phantom{\rule{2.77695pt}{0ex}}\mathsf{\text{dB}}$ and normalized Doppler frequency is f_{d}T_{s} = 0.02. Each frame that will be transmitted in the data slot contains a coded block. Considering the packet length of 47 bits, 0, 1, 2, or 4 packets can be transmitted in a frame based on the channel state so $\mathcal{C}=\left\{0,\phantom{\rule{0.3em}{0ex}}1,\phantom{\rule{0.3em}{0ex}}2,\phantom{\rule{0.3em}{0ex}}4\right\}$.
Regarding the per stage utility function (7) we use $U\left({\stackrel{\u0304}{r}}_{k}\right)=\mathrm{log}\left(\epsilon +{\stackrel{\u0304}{r}}_{k}\right)$, and $\mathsf{\text{V}}\left({\stackrel{\u0304}{q}}_{k}\right)={\left({\stackrel{\u0304}{q}}_{k}\right)}^{2}$ to avoid very small rates and large queue sizes. Logarithmic utility is used to provide proportional fairness in the network [14] and prevent selfish rate maximization of the link. Also it provides fairness among multiple flows over a single link [27]. Recently, it has been shown, based on experimental results and WeberFechner psychophysical law, that user experience and satisfaction follows logarithm laws, and quality of experience (QoE) versus rate is formulated as QoE(r) = log (ar + b) [33]. In the utility function, energy and queue sizes are used with negative weights to minimize energy consumption and delay. Remember that the energy consumed at the k th timeslot is given by E_{ k } = eI_{ k } (1  B_{ k } ). Therefore
We also use the following parameters for simulations unless otherwise indicated: θ_{ q } = θ_{ r } = 0.7, L = 12, α = 0.005, βe = 1, , P_{ B } = 0.1, ε = 0.001. Using these parameters, Figure 3 shows the selected per stage utility which is an increasing function of ${\stackrel{\u0304}{r}}_{k}$ and decreasing function of ${\stackrel{\u0304}{q}}_{k}$. Our selected parameters result in a utility function which is negative; however, behavior of this function versus system state is such that by maximizing it we will maximize rate while minimizing energy and delay.
As described in Section 4, continuous state variables, ${\stackrel{\u0304}{r}}_{k}$ and ${\stackrel{\u0304}{q}}_{k}$, should be discretized in order to achieve a finite state system and dynamic programming solution. We set M_{ r } = 21, and M_{ q } = 13 for discretization. It is shown in our simulations that enhancement achieved by selecting greater values for M_{ r } , and M_{ q } is insignificant. Also the maximum queue size is assumed to be L = 12 and the number of arrival packets at each stage r_{ k } is limited by 4.
6.1. FTH results
As indicated earlier for FTH problem we use (8) as the final stage utility function with η = 5 as the price for packets remaining in the queue where $U\left({\stackrel{\u0304}{r}}_{k}\right)=\mathrm{log}\left(\epsilon +{\stackrel{\u0304}{r}}_{k}\right)$, and $\mathsf{\text{V}}\left({\stackrel{\u0304}{q}}_{k}\right)={\left({\stackrel{\u0304}{q}}_{k}\right)}^{2}$. We assume that the initial state of the link is ${\stackrel{\u0304}{r}}_{1}=0.1$, ${\stackrel{\u0304}{q}}_{1}={q}_{1}=0$ and consider a flat fading Rayleigh channel. The recursive algorithm (16)(18) is used to obtain the optimal control policy (over the discretized state space) for different values of N. The transition probabilities of the Markov models are computed as described earlier.
The optimal control policies are then used in Monte Carlo simulations, over the channel response γ_{ k } and the channel occupation B_{ k } processes, to maximize J_{1}(s_{1}), the sum of the utilities of the N stages. Figure 4 illustrates the J_{1}(s_{1}), as a function of the number of timeslots N, for different channel correlation models and two values of average SNR. This figure shows that the performance is enhanced by exploiting the channel correlation through the FSMC models, mainly for large values of N. It also shows that the use of secondorder FSMC is not worthwhile in these cases. For N > 50, J_{1} (s_{1}) varies almost as a linear function of N where the slope depends on the channel correlation model and the average SNR.
We have also investigated the dependence of performance on the initial state of the link. Figure 5 illustrates the average utility, J_{1}(s_{1})/N, versus the initial EWMA rate ${\stackrel{\u0304}{r}}_{1}$ for different values of N. As expected the higher initial EWMA rate, the higher average utility. It also shows that sensitivity to the initial state decreases as the number of slots increases.
Size of the grid used for discretization, M_{ q } and M_{ r } , can affect performance of the system. In Table 1 we provide FTH performance of the algorithms that have used different values of M_{ q } and M_{ r } for N = 40. We can see that enhancement achieved by selecting values greater than M_{ r } = 21, and M_{ q } = 13 is negligible.
6.2. ITH results
We use the modified relative value iteration algorithm (25), with τ = 0.9, in order to find the optimal control policies for the infinite time horizon problems. Unless otherwise indicated, in the following results we have considered the firstorder FSMC channel model. In each iteration the algorithm computes new values of I^{(n)}(s) and r^{(n)}(s) for all possible states and finally it converges to the optimal control policy (for the discretized problem) μ*(s) = (I*(s), r*(s)). Figure 6 illustrates the convergence of the iterative algorithm, by showing the percentage of decisions, (I^{(n)}(s), r^{(n)}(s)). which are modified in each iteration in comparison with the previous iteration. Apart from the optimal control policy, the algorithm also provides the optimal relative value function, h*(s). Figure 7a,b illustrates r*(s), and h*(s) versus some elements of the state vector while fixing the others. They show interesting properties of the optimal policy and relative value function with respect to the system state. For example, Figure 7a shows that the arrival rate should be reduced as the channel goes to the fade state or as the EWMA of rate increases. Figure 7b indicates that the relative utility function decreases as the queue size increases or the channel goes to the fade state.
Figure 8 demonstrates the optimal actions for a particular realization of the channel process in a period of 200 timeslots. Note that when the channel goes to a deep fade during timeslots 231 to 282 the link does not access the channel (I_{ k } = 0) so there is not energy consumption in this period (E_{ k } = 0). Also, new packet arrivals are reduced to prevent high queue backlog but kept at a minimum rate to prevent $\mathrm{log}\left(\epsilon +{\stackrel{\u0304}{r}}_{k}\right)$ from very negative values. After the deep fade finishes the link starts to transmit backlogged packets while keeping slow arrival rate until timeslot 288.
Based on the selected format of the per stage utility (7), we can reduce the energy consumption by increasing β. However, this is achieved at the cost of reducing the transmission rate and increasing the delay as shown in Figure 9. In other words, the figure shows the tradeoff between energy, rate, and delay as a function of β. Here, the average delay is calculated using the little's low: $\stackrel{\u0304}{D}={\sum}_{k}{q}_{k}/{\sum}_{k}{r}_{k}$ [34].
Figure 10 shows the performance of the optimal policies, obtained from different channel correlation models, as a function of the time variability of the channel. In particular, it shows the resulting average utility per stage, as a function of the normalized Doppler frequency f_{d}T_{s}, for the first and secondorder FSMC models and different values of the EWMA parameters for packet arrivals and queue occupancy. It shows that average utility is higher for fading channels with higher f_{d}T_{s} since channel remains for a short time in deep fades. For θ = θ_{ q } = θ_{ r } = 0.7, which corresponds to larger averaging time of rate and queue size, both channel models exhibit similar performance. However, for θ = θ_{ q } = θ_{ r } = 0.3 we see that the more accurate secondorder FSMC model enhances the performance of the link compared to the firstorder FSMC model.
7. Conclusions
We addressed the problem of optimal channel access and rate adaptation in the links of CSMA wireless networks. We defined a utility function that trades off the energy consumption and the average packet transmission rate and delay. By using dynamic programming, we derive algorithms and optimal policies that maximize the average utility by adapting the arrival packet rate and channel access as functions of the queue occupancy, channel state, and smoothed rate. The optimal policies can be computed and stored offline. Then, they can be used online for dynamic access control and queue management of the link. The proposed algorithms exploit the time correlation of the channel by means of different FSMC models. Both FTH and ITH problems were addressed. In the first case, the average utility is optimized for a finite time period, whereas in the second case, the longterm average utility is maximized. Structural properties of the optimal solution are investigated and it is shown that optimal transmission policy has a threshold structure versus the channel state. For the ITH problem we proved the existence of a channel state that the link should always transmit when the channel is in that state or in a better one. Numerical results show that the overall performance of the link can be enhanced by increasing the order of the FSMC channel model. However, it increases the complexity of the algorithms and the memory required to store the optimal policies.
Appendix
Proof of Lemma 1
The difference between right and lefthand of inequality (28) can be calculated using the channel transition probabilities:
We partition the summation and rewrite it as
the first inequality is a result of $g\left({\u0108}_{i}\right)f\left({\u0108}_{i}\right)\ge 0$ and the second one considers $g\left({\u0108}_{j}\right)\le g\left({\u0108}_{i}\right)$, i = j + 1, ... , M, and $f\left({\u0108}_{j}\right)\ge f\left({\u0108}_{i}\right)\phantom{\rule{2.77695pt}{0ex}}i=1,...,j$. Since ${\sum}_{i=1}^{M}{P}_{{b}^{},b,i}={\sum}_{i=1}^{M}{P}_{{a}^{},a,i}=1$ we have ${\sum}_{i=1}^{i}{P}_{{a}^{},a,i}{P}_{{b}^{},b,i}={\sum}_{i=j+1}^{M}{P}_{{b}^{},b,i}{P}_{{a}^{},a,i}$ thus
where the second inequality is a result of Assumption 2. ■
References
 1.
Karmokar AK, Djonin DV, Bhargava VK: Optimal and suboptimal packet scheduling over timevarying fading channels. IEEE Trans Wirel Commun 2006, 5(2):446457.
 2.
Djonin DV, Krishnamurthy V: MIMO transmission control in fading channelsa constrained Markov decision process formulation with monotone randomized policies. IEEE Trans Signal Process 2007, 55(10):50695083.
 3.
Wang H, Mandayam NB: A simple packettransmission scheme for wireless data over fading channels. IEEE Trans Commun 2004, 52(7):10551059. 10.1109/TCOMM.2004.831354
 4.
UysalBiyikoglu E, Prabhakar B, Gamal AE: Energyefficient packet transmission over a wireless link. IEEE/ACM Trans Netw 2002, 10(4):487499. 10.1109/TNET.2002.801419
 5.
Wang H, Mandayam NB: Opportunistic file transfer over a fading channel under energy and delay constraints. IEEE Trans Commun 2005, 53(4):632. 10.1109/TCOMM.2005.844934
 6.
Berry R, Gallager R: Communication over fading channels with delay constraints. IEEE Trans Inf Theory 2002, 48(5):11351149. 10.1109/18.995554
 7.
Goyal M, Kumar A, Sharma V: Power constrained and delay optimal policies for scheduling transmission over a fading channel. In Proc INFOCOM. San francisco, USA; 2003:311320.
 8.
Rajan D, Subharwal A, Aazhang B: Delay and rate constrained transmission policies over wireless channels. Proc IEEE GLOBECOM Conference 2001, 806810.
 9.
IEEE: Wireless LAN medium access control (MAC) and physical layer (PHY) specifications. IEEE standard 802.11 2006.
 10.
IEEE: Wireless medium access control (MAC) and physical layer (PHY) specifications for lowrate wireless personal area networks (WPANs). In IEEE Std 802.15.4 Proceedings of ACM Sigmetrics. Seattle, WA, USA; 2006. S Rajagopalan, D Shah, J Shin, Network adiabatic theorem: an efficient randomized protocol for contention resolution 2009, pp. 133144
 11.
Jiang L, Leconte M, Ni J, Srikant R, Walrand J: Fast mixing of parallel Glauber dynamics and lowdelay CSMA scheduling. 2010.
 12.
Barcelo J, Bellalta B, Cano C, Sfairopoulou A, Oliver M, Verma K: Towards a collisionfree WLAN: dynamic parameter adjustment in CSMA/E2CA. EURASIP J Wirel Commun Netw 2011. doi:10.1155/2011/708617
 13.
Kar K, Sarkar S, Tassiulas L: Achieving proportional fairness using local information in Aloha networks. IEEE Trans Autom Control 2004, 49(10):18581862. 10.1109/TAC.2004.835596
 14.
MohsenianRad AH, Huang J, Chiang M, Wong VWS: Utilityoptimal random access: optimal performance without frequent explicit message passing. IEEE Trans Wirel Commun 2009, 8(2):898911.
 15.
Wang X, Kar K: Crosslayer rate control in multihop wireless networks with random access. IEEE J Sel Areas Commun 2006, 24(8):15481559.
 16.
Khodaian M, Khalaj BH: Delay constrained utility maximization in multihop random access networks. IET Commun 2010, 4(16):19081918. 10.1049/ietcom.2009.0622
 17.
Liu J, Stoylar A, Chiang M, Poor HV: Queue based random access in wireless networks: optimality and stability. IEEE Trans Inf Theory 2009, 55(9):40874098.
 18.
Warrier A, Janakiraman S, Ha S, Rhee I: DiffQ: practical differential backlog congestion control for wireless networks. In Proceedings of IEEE INFOCOM. Rio de Janeiro, Brazil; 2009:262270.
 19.
Nardelli B, Lee J, Lee K, Yi Y, Chong S, Knightly E, Chiang M: Experimental evaluation of optimal CSMA. In Proceedings of IEEE INFOCOM. Shanghai, China; 2011:11881196.
 20.
Sadeghi P, Kennedy RA, Rapajic PB, Shams R: Finite state Markov modeling of fading channels. IEEE Signal Process Mag 2008, 57: 5780.
 21.
Ni J, Tan B, Srikant R: QCSMA: queuelength based CSMA/CA algorithms for achieving maximum throughput and low delay in wireless networks. In Proceedings of IEEE INFOCOM MiniConference. San Diego, CA, USA; 2010:15.
 22.
Bharghavan V, Demers A, Shenker S, Zhang L: MACAW: a media access protocol for wireless LAN's. In Proceedings of ACM SIGCOMM. London, UK; 1994:212225.
 23.
Bianchi G: Performance analysis of the IEEE 802.11 distributed coordination function. IEEE J Sel Areas Commun 2000, 18(3):535547. 10.1109/49.840210
 24.
Vandalore B, Feng W, Jain R, Fahmy S: A survey of application layer techniques for adaptive streaming of multimedia. RealTime Imag 2001, 7(3):221235. 10.1006/rtim.2001.0224
 25.
Floyd S, Jacobson V: Random early detection gateways for congestion avoidance. IEEE/ACM Trans Netw 1993, 1(4):397413. 10.1109/90.251892
 26.
ONeill D, Akuiyibo E, Boyd SP, Goldsmith AJ: Optimizing adaptive modulation in wireless networks via multiperiod network utility maximization. In IEEE International Conference on Communications. Cape Town, South Africa; 2010:15.
 27.
Akuiyibo E, Boyd SP: Adaptive modulation with smoothed flow utility. EURASIP J Wirel Commun Netw 2010. doi:10.1155/2010/815213
 28.
Montgomery DC: Introduction to Statistical Quality Control. 3rd edition. John Wiley & Sons, New York; 1996.
 29.
Wang HS, Moayeri N: Finitestate markov channela useful model for radio communication channels. IEEE Trans Veh Technol 1995, 44: 163171. 10.1109/25.350282
 30.
Bertsekas DP: Dynamic Programming and Optimal Control. Volume I. 3rd edition. Athena Scientific, Belmont; 2005.
 31.
Bertsekas DP: Convergence of discretization procedures in dynamic programming. IEEE Trans Autom Control 1975, 20: 415419. 10.1109/TAC.1975.1100984
 32.
Reichl P, Tuffin B, Schatz R: Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience. Telecommun Syst 2011., 47: doi:10.1007/s1123501195037
 33.
Bertsekas D, Gallager R: Data Networks. 2nd edition. Prentice Hall, Englewood Cliffs, NJ; 1992.
Acknowledgements
This study was supported in part by the Spanish Government, Ministerio de Ciencia e Innovación (MICINN), under projects COMONSENS (CSD200800010, CONSOLIDERINGENIO 2010 program) and COSIMA (TEC201019545C0403), in part by Iran Telecommunication Research Center under contract 6947/500, and in part by Iran National Science Foundation under grant number 87041174. This study was completed while M. Khodaian was at CEIT and TECNUN (University of Navarra).
Author information
Additional information
Competing interests
The authors declare that they have no competing interests
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
About this article
Received
Accepted
Published
DOI
Keywords
 adaptive control
 dynamic programming
 wireless channel
 CSMA