 Research
 Open Access
Adaptive access and rate control of CSMA for energy, rate, and delay optimization
 Mahdi Khodaian^{1},
 Jesús Pérez^{2}Email author,
 Babak H Khalaj^{1} and
 Pedro M Crespo^{3}
https://doi.org/10.1186/16871499201227
© Khodaian et al; licensee Springer. 2012
 Received: 8 September 2011
 Accepted: 30 January 2012
 Published: 30 January 2012
Abstract
In this article, we present a crosslayer adaptive algorithm that dynamically maximizes the average utility function. A per stage utility function is defined for each link of a carrier sense multiple accessbased wireless network as a weighted concave function of energy consumption, smoothed rate, and smoothed queue size. Hence, by selecting weights we can control the tradeoff among them. Using dynamic programming, the utility function is maximized by dynamically adapting channel access, modulation, and coding according to the queue size and quality of the timevarying channel. We show that the optimal transmission policy has a threshold structure versus the channel state where the optimal decision is to transmit when the wireless channel state is better than a threshold. We also provide a queue management scheme where arrival rate is controlled based on the link state. Numerical results show characteristics of the proposed adaptation scheme and highlight the tradeoff among energy consumption, smoothed data rate, and link delay.
Keywords
 adaptive control
 dynamic programming
 wireless channel
 CSMA
1. Introduction
In wireless networks, mobile devices are usually battery powered with a limited amount of energy. Therefore, minimization of energy consumption while maintaining the quality of service in the network is crucial. This must be accomplished by adapting the transmission parameters to the system dynamics and to the timevarying channel of the links. In this article, we present a crosslayer adaptive algorithm that dynamically maximizes the average utility function of a carrier sense multiple access (CSMA)based wireless link.
Benefits of such adaptation schemes are shown in some prior works in terms of energy efficiency [1–8]. In such works various control algorithms have been proposed that tradeoff among different goals such as energy consumption, average delay, packet dropping probability and bit error rate, and dynamically adapt the transmission parameters to the channel and system state. The aforementioned works assume pointtopoint links with dedicated channels. However, in data transmission networks, where data are generated at random time instances, random access schemes are used to efficiently exploit channel resources. In such systems, there are more users than available channels, and at any given time only a subset of users can access the channels. Therefore, the optimality of channel access decision is crucial in random access networks. Random access is widely used in ad hoc networks as it can be implemented in a distributed manner. Wireless local area networks (WLAN) and practical personal or sensor networks usually use random access control in their ad hoc operation mode [9, 10]. On the other hand, it is shown recently that CSMA protocols can achieve maximum stable throughput [11] while keeping bounded queuing delay [12], and it can achieve a collision free WLAN [13].
Optimization of random access networks was first proposed in order to achieve single hop proportional fairness for slotted ALOHA networks [14]. Different types of fairness are also considered and random access control is modeled as a utility maximization problem in [15]. In addition, the crosslayer optimization problem of random access control and transmission control protocol is solved as a network utility maximization problem [16]. Newtonlike algorithms are also provided for energy and throughput optimization with endtoend delay constraint in multi hop random access network [17]. However, in the aforementioned articles static transmission probability was used and opportunity of time varying and adaptive control was ignored.
On the other hand, queuebased random access algorithms were studied in [18], where access probabilities are assumed to be adapted based on queue sizes. Stability of the proposed algorithms was verified and their delay performance was shown to surpass fixed optimization algorithms. Also a heuristic differential queuebased scheduling algorithm is proposed in [19] which shows superior performance compared to 802.11 through experimental results. However, such queuebased algorithms are inappropriate for fading channels and prioritize links with low channel quality, which results in low energy efficiency [20].
In this article, we propose crosslayer adaptive algorithms; derived from dynamic programming, for distributed optimization of the links in CSMAbased wireless networks operating in mobile environments. As a performance metric, we define the per stage utility of the link as a weighted concave function of energy consumption, smoothed data rate, and smoothed queue size in the link, where the weights are assigned based on the desired tradeoff among them. The algorithms maximize the average utility by dynamically adapting the channel access decision and transmit data rate (by selecting different modulation and coding schemes) according to the queue size of the link and the availability and quality of the timevarying channel (channel state is assumed to be known at the transmitter). Both, finitetime horizon (FTH) and infinitetime horizon (ITH) problems are considered. In the first case, the utility sum is maximized for a finite time period, whereas in the second case, the longterm average utility is maximized.
We consider a mobile environment with frequencyflat timevarying channel response. This requires suitable models of the wireless channel dynamics. Here, we use finitestate Markov chains (FSMC) to model channel dynamics, such that channel timecorrelation at network links is partially exploited by the proposed algorithms. Although the physical wireless channel is inherently nonMarkovian, it has been shown that stationary Markov chains can capture the essence of the channel dynamics [21]. Many transmission adaptation algorithms are based on firstorder Markov channel models [1, 2]. Here, we consider first and secondorder Markov chains to model characteristics of network links.
The numerical simulations show the benefits of the proposed adaptation algorithms in terms of energy efficiency, and highlight the tradeoff among energy consumption, smoothed data rate, and delay in links of a CSMA network. They also show that the use of suitable Markov model for the wireless channel improves performance of the adaptation algorithm, mainly for slow fading channels. Algorithms based on uncorrelated, first and secondorder Markov models are considered and their performance is compared through simulations.
The rest of the article is organized as follows. Section 2 presents the system model and in particular it describes the model of the network links as well as wireless channel models. In Section 3, per stage utility of the links is defined. Consequently, the utility sum maximization for a finite time period is formulated as an optimal finitehorizon control problem. Similarly, the longterm average utility maximization is formulated as an optimal infinitehorizon control problem. Section 4 uses dynamic programming to compute the optimal adaptation policies for the problems formulated in Section 3. We have investigated structural properties of the optimal solution in Section 5. Numerical results and comparisons are described in Section 6. Finally, Section 7 concludes the article.
2. System model
In this section, we describe the model of the random access links as well as wireless channel models.
2.1. Link model
Timeslot k is defined as the time interval [(k  1)T_{s}, kT_{s}). We use I_{ k } to denote the channel access, where I_{ k } = 1 indicates that the link has decided to access the channel at the k th timeslot. The control policy adapts I_{ k } in each slot based on the system and channel state. Also B_{ k } = 1 indicates that the link should delay its transmission because the channel is already occupied by another link. We model B_{ k } as a Bernoulli process where P_{ B } Pr{B_{ k } = 1} is the channel occupancy probability. The Bernoulli distribution is widely used to model the statistics of B_{ k } in CSMA networks [24].
Where C_{ k } indicates the maximum number of packets that can be transmitted during the k th data slot. C_{ k } depends on the channel state, and it is assumed to be known at the transmitter at the beginning of each timeslot. We call the data that the physical layer transmits in one time slot a frame and the link consumes a constant energy e for transmission of frame in the data slot. Thus, the energy consumed in the k th timeslot will be E_{ k } = eI_{ k } (1  B_{ k } ).
Note that ${\stackrel{\u0304}{q}}_{k}$ and ${\stackrel{\u0304}{r}}_{k}$ can be viewed as "smoothed" measures of the delay and data rate in the link. The parameters θ_{ q } and θ_{ r } determines the time scale over which the smoothing is performed. The smaller the value of θ_{ r } or θ_{ q } , the shorter the time period of moving average (smoothing). Values of θ_{ r } and θ_{ q } are determined based on the tolerance of the applications to the delay and data rate variations in the link. Random early detection protocol has used the EWMA of the delay (${\stackrel{\u0304}{q}}_{k}$) as a criterion for congestion control [26]. In addition, the EWMA of the rate (or smoothed rate), ${\stackrel{\u0304}{r}}_{k}$, has been used in [27, 28] as a measure of the quality of service. EWMA is also used as a metric in statistical quality control [29].
2.2. Channel model
We consider a frequencyflat blockfading channel, where the channel remains constant during each timeslot, and can change for consecutive timeslots. Therefore, we assume that the duration of each timeslot (T_{ s } ) is less than the coherence time of the channel. Hence, channel responses at different timeslots can be correlated. The channel power gain at the k th timeslot is denoted by γ_{ k } . Since we assume constant transmit power, the received signaltonoise ratio (SNR) in the link for the k th timeslot will be proportional to γ_{ k } . The fading range 0 ≤ γ is partitioned into M disjoint regions so that the j th region is defined as R_{ j } = {γ : A_{ j } ≤ γ < A_{j+1}}, where A_{1} = 0 and A_{M+1}= ∞. The channel for the k th timeslot is in state j if γ_{ k } ∈ R_{ j } . Also the values of A_{ j } are selected according to the adaptive modulation and coding as follows. Consider that transmitter has a set of modulation and coding schemes {Q_{1}, Q_{2}, ... , Q_{ M } } to select from in each time slot. We select A_{ j } ; j = 2, ... , M such that if channel is in state j, transmitter can use Q_{ j } and ensures that the frames transmitted with this scheme have error probability less than FER_{th} which is a target threshold for frame error rate (FER).
Subsequently, we consider three models for the random process C_{ k } , with diverse degrees of complexity.
1. Uncorrelated model
In this model, the channel response at different timeslots are assumed uncorrelated so, where P_{ j } is the probability of the channel state R_{ j } . This simple model may be accurate for fading channels that exhibit high timevariability. It is also the fitting model when there is no prior information about the channel time correlation.
2. Firstorder markov model
To model the time correlation of the channel we use an M state FSMC [30] with time discretized to T_{s} and transition probabilities as
The transition probabilities depend on the normalized Doppler frequency f_{d}T_{s} which determines the rate of variation of the channel with respect to the timeslot duration, where f_{d} is the channel Doppler frequency. Although the physical wireless channel is inherently nonMarkovian, it has been shown that an FSMC can capture the essence of the channel dynamics when the number of regions/states (M) is low and the channel fades slow enough (see for example [21] and references therein). Note that the uncorrelated model can be viewed as a particular case of FSMC where P_{ i,j } = P_{ j } , ∀i.
3. Secondorder Markov model
In this article, we use the socalled Cartesian product method [21] for the secondorder models. We will investigate the effect of the FSMC order on the performance of the resulting algorithm through numerical results. Note that the formulation of the firstorder Markov model can be considered as a special case of the secondorder model with P_{ i,j } = P_{ l,i,j } for any l.
3. Problem formulation
We consider a wireless link in a CSMA network which desires to optimize its transmission rate, energy consumption, and delay. We distinguish two dynamic optimization problems: FTH and ITH problems. In the FTH problem, the performance of the link is optimized over a finite number of timeslots, whereas in the ITH problem the link performance is optimized considering an infinite number of timeslots. Next, they are formulated as dynamic programming problems.
3.1. Finite time horizon
where the expectation is taken over the random process C_{ k } . The function g(s_{ k } , μ_{ k } ) is the utility per stage and is a measure of the quality of service of the link at each timeslot. It depends on the action vector μ_{ k } = (I_{ k } , r_{ k } ) and on the system state vector. We consider a secondorder Markov model for C_{ k } and include component C_{k1}in the state vector ${s}_{k}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k},{C}_{k1}\right)$. Note that the firstorder model can be considered as a special case with ${s}_{k}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k}\right)$. Considering φ(·) as the state update function we can write: s_{k+1}= φ (s_{ k } , μ_{ k } , C_{k+1}, B_{ k } ). In (6) g_{N+1}is the final stage utility which depends only on the final state of the system, s_{N+1}and it can include some limitations or penalties on the final state of the system.
where U(·), and V(·) are suitable continuous, concave functions, and parameters α and β control the tradeoff between rate, energy, and delay in the utility function. A similar formulation for per stage utility is used in [27, 28] for multiperiod utility maximization while queue management and thus queue sizes were not considered.
3.2. Infinite time horizon
where the action and state vectors as well as the per stage utility function are defined similar to the FTH problem. We consider both the first and secondorder models for the channel state by applying appropriate format of s_{ k } .
4. Optimal adaptive control
To maximize FTH or ITH utility functions the controller should decide optimal actions ${\mu}_{k}^{*}\left({s}_{k}\right)$ at the beginning of each timeslot as a function of the system state s_{ k } . Note that the decision must be causal since future system states are unknown due to the randomness of the channel state (C_{ k } ) and occupancy (B_{ k } ). In this section, by using the DP algorithm [31], we derive algorithms that compute the optimal control functions for the FTH and ITH problems. It is important to remark that the resulting optimal control functions are computed and stored offline. Then, they will be used online to dynamically adapt the actions to the system state. As described earlier, the system state definition can support uncorrelated, first and secondorder channel models so we do not limit the solution to any specific channel model.
4.1. Per stage adaptation to maximize FTH utility
The function J_{ k } (s_{ k } ) is the maximum expected accumulative utility, achieved under optimal decision, when the system is in state s_{ k } at the k th stage. Thus, J_{1} (s_{1}) is the expected total utility for N stages when the initial state is s_{1}.
In the above equation, we have ${s}_{k+1}^{d}=\left({q}_{k+1},{\stackrel{\u0304}{q}}_{m}^{d},{\stackrel{\u0304}{r}}_{l}^{d},{C}_{k+1},{C}_{k}\right)$ and ${s}_{k+1}=\left({q}_{k+1},{\stackrel{\u0304}{q}}_{k+1},{\stackrel{\u0304}{r}}_{k+1},{C}_{k+1},{C}_{k}\right)$. Furthermore q_{k+1}, ${\stackrel{\u0304}{q}}_{k+1}$ and ${\stackrel{\u0304}{r}}_{k+1}$ are given by (1), (2), and (3), respectively. We consider the secondorder Markov model by using both C_{k1}and C_{ k } in the state vector. The other channel models can be considered as its special case. The solution provided in Equations (17)(19) is valid for any concave and continuous function of g.
Since ${\stackrel{\u0304}{r}}_{k}^{d}$ and ${\stackrel{\u0304}{q}}_{k}^{d}$ are independent of the decision in the k th timeslot, $\mathsf{\text{U}}\left({\stackrel{\u0304}{r}}_{k}^{d}\right)\alpha \mathsf{\text{V}}\left({\stackrel{\u0304}{q}}_{k}^{d}\right)$ do not affect the maximization in (21). Also the summations in (20) and (21) are over all M channel states and two possible channel occupancy conditions.
The discrete DP algorithm can be executed offline and the resulting optimal policy can be stored in a lookup table available at the transmitter. Then, it will be used online to dynamically adapt the action to the system state.
4.2. Statebased adaptive control to optimize average utility per stage
We denote the optimal policy as π* which produces the maximum average utility per stage J*. Both π* and J* are independent of the initial state since the influence of the utility of the early stages on the average utility reduces to 0 as N → ∞. Moreover, since the utility per stage, the transition probabilities (4), and the state update Equations (1)(3) are all stationary, the optimal policy will be stationary (does not change from stage to stage). Therefore, it is a single function, μ*(s), that maps the system states to actions regardless of the stage.
where s^{+} indicates the successor state of the current state s. Considering φ(·) as the state update function s^{+} = φ (s, μ, C^{+}, B). The expectation in Equation (23) is over the random processes {B_{ k } } and {C_{ k } }.
Therefore, we apply (27) and compute h^{(n)}(s^{ d } ) for all possible discrete states. For the uncorrelated and firstorder channel models there are (L × M_{ q } × M_{ r } × M) discrete states and for the secondorder channel model this number should be multiplied by M.
5. Structural properties of the optimal solution
In the previous section, we provided DP algorithms that can be applied to find optimal decisions through numerical calculations. In this section, we investigate some structural properties of the solution. We use the following practical assumptions throughout this section.
Assumption 1: Per stage utility function has a format of (7) and (8) with U(·), and V(·) as increasing functions.
where ${P}_{{a}^{},a,i}$ is the probability of going from channel states $\left({\u0108}_{{a}^{}},{\u0108}_{a}\right)$ to the next state ${C}^{+}={\u0108}_{i}$ as defined for second order Markov model.
Assumption 2 is valid in practice for Markov channels since $\left({\u0108}_{{a}^{}},\phantom{\rule{0.3em}{0ex}}{\u0108}_{a}\right)$ is supposed to be lower than and $\left({\u0108}_{{b}^{}},{\u0108}_{b}\right)$and each side of the inequality calculates the probability of going to the first j states with lowest rates. For example, if ${P}_{{b}^{},b,1}\le {P}_{{a}^{},a,1}$ then the inequality will be true for j = 1 for and assumption 2 holds. If the inequality turns out to be true for any value of j then the assumption is correct. Based on this assumption we provide the following lemma:
Proof is provided in the Appendix.
5.1. Structural properties of FTH solution
The following theorem indicates monotonicity of J_{ k } (s_{ k } ) versus the state variables.
Theorem 1: J_{ k } (s_{ k } ) is a decreasing function of q_{ k } and ${\stackrel{\u0304}{q}}_{k}$, and an increasing function of ${\stackrel{\u0304}{r}}_{k}$ and C_{ k } for all values of k.
Proof: In order to prove the theorem we show through induction that for k = N + 1, ..., 1 we have J_{ k } (s_{ k } + Δ) ≤ J_{ k } (s_{ k } ) for any vector Δ that increase q_{ k } and ${\stackrel{\u0304}{q}}_{k}$, and decrease ${\stackrel{\u0304}{r}}_{k}$ and C_{ k } .
Thus, ${J}_{k}\left({s}_{k}\right)={G}_{k}\left({s}_{k},{\mu}_{k}^{*}\right)$ where ${\mu}_{k}^{*}$ is optimal decision for state s_{ k } . Also we define Δ = (δ_{1}, δ_{2},  δ_{3}, δ_{4}, δ_{5}) for any value of δ_{ i } ≥ 0, i = 1, ... , 5 such that ${s}_{k}+\mathrm{\Delta}=\left({q}_{k}+{\delta}_{1},{\stackrel{\u0304}{q}}_{k}+{\delta}_{2},{\stackrel{\u0304}{r}}_{k}{\delta}_{3},{C}_{k}{\delta}_{4},{C}_{k1}{\delta}_{5}\right)$ is an element of the state space.
For k = N + 1 we have ${J}_{N+1}\left({s}_{N+1}\right)=U\left({\stackrel{\u0304}{r}}_{N+1}\right)\gamma \mathsf{\text{V(}}{\stackrel{\u0304}{q}}_{N+1}\mathsf{\text{)}}\eta {q}_{N+1}$ and using assumption 1 it is clear that J_{N+1}(s_{N+1}+ Δ) ≤ J_{N+1}(s_{N+1}). Assuming J_{k+1}(s_{k+1}) is a monotonic function we show J_{ k } (s_{ k } ) is also monotonic for k = N, ... , 1 which completes the proof.
We consider φ (·) as the state update function and define two possible next states ${s}_{k+1,\mathrm{\Delta}}^{*}=\phi \left({s}_{k}+\mathrm{\Delta},{\mu}_{k,\mathrm{\Delta}}^{*},{C}_{k+1},{B}_{k}\right)$ and ${s}_{k+1}^{\#}=\phi \left({s}_{k},{\mu}_{k,\mathrm{\Delta}}^{*},{C}_{k+1},{B}_{k}\right)$. For known values of C_{k+1}and B_{ k } we can use (1)(3) and easily show that ${s}_{k+1,}^{{*}_{\mathrm{\Delta}}}={s}_{k+1}^{\#}+{\mathrm{\Delta}}^{\prime}$ in which ${\mathrm{\Delta}}^{\prime}=\left({\delta}_{1}^{\prime},{\delta}_{2}^{\prime},{\delta}_{3}^{\prime},{\delta}_{4}^{\prime},{\delta}_{5}^{\prime}\right)$ for some ${\delta}_{i}^{\prime}\ge 0$. Thus ${J}_{k+1}\left({s}_{k+1,\mathrm{\Delta}}^{*}\right)\le {J}_{k+1}\left({s}_{k+1}^{\#}\right)$.
Equation (30) together with (33) prove the theorem by showing: J_{ k } (s_{ k } + Δ) ≤ J_{ k } (s_{ k } ).
■
Assuming uncorrelated channel model the following theorem indicates the "threshold structure" of the optimal transmission policy versus the channel state.
Theorem 2: If the optimal access decision in state ${s}_{k}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k}\right)$ is ${I}_{k}^{*}\left({s}_{k}\right)=1$, then for another possible state ${s}_{k}^{\prime}=\left({q}_{k},{\stackrel{\u0304}{q}}_{k},{\stackrel{\u0304}{r}}_{k},{C}_{k}^{\prime}\right)$ in the same slot with improved channel state ${C}_{k}^{\prime}\ge {C}_{k}$ we have ${I}_{k}^{*}\left({s}_{k}^{\prime}\right)=1$.
which is in contrast to optimality of the ${\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)=\left({I}_{k}^{\prime}=0,{r}_{k}^{\prime}\right)$. Thus, we should have ${\mu}_{k}^{*}\left({s}_{k}^{\prime}\right)=\left({I}_{k}^{\prime}=1,{r}_{k}^{\prime}\right)$.
■
Note that Theorem 2 may be incorrect when channel state is time correlated. For example, consider two possible channel states C_{ k } and ${C}_{k}^{\prime}$, with ${C}_{k}<{C}_{k}^{\prime}$ and assume that optimal decision is to transmit for a state with C_{ k } . Also assume that probability of going from ${C}_{k}^{\prime}$ to a better channel state and from C_{ k } to a worse channel state is high. So, we can argue heuristically that in this condition it may be optimal to transmit data when channel is in state C_{ k } but not to transmit when it is in state ${C}_{k}^{\prime}$.
5.2. Structural properties of ITH solution
We provide structural properties of ITH solution in this section through the following theorems. First we show that relative value function, h*(s), is a monotonic function in Theorem 3 and then prove the threshold structure of access decision versus channel state in Theorem 4.
Theorem 3: h*(s), is a decreasing function of q and $\stackrel{\u0304}{q}$, and an increasing function of $\stackrel{\u0304}{r}$ and C.
Combining (38) and (40) we find that B _{ τ }h^{(n)}(s + Δ) ≤ B _{ τ }h^{(n)}(s). Using Equation (25) and taking into account that B _{ τ }h^{(n)}(s') is independent of the state vector, it can be easily shown that h^{(n + 1)}(s) is a monotonic function.
■
Assuming uncorrelated channel model the following theorem indicates existence of a threshold for channel state that the link should decide to transmit when channel state is better than or equal to that threshold.
Theorem 4: There exists a threshold, C_{th}, that for ${s}_{\mathsf{\text{th}}}=\left(q={C}_{\mathsf{\text{th}}},\stackrel{\u0304}{q},\stackrel{\u0304}{r},{C}_{\mathsf{\text{th}}}\right)$ with any $\stackrel{\u0304}{q}$ and $\stackrel{\u0304}{r}$, we have I*(s_{ th } ) = 1. Also for any s with C ≥ C_{th} and q ≥ C_{th} we have I*(s) = 1.
Proof: Assume in timeslot k we have C_{ k } = C^{max} and q_{ k } = C^{max}, transmission at this time has the energy cost of β e but it will reduce q by C^{max} which will reduce $\stackrel{\u0304}{q}$ by θ_{ q }C^{max} and also will reduce the future costs related to the queue size. However, transmission of theses C^{max} packets at any later time slot requires the same amount of energy. Thus, it is better to transmit these packets at state s_{th} to reduce the queue size as early as possible and reduce the future costs related to the queue size. We conclude that: "if C = C^{max} and q = C^{max} then I*(s) = 1" which proves existence of C_{th}.
In order to prove the second part of the theorem we assume $s=\left(q\ge {C}_{\mathsf{\text{th}}},\stackrel{\u0304}{q},\stackrel{\u0304}{r},C\ge {C}_{\mathsf{\text{th}}}\right)$ and consider optimal decisions μ*(s_{th}) = (I*(s_{th}), r*(s_{th})) and μ*(s) = (I*(s), r*(s)) for states s_{th} and , respectively. If I(s) = 0 we can show similar to the proof of Theorem 2 that it cannot be an optimal transmission policy.
■
6. Numerical results
For numerical analysis of the adaptive control algorithms provided in Section 4 we consider a lightweight sensor in a wireless network that may transmit its status using few bits. In each timeslot the sensor may send its own packet or forward packets of other sensors. We assume a Rayleigh flat fading channel, and use a set of simple Modulation and Coding schemes. Note that our adaptive algorithm only requires the FSMC model which can be found for many practical fading channels [21] and do not depend on Rayleigh fading assumption or Modulation schemes. However, in this section we consider the following types of modulations joint with ReedSolomon (RS) coding:

Q_{1}: No transmission since link is in deep fade.

Q_{2}: BPSK with RS (63,47).

Q_{3}: QPSK with RS (127,94).

Q_{4}: 16QAM with RS (255, 188).
As described in Section 4, continuous state variables, ${\stackrel{\u0304}{r}}_{k}$ and ${\stackrel{\u0304}{q}}_{k}$, should be discretized in order to achieve a finite state system and dynamic programming solution. We set M_{ r } = 21, and M_{ q } = 13 for discretization. It is shown in our simulations that enhancement achieved by selecting greater values for M_{ r } , and M_{ q } is insignificant. Also the maximum queue size is assumed to be L = 12 and the number of arrival packets at each stage r_{ k } is limited by 4.
6.1. FTH results
As indicated earlier for FTH problem we use (8) as the final stage utility function with η = 5 as the price for packets remaining in the queue where $U\left({\stackrel{\u0304}{r}}_{k}\right)=\mathrm{log}\left(\epsilon +{\stackrel{\u0304}{r}}_{k}\right)$, and $\mathsf{\text{V}}\left({\stackrel{\u0304}{q}}_{k}\right)={\left({\stackrel{\u0304}{q}}_{k}\right)}^{2}$. We assume that the initial state of the link is ${\stackrel{\u0304}{r}}_{1}=0.1$, ${\stackrel{\u0304}{q}}_{1}={q}_{1}=0$ and consider a flat fading Rayleigh channel. The recursive algorithm (16)(18) is used to obtain the optimal control policy (over the discretized state space) for different values of N. The transition probabilities of the Markov models are computed as described earlier.
Performance versus discretization grid size
M _{ q }  M _{ r }  J_{1}(s_{1})/N 

4  5  2.3 
6  10  1.39 
13  21  1.213 
13  30  1.203 
20  30  1.201 
6.2. ITH results
7. Conclusions
We addressed the problem of optimal channel access and rate adaptation in the links of CSMA wireless networks. We defined a utility function that trades off the energy consumption and the average packet transmission rate and delay. By using dynamic programming, we derive algorithms and optimal policies that maximize the average utility by adapting the arrival packet rate and channel access as functions of the queue occupancy, channel state, and smoothed rate. The optimal policies can be computed and stored offline. Then, they can be used online for dynamic access control and queue management of the link. The proposed algorithms exploit the time correlation of the channel by means of different FSMC models. Both FTH and ITH problems were addressed. In the first case, the average utility is optimized for a finite time period, whereas in the second case, the longterm average utility is maximized. Structural properties of the optimal solution are investigated and it is shown that optimal transmission policy has a threshold structure versus the channel state. For the ITH problem we proved the existence of a channel state that the link should always transmit when the channel is in that state or in a better one. Numerical results show that the overall performance of the link can be enhanced by increasing the order of the FSMC channel model. However, it increases the complexity of the algorithms and the memory required to store the optimal policies.
Appendix
Proof of Lemma 1
where the second inequality is a result of Assumption 2. ■
Declarations
Acknowledgements
This study was supported in part by the Spanish Government, Ministerio de Ciencia e Innovación (MICINN), under projects COMONSENS (CSD200800010, CONSOLIDERINGENIO 2010 program) and COSIMA (TEC201019545C0403), in part by Iran Telecommunication Research Center under contract 6947/500, and in part by Iran National Science Foundation under grant number 87041174. This study was completed while M. Khodaian was at CEIT and TECNUN (University of Navarra).
Authors’ Affiliations
References
 Karmokar AK, Djonin DV, Bhargava VK: Optimal and suboptimal packet scheduling over timevarying fading channels. IEEE Trans Wirel Commun 2006, 5(2):446457.View ArticleGoogle Scholar
 Djonin DV, Krishnamurthy V: MIMO transmission control in fading channelsa constrained Markov decision process formulation with monotone randomized policies. IEEE Trans Signal Process 2007, 55(10):50695083.MathSciNetView ArticleGoogle Scholar
 Wang H, Mandayam NB: A simple packettransmission scheme for wireless data over fading channels. IEEE Trans Commun 2004, 52(7):10551059. 10.1109/TCOMM.2004.831354View ArticleGoogle Scholar
 UysalBiyikoglu E, Prabhakar B, Gamal AE: Energyefficient packet transmission over a wireless link. IEEE/ACM Trans Netw 2002, 10(4):487499. 10.1109/TNET.2002.801419View ArticleGoogle Scholar
 Wang H, Mandayam NB: Opportunistic file transfer over a fading channel under energy and delay constraints. IEEE Trans Commun 2005, 53(4):632. 10.1109/TCOMM.2005.844934View ArticleGoogle Scholar
 Berry R, Gallager R: Communication over fading channels with delay constraints. IEEE Trans Inf Theory 2002, 48(5):11351149. 10.1109/18.995554MathSciNetView ArticleGoogle Scholar
 Goyal M, Kumar A, Sharma V: Power constrained and delay optimal policies for scheduling transmission over a fading channel. In Proc INFOCOM. San francisco, USA; 2003:311320.Google Scholar
 Rajan D, Subharwal A, Aazhang B: Delay and rate constrained transmission policies over wireless channels. Proc IEEE GLOBECOM Conference 2001, 806810.Google Scholar
 IEEE: Wireless LAN medium access control (MAC) and physical layer (PHY) specifications. IEEE standard 802.11 2006.Google Scholar
 IEEE: Wireless medium access control (MAC) and physical layer (PHY) specifications for lowrate wireless personal area networks (WPANs). In IEEE Std 802.15.4 Proceedings of ACM Sigmetrics. Seattle, WA, USA; 2006. S Rajagopalan, D Shah, J Shin, Network adiabatic theorem: an efficient randomized protocol for contention resolution 2009, pp. 133144Google Scholar
 Jiang L, Leconte M, Ni J, Srikant R, Walrand J: Fast mixing of parallel Glauber dynamics and lowdelay CSMA scheduling. 2010.Google Scholar
 Barcelo J, Bellalta B, Cano C, Sfairopoulou A, Oliver M, Verma K: Towards a collisionfree WLAN: dynamic parameter adjustment in CSMA/E2CA. EURASIP J Wirel Commun Netw 2011. doi:10.1155/2011/708617Google Scholar
 Kar K, Sarkar S, Tassiulas L: Achieving proportional fairness using local information in Aloha networks. IEEE Trans Autom Control 2004, 49(10):18581862. 10.1109/TAC.2004.835596MathSciNetView ArticleGoogle Scholar
 MohsenianRad AH, Huang J, Chiang M, Wong VWS: Utilityoptimal random access: optimal performance without frequent explicit message passing. IEEE Trans Wirel Commun 2009, 8(2):898911.View ArticleGoogle Scholar
 Wang X, Kar K: Crosslayer rate control in multihop wireless networks with random access. IEEE J Sel Areas Commun 2006, 24(8):15481559.View ArticleGoogle Scholar
 Khodaian M, Khalaj BH: Delay constrained utility maximization in multihop random access networks. IET Commun 2010, 4(16):19081918. 10.1049/ietcom.2009.0622MathSciNetView ArticleGoogle Scholar
 Liu J, Stoylar A, Chiang M, Poor HV: Queue based random access in wireless networks: optimality and stability. IEEE Trans Inf Theory 2009, 55(9):40874098.View ArticleGoogle Scholar
 Warrier A, Janakiraman S, Ha S, Rhee I: DiffQ: practical differential backlog congestion control for wireless networks. In Proceedings of IEEE INFOCOM. Rio de Janeiro, Brazil; 2009:262270.Google Scholar
 Nardelli B, Lee J, Lee K, Yi Y, Chong S, Knightly E, Chiang M: Experimental evaluation of optimal CSMA. In Proceedings of IEEE INFOCOM. Shanghai, China; 2011:11881196.Google Scholar
 Sadeghi P, Kennedy RA, Rapajic PB, Shams R: Finite state Markov modeling of fading channels. IEEE Signal Process Mag 2008, 57: 5780.View ArticleGoogle Scholar
 Ni J, Tan B, Srikant R: QCSMA: queuelength based CSMA/CA algorithms for achieving maximum throughput and low delay in wireless networks. In Proceedings of IEEE INFOCOM MiniConference. San Diego, CA, USA; 2010:15.Google Scholar
 Bharghavan V, Demers A, Shenker S, Zhang L: MACAW: a media access protocol for wireless LAN's. In Proceedings of ACM SIGCOMM. London, UK; 1994:212225.Google Scholar
 Bianchi G: Performance analysis of the IEEE 802.11 distributed coordination function. IEEE J Sel Areas Commun 2000, 18(3):535547. 10.1109/49.840210View ArticleGoogle Scholar
 Vandalore B, Feng W, Jain R, Fahmy S: A survey of application layer techniques for adaptive streaming of multimedia. RealTime Imag 2001, 7(3):221235. 10.1006/rtim.2001.0224View ArticleGoogle Scholar
 Floyd S, Jacobson V: Random early detection gateways for congestion avoidance. IEEE/ACM Trans Netw 1993, 1(4):397413. 10.1109/90.251892View ArticleGoogle Scholar
 ONeill D, Akuiyibo E, Boyd SP, Goldsmith AJ: Optimizing adaptive modulation in wireless networks via multiperiod network utility maximization. In IEEE International Conference on Communications. Cape Town, South Africa; 2010:15.Google Scholar
 Akuiyibo E, Boyd SP: Adaptive modulation with smoothed flow utility. EURASIP J Wirel Commun Netw 2010. doi:10.1155/2010/815213Google Scholar
 Montgomery DC: Introduction to Statistical Quality Control. 3rd edition. John Wiley & Sons, New York; 1996.Google Scholar
 Wang HS, Moayeri N: Finitestate markov channela useful model for radio communication channels. IEEE Trans Veh Technol 1995, 44: 163171. 10.1109/25.350282View ArticleGoogle Scholar
 Bertsekas DP: Dynamic Programming and Optimal Control. Volume I. 3rd edition. Athena Scientific, Belmont; 2005.Google Scholar
 Bertsekas DP: Convergence of discretization procedures in dynamic programming. IEEE Trans Autom Control 1975, 20: 415419. 10.1109/TAC.1975.1100984MathSciNetView ArticleGoogle Scholar
 Reichl P, Tuffin B, Schatz R: Logarithmic laws in service quality perception: where microeconomics meets psychophysics and quality of experience. Telecommun Syst 2011., 47: doi:10.1007/s1123501195037Google Scholar
 Bertsekas D, Gallager R: Data Networks. 2nd edition. Prentice Hall, Englewood Cliffs, NJ; 1992.Google Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.