Adaptive access and rate control of CSMA for energy, rate, and delay optimization

2.1. Link model

We consider an ad hoc network where links use CSMA protocol similar to the one provided in [22] which prevents collision among links and also resolves hidden and exposed node problems which exist in wireless networks [23]. As shown in Figure 1, we assume a slotted transmission model where each timeslot, of duration T_s, contains both a data slot and a number of control mini slots. When the link has a packet to transmit, it should wait for a random value of W control mini-slots, and if no other link has reserved the channel earlier, it will send a short request to send packet to reserve the channel. Then, the potential receiver which also perceives that the channel is idle will response with a clear to send (CTS) packet that allows the transmitter to transmit and informs possible interfering nodes that the channel will be used. Once the transmitter receives the CTS, it sends its packet in the data slot.

Timeslot k is defined as the time interval [(k - 1)T_s, kT_s). We use I _k to denote the channel access, where I _k = 1 indicates that the link has decided to access the channel at the k th timeslot. The control policy adapts I _k in each slot based on the system and channel state. Also B _k = 1 indicates that the link should delay its transmission because the channel is already occupied by another link. We model B _k as a Bernoulli process where P _B Pr{B _k = 1} is the channel occupancy probability. The Bernoulli distribution is widely used to model the statistics of B _k in CSMA networks [24].

Where C _k indicates the maximum number of packets that can be transmitted during the k th data slot. C _k depends on the channel state, and it is assumed to be known at the transmitter at the beginning of each timeslot. We call the data that the physical layer transmits in one time slot a frame and the link consumes a constant energy e for transmission of frame in the data slot. Thus, the energy consumed in the k th timeslot will be E _k = eI _k (1 - B _k ).

Note that ${\bar{q}}_k$ and ${\bar{r}}_k$ can be viewed as "smoothed" measures of the delay and data rate in the link. The parameters θ _q and θ _r determines the time scale over which the smoothing is performed. The smaller the value of θ _r or θ _q , the shorter the time period of moving average (smoothing). Values of θ _r and θ _q are determined based on the tolerance of the applications to the delay and data rate variations in the link. Random early detection protocol has used the EWMA of the delay (${\bar{q}}_k$) as a criterion for congestion control [26]. In addition, the EWMA of the rate (or smoothed rate), ${\bar{r}}_k$, has been used in [27, 28] as a measure of the quality of service. EWMA is also used as a metric in statistical quality control [29].

2.2. Channel model

We consider a frequency-flat block-fading channel, where the channel remains constant during each timeslot, and can change for consecutive timeslots. Therefore, we assume that the duration of each timeslot (T _s ) is less than the coherence time of the channel. Hence, channel responses at different timeslots can be correlated. The channel power gain at the k th timeslot is denoted by γ _k . Since we assume constant transmit power, the received signal-to-noise ratio (SNR) in the link for the k th timeslot will be proportional to γ _k . The fading range 0 ≤ γ is partitioned into M disjoint regions so that the j th region is defined as R _j = {γ : A _j ≤ γ < A_j+1}, where A₁ = 0 and A_M+1= ∞. The channel for the k th timeslot is in state j if γ _k ∈ R _j . Also the values of A _j are selected according to the adaptive modulation and coding as follows. Consider that transmitter has a set of modulation and coding schemes {Q₁, Q₂, ... , Q _M } to select from in each time slot. We select A _j ; j = 2, ... , M such that if channel is in state j, transmitter can use Q _j and ensures that the frames transmitted with this scheme have error probability less than FER_th which is a target threshold for frame error rate (FER).

Subsequently, we consider three models for the random process C _k , with diverse degrees of complexity.

3.1. Finite time horizon

where the expectation is taken over the random process C _k . The function g(s _k , μ _k ) is the utility per stage and is a measure of the quality of service of the link at each timeslot. It depends on the action vector μ _k = (I _k , r _k ) and on the system state vector. We consider a second-order Markov model for C _k and include component C_k-1in the state vector $s_k=\left(q_k,{\bar{q}}_k,{\bar{r}}_k,C_k,C_{k-1}\right)$. Note that the first-order model can be considered as a special case with $s_k=\left(q_k,{\bar{q}}_k,{\bar{r}}_k,C_k\right)$. Considering φ(·) as the state update function we can write: s_k+1= φ (s _k , μ _k , C_k+1, B _k ). In (6) g_N+1is the final stage utility which depends only on the final state of the system, s_N+1and it can include some limitations or penalties on the final state of the system.

where U(·), and -V(·) are suitable continuous, concave functions, and parameters α and β control the tradeoff between rate, energy, and delay in the utility function. A similar formulation for per stage utility is used in [27, 28] for multi-period utility maximization while queue management and thus queue sizes were not considered.

3.2. Infinite time horizon

where the action and state vectors as well as the per stage utility function are defined similar to the FTH problem. We consider both the first- and second-order models for the channel state by applying appropriate format of s _k .

4.1. Per stage adaptation to maximize FTH utility

The function J _k (s _k ) is the maximum expected accumulative utility, achieved under optimal decision, when the system is in state s _k at the k th stage. Thus, J₁ (s₁) is the expected total utility for N stages when the initial state is s₁.

In the above equation, we have $s_{k+1}^d=\left(q_{k+1},{\bar{q}}_m^d,{\bar{r}}_l^d,C_{k+1},C_k\right)$ and $s_{k+1}=\left(q_{k+1},{\bar{q}}_{k+1},{\bar{r}}_{k+1},C_{k+1},C_k\right)$. Furthermore q_k+1, ${\bar{q}}_{k+1}$ and ${\bar{r}}_{k+1}$ are given by (1), (2), and (3), respectively. We consider the second-order Markov model by using both C_k-1and C _k in the state vector. The other channel models can be considered as its special case. The solution provided in Equations (17)-(19) is valid for any concave and continuous function of g.

Since ${\bar{r}}_k^d$ and ${\bar{q}}_k^d$ are independent of the decision in the k th timeslot, $\mathsf{\text{U}}\left({\bar{r}}_k^d\right)-\alpha \mathsf{\text{V}}\left({\bar{q}}_k^d\right)$ do not affect the maximization in (21). Also the summations in (20) and (21) are over all M channel states and two possible channel occupancy conditions.

The discrete DP algorithm can be executed offline and the resulting optimal policy can be stored in a look-up table available at the transmitter. Then, it will be used online to dynamically adapt the action to the system state.

4.2. State-based adaptive control to optimize average utility per stage

We denote the optimal policy as π* which produces the maximum average utility per stage J*. Both π* and J* are independent of the initial state since the influence of the utility of the early stages on the average utility reduces to 0 as N → ∞. Moreover, since the utility per stage, the transition probabilities (4), and the state update Equations (1)-(3) are all stationary, the optimal policy will be stationary (does not change from stage to stage). Therefore, it is a single function, μ*(s), that maps the system states to actions regardless of the stage.

where s⁺ indicates the successor state of the current state s. Considering φ(·) as the state update function s⁺ = φ (s, μ, C⁺, B). The expectation in Equation (23) is over the random processes {B _k } and {C _k }.

Therefore, we apply (27) and compute h⁽ⁿ⁾(s ^d ) for all possible discrete states. For the uncorrelated and first-order channel models there are (L × M _q × M _r × M) discrete states and for the second-order channel model this number should be multiplied by M.

5.1. Structural properties of FTH solution

The following theorem indicates monotonicity of J _k (s _k ) versus the state variables.

Theorem 1: J _k (s _k ) is a decreasing function of q _k and ${\bar{q}}_k$, and an increasing function of ${\bar{r}}_k$ and C _k for all values of k.

Proof: In order to prove the theorem we show through induction that for k = N + 1, ..., 1 we have J _k (s _k + Δ) ≤ J _k (s _k ) for any vector Δ that increase q _k and ${\bar{q}}_k$, and decrease ${\bar{r}}_k$ and C _k .

Thus, $J_k\left(s_k\right)=G_k\left(s_k,{\mu }_k^{*}\right)$ where ${\mu }_k^{*}$ is optimal decision for state s _k . Also we define Δ = (δ₁, δ₂, - δ₃, -δ₄, -δ₅) for any value of δ _i ≥ 0, i = 1, ... , 5 such that $s_k+\mathrm{\Delta }=\left(q_k+{\delta }_1,{\bar{q}}_k+{\delta }_2,{\bar{r}}_k-{\delta }_3,C_k-{\delta }_4,C_{k-1}-{\delta }_5\right)$ is an element of the state space.

For k = N + 1 we have $J_{N+1}\left(s_{N+1}\right)=U\left({\bar{r}}_{N+1}\right)-\gamma \mathsf{\text{V(}}{\bar{q}}_{N+1}\mathsf{\text{)}}-\eta q_{N+1}$ and using assumption 1 it is clear that J_N+1(s_N+1+ Δ) ≤ J_N+1(s_N+1). Assuming J_k+1(s_k+1) is a monotonic function we show J _k (s _k ) is also monotonic for k = N, ... , 1 which completes the proof.

We consider φ (·) as the state update function and define two possible next states $s_{k+1,\mathrm{\Delta }}^{*}=\phi \left(s_k+\mathrm{\Delta },{\mu }_{k,\mathrm{\Delta }}^{*},C_{k+1},B_k\right)$ and $s_{k+1}^{\#}=\phi \left(s_k,{\mu }_{k,\mathrm{\Delta }}^{*},C_{k+1},B_k\right)$. For known values of C_k+1and B _k we can use (1)-(3) and easily show that $s_{k+1,}^{{*}_{\mathrm{\Delta }}}=s_{k+1}^{\#}+{\mathrm{\Delta }}^{\prime }$ in which ${\mathrm{\Delta }}^{\prime }=\left({\delta }_1^{\prime },{\delta }_2^{\prime },-{\delta }_3^{\prime },-{\delta }_4^{\prime },-{\delta }_5^{\prime }\right)$ for some ${\delta }_i^{\prime }\ge 0$. Thus $J_{k+1}\left(s_{k+1,\mathrm{\Delta }}^{*}\right)\le J_{k+1}\left(s_{k+1}^{\#}\right)$.

Equation (30) together with (33) prove the theorem by showing: J _k (s _k + Δ) ≤ J _k (s _k ).

   ■

Assuming uncorrelated channel model the following theorem indicates the "threshold structure" of the optimal transmission policy versus the channel state.

Theorem 2: If the optimal access decision in state $s_k=\left(q_k,{\bar{q}}_k,{\bar{r}}_k,C_k\right)$ is $I_k^{*}\left(s_k\right)=1$, then for another possible state $s_k^{\prime }=\left(q_k,{\bar{q}}_k,{\bar{r}}_k,C_k^{\prime }\right)$ in the same slot with improved channel state $C_k^{\prime }\ge C_k$ we have $I_k^{*}\left(s_k^{\prime }\right)=1$.

which is in contrast to optimality of the ${\mu }_k^{*}\left(s_k^{\prime }\right)=\left(I_k^{\prime }=0,r_k^{\prime }\right)$. Thus, we should have ${\mu }_k^{*}\left(s_k^{\prime }\right)=\left(I_k^{\prime }=1,r_k^{\prime }\right)$.

   ■

Note that Theorem 2 may be incorrect when channel state is time correlated. For example, consider two possible channel states C _k and $C_k^{\prime }$, with $C_k<C_k^{\prime }$ and assume that optimal decision is to transmit for a state with C _k . Also assume that probability of going from $C_k^{\prime }$ to a better channel state and from C _k to a worse channel state is high. So, we can argue heuristically that in this condition it may be optimal to transmit data when channel is in state C _k but not to transmit when it is in state $C_k^{\prime }$.

5.2. Structural properties of ITH solution

We provide structural properties of ITH solution in this section through the following theorems. First we show that relative value function, h*(s), is a monotonic function in Theorem 3 and then prove the threshold structure of access decision versus channel state in Theorem 4.

Theorem 3: h*(s), is a decreasing function of q and $\bar{q}$, and an increasing function of $\bar{r}$ and C.

Combining (38) and (40) we find that B _τ h⁽ⁿ⁾(s + Δ) ≤ B _τ h⁽ⁿ⁾(s). Using Equation (25) and taking into account that B _τ h⁽ⁿ⁾(s') is independent of the state vector, it can be easily shown that h^{(n + 1)}(s) is a monotonic function.

   ■

Assuming uncorrelated channel model the following theorem indicates existence of a threshold for channel state that the link should decide to transmit when channel state is better than or equal to that threshold.

Theorem 4: There exists a threshold, C_th, that for $s_{\mathsf{\text{th}}}=\left(q=C_{\mathsf{\text{th}}},\bar{q},\bar{r},C_{\mathsf{\text{th}}}\right)$ with any $\bar{q}$ and $\bar{r}$, we have I*(s _th ) = 1. Also for any s with C ≥ C_th and q ≥ C_th we have I*(s) = 1.

Proof: Assume in timeslot k we have C _k = C^max and q _k = C^max, transmission at this time has the energy cost of β e but it will reduce q by C^max which will reduce $\bar{q}$ by θ _q C^max and also will reduce the future costs related to the queue size. However, transmission of theses C^max packets at any later time slot requires the same amount of energy. Thus, it is better to transmit these packets at state s_th to reduce the queue size as early as possible and reduce the future costs related to the queue size. We conclude that: "if C = C^max and q = C^max then I*(s) = 1" which proves existence of C_th.

In order to prove the second part of the theorem we assume $s=\left(q\ge C_{\mathsf{\text{th}}},\bar{q},\bar{r},C\ge C_{\mathsf{\text{th}}}\right)$ and consider optimal decisions μ*(s_th) = (I*(s_th), r*(s_th)) and μ*(s) = (I*(s), r*(s)) for states s_th and , respectively. If I(s) = 0 we can show similar to the proof of Theorem 2 that it cannot be an optimal transmission policy.

   ■

6.1. FTH results

As indicated earlier for FTH problem we use (8) as the final stage utility function with η = 5 as the price for packets remaining in the queue where $U\left({\bar{r}}_k\right)=\log \left(\epsilon +{\bar{r}}_k\right)$, and $\mathsf{\text{V}}\left({\bar{q}}_k\right)={\left({\bar{q}}_k\right)}^2$. We assume that the initial state of the link is ${\bar{r}}_1=0.1$, ${\bar{q}}_1=q_1=0$ and consider a flat fading Rayleigh channel. The recursive algorithm (16)-(18) is used to obtain the optimal control policy (over the discretized state space) for different values of N. The transition probabilities of the Markov models are computed as described earlier.

The optimal control policies are then used in Monte Carlo simulations, over the channel response γ _k and the channel occupation B _k processes, to maximize J₁(s₁), the sum of the utilities of the N stages. Figure 4 illustrates the J₁(s₁), as a function of the number of timeslots N, for different channel correlation models and two values of average SNR. This figure shows that the performance is enhanced by exploiting the channel correlation through the FSMC models, mainly for large values of N. It also shows that the use of second-order FSMC is not worthwhile in these cases. For N > 50, J₁ (s₁) varies almost as a linear function of N where the slope depends on the channel correlation model and the average SNR.

We have also investigated the dependence of performance on the initial state of the link. Figure 5 illustrates the average utility, J₁(s₁)/N, versus the initial EWMA rate ${\bar{r}}_1$ for different values of N. As expected the higher initial EWMA rate, the higher average utility. It also shows that sensitivity to the initial state decreases as the number of slots increases.

6.2. ITH results