 Research
 Open Access
Minimumlength scheduling with rate control in wireless networks: a shortest path approach
 Anna Pantelidou^{1}Email author and
 Anthony Ephremides^{2}
https://doi.org/10.1186/168714992011115
© Pantelidou and Ephremides; licensee Springer. 2011
 Received: 23 November 2010
 Accepted: 30 September 2011
 Published: 30 September 2011
Abstract
In this paper, the minimumlength scheduling problem in wireless networks is studied, where each source of traffic has a finite amount of data to deliver to its corresponding destination. Our objective is to obtain a joint scheduling and rate control policy to minimize the total time required to deliver this finite amount of data from all sources. First, networks with timeinvariant channels are considered. An optimal solution is provided by formulating the minimumlength scheduling problem as finding a shortest path on a singlesource directed acyclic graph. However, finding the shortest paths is computationally hard since the number of vertices and edges of the graph increases exponentially in the number of network nodes, as well as in the initial traffic demand values. Toward this end, a simplified version of the problem is considered for which we explicitly characterize the optimal solution. Next, our results are generalized to timevarying channels. First, it is shown that in case of timevarying channels, the minimumlength scheduling problem can be formulated as a stochastic shortest path problem and then an optimal policy is provided that is based on stochastic control. Finally, our analytical results are illustrated with a set of numerical examples.
Keywords
 Crosslayer design
 Minimumlength scheduling
 Rate control
 Stochastic shortest paths
I. Introduction
The problem of minimumlength scheduling involves obtaining a sequence of activations of wireless nodes so that a finite amount of data residing at a subset of the nodes in the network reaches its intended destinations in minimum time. It is closely related to the problems of network throughput or stable throughput maximization, since minimizing the time to deliver a fixed amount of data can be seen as maximizing the effective rate at which data traverse the network. However, it also differs from them in that some network resources become available to heavily loaded nodes when the lightly loaded ones become relieved. Furthermore, it can be introduced as a useful alternative metric that characterizes the trafficcarrying capabilities of wireless networks with nonstationary and nonergodic channel variations, where the commonly used performance criteria of stable throughput and network capacity are not well defined. Although in this paper we focus on networks with stationary and ergodic channel behavior, we expect our analysis to yield valuable insights regarding the more general case of nonergodic and nonstationary wireless channels.
The topic of obtaining schedules of minimum length has attracted the attention of the research community, as in [1–10] to sample a few. The first formulations of the problem of scheduling for efficient access to a shared channel, which we are aware of, appeared in [1, 2]. A simple collision channel model had been considered but with the possibility of spatial reuse. That is, an "interference map" was assumed in terms of a graph that described all the independent sets of nodes in the graph, namely those sets of nodes that do not include "adjacent" nodes. The objective was to determine the shortest length of a frame of slots that would allow all nodes to transmit once in the frame without violating the "interference" constraint imposed by the interference rules on the graph. It was shown that the problem is NPcomplete, and a distributed heuristic was developed that showed decent performance compared to the optimum that was computable in "small" instances of the problem.
This problem was revisited in more generality through a continuous approximation of the structure of the frame schedule in [3], where the authors obtain a centralized, polynomial time algorithm for static networks that finds a schedule of minimum length satisfying a set of linktraffic requirements. However, in [3], modeling of the physical layer is overly simplified as it is assumed that any two links can be successfully activated simultaneously as long as they do not share any common vertices. This simplification relates the minimumlength scheduling problem to the problem of obtaining a maximal matching in a nonbipartite graph [11]. However, due to the broadcast nature of the wireless medium all, concurrent transmissions can potentially contribute to the total amount of interference at each receiver and, thus, cause a reception to fail.
A commonly accepted model for capturing the effects of interference is the Signal to Interference plus Noise Ratio (SINR) criterion under which the outcome of a transmission depends on the ratio of the signal power at a receiver to the noise and the total interference. If this ratio exceeds a certain threshold, then the transmission is assumed to be successful. Although still an approximation, this model is reasonable and captures the overall interference generated by the simultaneous transmissions in the network. A variation of this problem formulation that incorporated some physical layer attributes was studied in [4, 5].
In [5], the authors consider the problem of obtaining a schedule of minimum length under the SINR interference model. They assume that the transmission rates are fixed and that each transmitting node selects optimally its transmission power. In [5], the minimumlength scheduling problem is formulated as a linear program [12] that can possibly have a prohibitively large number of variables and thus is hard to solve. In [6–9], the authors consider the minimumlength scheduling problem for different sets of optimization parameters. Specifically, they consider the cases where (1) both the transmission powers and rates are fixed, (2) the transmission powers can be optimized, but the transmission rates are fixed, and (3) the transmission powers are fixed and each transmitter is allowed to choose its rate from a predetermined, finite set of rates, common to all transmitters. In [6–9], the minimumlength scheduling problem is also formulated as a complex linear program, with a relatively small number of constraints and a large number of variables. To address the high complexity, the authors employ the technique of column generation [12], whose running time is faster, on average, than that of the original linear program.
Most of the prior work on the minimumlength scheduling problem assumes that the transmission rates are fixed. However, due to the broadcast nature of the wireless medium, the parameters of the physical layer, such as the transmission powers and rates, are coupled with the scheduling decisions that can be made at the medium access control. In the recent years, there is ample evidence of how the variables associated with a particular layer depend on variables associated with other layers. Exploiting the links and dependencies between such variables usually yields superior solutions (see e.g., [13–17]). This is similar, if not equivalent, to maximizing a function of multiple variables by considering its dependence on all the variables rather than on a subset of them alone. Therefore, due to this coupling between the physical layer and the medium access control in wireless systems, it is clear that a joint optimization of link activation and rate control will yield a better performance, which is the focus of this paper.
In [10], the authors consider a crosslayer view of the minimumlength scheduling problem for static, singlehop networks through rate control and formulate the problem as a shortest path on a directed acyclic graph (DAG). In the first part of this paper, we consider static networks where the channel effect is due to pure path loss. We first assume a slottedtime model and formulate the minimumlength scheduling problem as a shortest path between a given sourcedestination pair on a DAG. We obtain an optimal joint scheduling and rate control solution that provides a shortest path. Although finding a shortest path on a DAG has a polynomial complexity in the number of vertices and edges, this number grows exponentially as the size of the network and initial data traffic load increase. For this reason, we make the following simplifications. We first map the discretetime problem to a continuoustime equivalent, where slots are eliminated. We then restrict the possible scheduling and rate control decisions to either communication "one at a time", in a Time Division Multiple Access (TDMA) fashion, or "all together" for all time. A similar approach for the problems of sumrate maximization and proportional fairness was considered in [18]. We explicitly characterize the optimal solution of this reduced problem. Understanding the behavior of the optimal policy, even for the reduced problem, is significant since it provides valuable intuition about when scheduling and rate control actions for one or the other extreme are preferable. This intuition, for example, can improve the performance of the column generation technique in [6–9] by providing the algorithm with those scheduling and rate control actions that are expected to be employed by an optimal policy in the reduced problem and thus improve that heuristic.
However, the cited work (i.e., [1–10]) studies the minimumlength scheduling problem only for timeinvariant wireless networks. Since the wireless channel in reality is timevarying, in the second part of this paper, we consider the timevarying case. Our goal then becomes to find an optimal policy that minimizes the expected time required to deliver all the traffic to its respective destinations. We solve the minimumlength scheduling problem by formulating it as a stochastic shortest path, which is a special case of a Markov Decision Process (MDP) [19]. We obtain an optimal scheduling and rate control policy through stochastic control methods. Our approach is similar to the works in [20] and [21] where the minimumlength scheduling problem is formulated as an MDP under exact and statistical knowledge of the underlying channel conditions, respectively. For timeinvariant channel processes, this model reduces to finding a shortest path on a DAG and methods described in the first part of this paper are applicable to compute the optimal solution.
Our work differs from [1–3] since we model the interference more accurately through the SINR interference model. We follow a different approach from [4–9] since we formulate the minimumlength scheduling problem as finding a shortest path on a singlesource DAG, and we give an optimal graphtheoretic algorithm. Furthermore, we provide an explicit characterization of an optimal policy for a simplified, continuoustime model that is obtained by reducing the set of feasible scheduling and rate control decisions to either transmission in the "one at a time" fashion, as in TDMA, or in the "all together" mode. Our results are different from [4, 5] since we consider joint scheduling and rate control decisions. Finally, and more importantly, we generalize existing work by considering the more realistic case of timevarying channels.
The rest of this paper is organized as follows. In Section II, we present the network model. In Section III, we consider static wireless networks. In particular, in Section IIIA, we present a graphtheoretic formulation of the minimumlength scheduling problem as a shortest path problem on a singlesource directed acyclic graph. In Section IIIB, we first map the problem to a continuoustime model and we then restrict the set of feasible scheduling and rate control actions that can be employed. By doing so, we are able to explicitly characterize an optimal policy that finds a schedule of minimum length. Then, in Section IV, we consider timevarying wireless networks. In Section IVA, we formulate the minimumlength scheduling problem as a stochastic shortest path, and in Section IVB, we provide an optimal solution by employing the principles of stochastic control theory. Specifically, we use the value iteration method [19] to find a stochastic shortest path. In Section V, we complement our analytical results with some numerical experiments, and finally, in Section VI, we conclude the paper. In Appendix 1, a table of variables is provided to make the notation comprehensively clearer to those who prefer to see the full notational picture. An optimal algorithm that computes a shortest path on a DAG is given in Appendix 2. The proofs of our results appear in Appendices 34.
II. Model formulation
We consider a slottedtime, singlehop, wireless network consisting of K transmitter and receiver pairs. Without loss of generality, we assume that the slot duration is equal to 1 sec. Each transmitter has a finite amount of data units, e.g., a file of packets or bits to deliver to its corresponding receiver. The objective is to activate the transmitters so that the time to deliver all the traffic to the intended receivers is minimized. The singlehop network assumption, albeit simplifying, is interesting and highly nontrivial since it captures the fundamental problems that arise due to interference, when multiple nodes attempt simultaneous channel access. We denote by $K=\left\{1,\dots ,K\right\}$ the set of all transmitter and receiver pairs in the network. At every time slot, each transmitter $k\in K$ can either transmit at its maximum transmission power ${P}_{k}^{\text{max}}$ or remain silent. We denote the transmission power level of the k th transmitter at time slot t by P_{ k }( t ), where ${P}_{k}\left(t\right)\in \left\{0,{P}_{k}^{max}\right\}$.
It is assumed that each transmitter k has a fixed amount of d_{ k } bits to deliver to its corresponding destination. We denote by d = (d_{1}, ..., d_{ K } ) the vector of initial data traffic at each transmitter. We also denote by X_{ k } (t) the queue size at transmitter k at time slot t and by X(t) = (X_{1}(t), ..., X_{ K } (t)) the corresponding vector of queue sizes at all transmitters in the network. The queue size of each transmitter at time slot 0 is equal to its initial data traffic, i.e., X(0) = d. The state space of the process ${\left\{\text{X}\left(t\right)\right\}}_{t=0}^{\infty}$ is denoted by $X$.
where N_{ k } is the thermal noise power at receiver k.
It is known that the maximum transmission rate is an increasing function of the SINR threshold (see e.g., [22]). This gives rise to the following tradeoff: By increasing the transmission rate, the number of transmitters that can successfully satisfy the SINR criterion concurrently decreases. On the other hand, by decreasing the transmission rate, a higher number of transmitters can jointly satisfy the SINR criterion. Thus, it is not clear whether allowing more concurrent transmissions (less time sharing) at lower rates is preferable to allowing fewer concurrent transmissions (more time sharing) at higher rates. The answer is tightly dependent on the performance objective and on the network parameters, such as the transmission powers, channel conditions, etc.
A joint scheduling and rate control policy at any given time needs to decide (a) which transmitters to activate and (b) their respective transmission rates. This information can be captured by the Kdimensional rate vector r(t) = (r_{1}(t), ..., r_{ K } (t)), where r_{ k } (t) is the rate of transmitter k at slot t. If a transmitter is assigned a zero rate, then it is not activated by the policy. In other words, a transmission rate vector implicitly specifies the scheduling decisions. We define the set of all feasible rate vectors to contain those that are obtained by the following twostep procedure: We first identify all possible subsets of activated transmitters (by assigning to each transmitter k either power 0 or ${P}_{k}^{max}$), and then, we assign them the maximum rates that allow all activated transmitters to jointly satisfy the SINR criterion. Thus, there exist 2 ^{ K }  1 such Kdimensional transmission power vectors, each of which corresponds to an achievable rate vector. Clearly, the set of achievable rates depends on the current channel state $\text{g}\in G$. Hence, for every channel state g, we denote by $R(\text{g})$ the finite, discrete set of feasible Kdimensional rate vectors. Then, the cardinality of $R(\text{g})$, i.e., $R(\text{g})$, is equal to 2 ^{ K } 1 for every channel state $\text{g}\in G$.
That is, we allow for the possibility of restricting the space of actions by limiting the range of allowable policies. If $A\left(\text{s}\right)$ is a strict subset of the overall feasible scheduling and rate control decisions, then scheduling will be suboptimal in general at the benefit of decreased complexity. Furthermore, by selectively choosing the elements of the set $A\left(\text{s}\right)$, it is possible to obtain performance close to optimal while achieving considerable reduction in computational complexity.
We assume that every admissible policy uses the channel state information rationally so that a scheduled transmission must always be successful. Naturally, as reflected by the cardinality of the set $R(\text{g})$, the policies we consider are nonidling, i.e., they always activate at least one transmitter that has a nonempty queue until all the queues in the network are empty. We call the class of stationary, nonidling policies given by the mapping (3) as admissible, and denote them by Π.
where the rate control and scheduling vectors r(t) are given according to the mapping in (3) and where [z]^{+} = max{z, 0}.
Clearly, the queue size at each transmitter k takes its maximum value at time slot 0, when it is equal to the initial demand d_{ k } and, due to the absence of external arrivals, it keeps decreasing over time until it reaches zero. Under the above model, we proceed to formulate the minimumlength scheduling problem for static and timevarying networks.
III. Static networks
In this section, we restrict our attention to static networks, where the channel qualities G_{(k, j)}(t) are equal for every time slot t, i.e., we ignore effects of fading or user mobility. Thus, the cardinality of the set $G$ is equal to one. To simplify notation, in this section, we denote the channel quality G_{(k, j)}(t) as G(k, j). We will drop this assumption in Section IV where we will consider timevarying channel processes. Furthermore, since there is a single channel state g, to simplify notation, we will write $R$ to denote $R(\text{g})$ for $\text{g}\in G$ and $A$ to denote $A\left(\text{s}\right)$ for s = (x, g), $\text{g}\in G$. At every time slot t, the scheduling and rate control policy identifies a rate vector $\text{r}\left(t\right)=\left({r}_{1}\left(t\right),\dots ,{r}_{K}\left(t\right)\right)\in A\subseteq R$ that specifies which transmitters are activated and their respective rates.
In the specific case of pure TDMA scheduling, combined with rate control, where only a single transmitter can be active at any given time, the solution of the above problem becomes trivial. Specifically, each transmitter must be active for as many time slots as needed to empty its queue. The required number of such time slots for each transmitter k is equal to the ratio of its initial demand d_{ k } divided by its corresponding rate when it accesses the channel individually, rounded upwards to the closest integer value. Then, the minimum total time that is needed until all the queues are empty is equal to the sum of the time slots required by each transmitter. The order in which the transmitters must be activated is immaterial; they can be chosen in a roundrobin or random fashion.
However, the solution of the optimization problem given by (5)(7) is, in general, a nontrivial discrete optimization problem. In the following subsections, we provide an optimal graphtheoretic algorithm by mapping it to a shortest path problem on a DAG, and in the sequel, we give an explicit characterization of the optimal policy for a reduced version of this problem.
A. The equivalent DAG representation
We construct the weighted DAG $\u1e20=\left(V,E\right)$ as follows: We assume that every vertex u ∈V of the DAG represents a queue size vector that can be obtained, starting from a vector of queues X(t), by employing some scheduling and rate control action chosen from the set $A$. Furthermore, every directed edge (u, v) ∈ E represents one such action in $A$. We say that the edge (u, v) is incident from u and incident to v. Hence, from every vertex x _{ i }, we can have $\leftA\right$ edges that are incident from x _{ i }, each corresponding to a different rate vector ${\text{r}}^{j},i=1,\dots ,\leftR\right$. Each such edge is incident to a graph node y_{ i } = [x_{ i }  r^{ i } ]^{+}. We disallow those edges that correspond to rate vectors, which activate transmitters with empty queues. Therefore, the actual number of edges that are incident from a vertex can be less than $\leftA\right$. The weight of each edge is equal to one. From now on, we will refer to action r^{ i } by means of the edge (x _{ i }, y_{ i }). The unique source node x_{0} of the DAG corresponds to the vector of initial demands, X(0).
As we observe from Figure 2 for any path of vertices < x_{0}, x_{1}, x_{2}, ..., x_{ m } >, the queue size vector of each vertex in the path has to be componentwise larger or equal to the queue size of any other vertex that succeeds it in the path and the queue size vectors of any two vertices on the graph cannot be the same. As a result, the overall graph of the different queues is directed and acyclic. Finally, it is clear that every path starting at the source x_{0} ends at the 0vector. Moreover, the weight of any subpath < x_{0}, x_{1}, x_{2}, ..., x_{ m } > is equal to its length m, which is effectively the number of time slots required to go from vertex x_{0} to vertex x_{ m } along the specified path, since by construction of this DAG the weight of each edge is equal to one time slot. Thus, the initial problem given by (5)(7) is transformed into a shortest path problem on a weighted singlesource DAG. A shortest path on the DAG G = (V, E) can be obtained through the DAGSHORTESTPATHS$\left(\u1e20,{\text{x}}_{0}\right)$ algorithm. The exact algorithm, taken from [11], is given in Appendix 2.
For the example given in Figure 2, the shortest path algorithm selects the sequence of actions r^{3}, r^{3}, r^{2}, and the minimum schedule length is equal to 3 sec (slots). Note that the sequences of actions r^{2}, r^{3}, r^{3} and r^{3}, r^{2}, r^{3} are also optimal as the order in which the actions are taken is immaterial in terms of minimizing the time needed to empty the queues, under the assumption of static channels. Also, it is worth mentioning that the length of the optimal schedule obtained through rate control is, naturally, no longer than that of TDMA (employing only actions r^{1} and r^{2}), which, in this example, has length 4. Furthermore, it is reasonable to expect that the difference in the schedule lengths under the two schemes can become significant when the number of transmitter/receiver pairs in the network increases or when the values of initial demands are large. This will also be illustrated through a set of numerical results in Section V.
The optimality of DAGSHORTESTPATHS$\left(\u1e20,{\text{x}}_{0}\right)$ can easily be verified (see e.g., [11], Theorem 24.5). Also, it is easy to see that its overall running time is Θ(V  + E). Hence, the number of operations needed to compute a shortest path on a singlesource DAG is of polynomial complexity in the number of vertices and edges. However, in our DAG construction, this number grows exponentially (1) in the number of transmitters when $A=R$ since from every vertex there exist 2 ^{ K }  1 potential edges that are incident from it and (2) as the initial demands increase. Therefore, the overall complexity of the algorithm becomes exponential, and despite its theoretical merit, it is rendered impractical.
B. Continuoustime model
To decrease the complexity that stems from the discrete nature of the minimumlength scheduling problem, we can map the problem given by (5)(7) to a continuoustime one. Therefore, instead of seeking the minimum number of time slots required to deliver all data traffic to its respective destinations, we will be interested to obtain the minimum duration or period of time that has to elapse until all network queues empty. In this way, the minimumlength scheduling problem becomes a linear program with a relatively small number of constraints and a large number of variables as in the formulations of [5], [6], and [7]. In order to solve this linear program, we follow a different approach than [5], [6], and [7]. In particular, we reduce the number of variables involved, i.e., the scheduling and rate control decisions that the policy employs, and then obtain an optimal solution for this reduced problem.
As is commonly done, to understand the essential features of a difficult problem, one needs to consider a simplified version that inherits and keeps the key features of the problem. Specifically, here, we restrict the set $A$ to contain only feasible rate vectors obtained by two simple schemes, namely scheduling a single transmitter at a time (in a TDMA fashion) or concurrently activating all the transmitters, as considered in [23, 24]. By doing so, we decrease the cardinality of $A$ to K + 1. Clearly, such a reduction is expected to yield suboptimal results. However, this simplification focuses attention on the tradeoff between transmitting more frequently (at a reduced rate) and transmitting less frequently (at a higher rate). We expect that this approach will help us gain valuable insights regarding the nature of optimal scheduling and rate control for the general problem.
The following theorem characterizes an optimal scheduling and rate control policy that solves (8)(10).
 1)If it is true that$\sum _{k=1}^{K}\frac{{r}_{k}^{0}}{{r}_{k}^{k}}\le 1,$(11)
 2)If it is true that$\sum _{k=1}^{K}\frac{{r}_{k}^{0}}{{r}_{k}^{k}}\ge 1,$(14)
The proof appears in Appendix 3. To completely characterize the policy, we need to specify the set $J$, which results from the following lemma.
The proof of the lemma appears in Appendix 4.
From the above, we conclude that the set $J$ contains the transmitters with the highest $\leftJ\right$ values of ${d}_{k}\u2215{r}_{k}^{0}$, where $\leftJ\right$ is given by Lemma 1. Hence, an optimal scheduling and rate control policy individually activates the transmitters that either have a very high initial demand or whose rates under concurrent operation are very low, e.g., due to excessive amounts of interference caused by other concurrent transmissions. Those transmitters must be further assisted toward emptying their queues by being granted individual access to the channel.
IV. Timevarying networks
In the previous section, we focused on timeinvariant channels. However, in reality, the wireless channel is timevarying, due to the effects of fading, node mobility, etc. In this section, we extend our model by considering timevarying channels. We make the following assumption on the wireless channel process ${\left\{\text{G}\left(t\right)\right\}}_{t=0}^{\infty}$.
where the expectation is with respect to the stationary probability distribution of the channel process.
We proceed to present a solution to the problem of (20)(22) through stochastic control methods by considering admissible policies in the class Π.
A. Stochastic shortest path formulation
Evidently, from (4), this Markov Chain is absorbing and from every nonterminating state a terminating state is reached with probability one in finite time under all admissible policies. Furthermore, once the system reaches any state in ${S}_{term}$, it remains there forever. Hence, the objective becomes to reach a terminating state in minimum expected time by choosing the next state. This will yield a schedule of minimum expected length. This is a stochastic shortest path problem, which is a special case of an MDP. If there is no randomness in the channel state, i.e., the entire wireless channel realization is known at priori at the very first time slot, our results of Sect ion III follow from this model as a special case.
The set of feasible scheduling and rate control actions corresponding to each system state $\text{s}=\left(\text{x},\text{g}\right)\in S$ is the set $A\left(\text{s}\right)\subseteq R\left(\text{g}\right)$. The system is driven by the timevarying channel process ${\left\{\text{G}\left(t\right)\right\}}_{t=0}^{\infty}$. Taking an action leads to different states with different probabilities depending on the evolution of the channel process unless the system has already reached a terminating state.
Note that from the Markovianess of the channel process and the admissibility of the policy π, the transition probability p_{ r }(s, s ') is time invariant and does not depend on the previous system states.
We define the cost of taking action r and going from state s to state s ' as ${\stackrel{\u0303}{c}}_{\text{r}}\left(\text{s},{s}^{\prime}\right)$. For every system state s, action $\text{r}\in A\left(\text{s}\right)$ and system state s ' such that p_{ r }(s, s ') > 0, we assume that ${\stackrel{\u0303}{c}}_{\text{r}}\left(\text{s},{s}^{\prime}\right)=1$. This represents the fact that in order to go from state s to state s ' by taking this action one needs to spend one time slot. Let us further define the cost per stage c_{ r }(s) to be the expected cost when, being at state $\text{s}\in S\backslash {S}_{term}$, control action $\text{r}\in A\left(\text{s}\right)$ is chosen. It is clear that ${c}_{\text{r}}\left(\text{s}\right)={\sum}_{{s}^{\prime}\in S}{p}_{\text{r}}\left(\text{s},{s}^{\prime}\right){\stackrel{\u0303}{c}}_{\text{r}}\left(\text{s},{s}^{\prime}\right)=1,\phantom{\rule{2.77695pt}{0ex}}\forall \text{s}\in S\backslash {S}_{term}$. Once a terminal state $\text{s}\in {S}_{term}$ is reached, no more cost is incurred and the system remains there forever, i.e., c_{ r }(s) = 0, $\forall \text{r}\in A\left(\text{s}\right)$, $\text{s}\in {S}_{term}$. Having said the above, the minimumlength scheduling problem can be formulated as a stochastic shortest path. Next, we provide an optimal policy that is obtained through Dynamic Programming [19].
B. An optimal policy
To solve the above shortest path problem, two commonly used methods are the policy iteration and the value iteration[19]. Due to the large state space of the problem, value iteration is easier to compute and, hence, will be used here. Consider the value iteration algorithm and the corresponding "expected " time ${T}_{k}\left(\text{s}\right)$ to empty the queues starting from state s at the k th iteration. Assume that ${T}_{0}\left(\text{s}\right)=\infty $ for all states $\text{s}\in S$. We borrow the following properties from [19].
Lemma 29, borrowed from [19], shows the optimality of the value iteration method to solve problems of stochastic control. Therefore, by employing the value iteration method, we can obtain a solution to the minimumlength scheduling problem in timevarying networks.
Although the value iteration method optimally solves the aforementioned stochastic shortest path problem, in general it may require an infinite number of iterations until it converges. However, if the Markov Chain of the system evolution is acyclic, then it was shown in [19] that the value iteration method for each state converges in a finite number of iterations (at most as many as the number of nonterminating states of the Markov Chain). It is easy to see that the Markov Chain driving our system is acyclic. This is because starting from one of the states whose queue size satisfies X(0) = d, the queue sizes in the network are nonincreasing with time as given in (4) under any admissible policy.
V. Numerical results
Since we have 2 transmitter/receiver pairs, there exist 3 possible rate vectors corresponding to each channel state, denoted by r^{ k } (g), k = 1, 2, when only the k th transmitter is activated and by r^{3}(g), when both transmitters are activated, under channel state g ∈ {B, G}.
We first assume that the initial demands are d_{1} = 4 bits and d_{2} = 6 bits, which is the case discussed in Section III. We consider 3 scenarios associated with different achievable rates for each channel state.

Scenario 1: When the k th transmitter is activated alone, its achievable rate is 3 bits/sec, and when both transmitters are activated simultaneously, the corresponding rate is 2 bits/sec for each. Under this scenario, the channel realization is immaterial and the minimum expected time to empty the queues is 3 sec (slots), i.e., equal to the result of the static network example of Section III.

Scenario 2: Under the good channel state, G, the achievable rates are equal to the case of Scenario 1, i.e., when the k th transmitter is activated alone, its achievable rate is 3 bits/sec and when both transmitters are activated simultaneously, the corresponding rate is 2 bits/sec for each. However, under the bad channel, B, the achievable rates are strictly worse (2 bits/sec for individual transmission and 1 bit/sec for each transmitter under concurrent transmission). Naturally, we anticipate that the expected time required to empty the queues is more than 3 sec (slots).

Scenario 3: We assume that under the bad channel state, B, the achievable rates are equal to the ones in Scenario 1, but the good channel is better and thus allows higher rates (4 bits/sec when a transmitter is activated individually and 3 bits/sec when they are both activated simultaneously). Naturally, the expected time to empty the queues will decrease to a value less than 3 sec (slots).
Expected time required to empty the queues for different values of initial demands, under Scenarios 13, assuming that t he channel starts from a good state.
Demands  Good Channel  Bad Channel  $E\left[T\right]$  

r^{ 1 }(G)  r^{ 2 }(G)  r^{ 3 }(G)  r^{ 1 }(B)  r^{ 2 }(B)  r^{ 3 }(B)  
$\left[\begin{array}{c}\hfill 4\hfill \\ \hfill 6\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  3.00 
$\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 2\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 1\hfill \end{array}\right]$  3.62  
$\left[\begin{array}{c}\hfill 4\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 4\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  2.79  
$\left[\begin{array}{c}\hfill 100\hfill \\ \hfill 100\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  50.00 
$\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 2\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 1\hfill \end{array}\right]$  66.86  
$\left[\begin{array}{c}\hfill 4\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 4\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 3\hfill \\ \hfill 0\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 3\hfill \end{array}\right]$  $\left[\begin{array}{c}\hfill 2\hfill \\ \hfill 2\hfill \end{array}\right]$  40.25 
VI. Conclusions
In this paper, we focused on the problem of joint scheduling and rate control in singlehop wireless networks under the objective of minimizing the required time to deliver all data traffic to the intended destinations. First, we focused on networks with timeinvariant links. We presented a graphtheoretic formulation for the minimumlength scheduling problem as a shortest path on a singlesource directed acyclic graph. Motivated by the combinatorial nature of the minimumlength scheduling problem, and we then mapped it to a continuoustime formulation and restricted the set of feasible scheduling and rate control actions. By doing so, we were able to explicitly characterize an optimal policy for the reduced problem that finds a schedule of minimum length. Finally, we considered timevarying wireless networks. We formulated the minimumlength scheduling problem as a stochastic shortest path and presented an optimal policy by employing the principles of stochastic control theory.
Appendix 1: Table 2 table of variables
Table of variables
Variable  Explanation 

K  Number of transmitter and receiver pairs 
$K$  Set of transmitter and receiver pairs 
${P}_{k}^{\text{max}}$  Maximum instantaneous transmission power of transmitter k 
P_{ k }(t)  Transmission power level of transmitter k at time slot t 
d _{ k }  Initial data traffic at transmitter k 
d  Vector of initial traffic at all transmitters 
X_{ k }(t)  Queue size of transmitter k at time slot t 
X(t)  Queue size vector of all transmitters in the network at slot t 
${\left\{\text{X}\left(t\right)\right\}}_{t=0}^{\infty}$  Process of queue sizes of all transmitters 
$X$  State space of the process ${\left\{\text{X}\left(t\right)\right\}}_{t=0}^{\infty}$ 
G_{(k, j)}(t)  Channel quality between transmitter k and receiver j at slot t 
G(t)  Channel state at time slot t 
${\left\{\text{G}\left(t\right)\right\}}_{t=0}^{\infty}$  Channel process 
$G$  State space of the channel process 
γ_{t,k}(r_{ k }(t))  SINR threshold at receiver k at slot t to receive at rate r_{ k }(t) 
r_{ k }(t)  Transmission rate of transmitter k at time slot t 
N _{ k }  Thermal noise power at receiver k 
r(t)  Vector of transmission rates of all transmitters at time slot t 
$R(\text{g})$  Set of all achievable rates under channel state g 
${\left\{\text{S}\left(t\right)\right\}}_{t=0}^{\infty}$  System process 
$S$  State space of the system process 
$A\left(\text{s}\right)$  Subset of the total rate control actions corresponding to state s = (x, g) 
π  Admissible policy 
Π  Class of admissible policies 
G(k, j)  Channel between transmitter k and receiver j in static networks 
$R$  Set of all achievable rates under static networks 
$A$  Subset of the total rate control actions in static networks 
T  Timeslot number 
$\u1e20=\left(V,E\right)$  Graph $\overline{G}$ with V vertices and E edges 
(u, v)  Edge that is incident from vertex u and incident to vertex v 
m  Path length on a DAG 
Action k  Action activating only transmitter k 
Action 0  Action activating all transmitters concurrently 
${r}_{k}^{k}$  Rate of transmitter k under Action k 
${r}_{k}^{0}$  Rate of transmitter k under Action 0 
τ_{ i }  Time duration that Action i ∈ {0, ..., K} is used 
$J$  Set of transmitters to be activated individually 
p_{ G }(g, g ')  Transition probability from channel state g to g ' 
${S}_{term}$  Set of terminating states 
p_{ r }(s, s ')  Transition probability from system state s to s ' under action r 
${\stackrel{\u0303}{c}}_{\text{r}}\left(\text{s},{s}^{\prime}\right)$  Cost of taking action r and going from state s to state s ' 
c_{ r }(s)  Expected cost when being at state s action r is chosen 
${T}^{\pi}\left(\text{s}\right)$  Expected time to empty the queues starting from s under policy π 
${T}^{\star}\left(\text{s}\right)$  Minimum expected time to empty the queues starting from state s 
${T}_{k}\left(\text{s}\right)$  Expected time to empty the queues starting from s at the k th iteration 
π'  Optimal rate control policy 
Appendix 2: Finding a shortest path on a dag
Shortest path problems on singlesource DAGs can be solved optimally in polynomial time [11]. Below, we provide an optimal algorithm, taken from [11], that finds a shortest path on a DAG.
In order to compute a shortest path, we first need to sort the DAG in topological order and then use a sequence of edge relaxations until we obtain a shortest path from the source x_{0} to the vertex corresponding to the 0 vector. Topological order is a linear ordering of all the vertices of the DAG so that for every edge (x _{ i }, x _{ j }), the vertex x_{ i } appears before x_{ j } in the ordering. The process of edge relaxation verifies whether the current bestknown path from the source x_{0} to a vertex y can be improved by passing through a different vertex x.
We proceed with a few definitions that will be useful in the rest of this appendix. We define the distance of a vertex x to be the minimum distance in terms of edges that must be traversed from the source to reach x. We also denote by δ[x] an upper bound on the distance of vertex x. For every edge (x, y), we say that x is the predecessor of y and we write x = Pred[y]. We denote by Adj[x] a list that contains all the vertices y that are adjacent to x, i.e., such that there exists an edge (x, y) ∈ E. The pseudocode of the algorithm is provided below:
 1
topologically sort the vertices of $\overline{G}$
 2
INITIALIZESINGLESOURCE$\left(\u1e20,{\text{x}}_{0}\right)$
 3
for each vertex x taken in topologically sorted order
 4
do for each vertex y ∈ Adj[x]
 5
do RELAX(x,y)
The topological sorting of the first line of the algorithm can be completed in Θ(V + E) time, by running a DepthFirst Search (DFS) [11]. The second line of the algorithm involves the initialization of various variables as shown next:
 1
for each vertex x ∈ V
 2
do δ[x] ← ∞
 3
Pred[x] ← NIL
 4
δ[x _{0}] ← 0
This process requires time order Θ(V). Finally, in lines 35 of the DAGSHORTEST PATHS$\left(\u1e20,{\text{x}}_{0}\right)$ algorithm, at each time step, the next vertex in the topological order is selected and a sequence of relaxations over all edges that are incident from this vertex is performed. The procedure RELAX(x,y), given next, verifies whether the current shortest path from x_{0} to y can be improved by passing through x.
 1
if δ[y] > δ[x] + 1
 2
then δ[y] ← δ[x]+1
 3
Pred[y]←x
Appendix 3: Proof of Theorem 1
 1)For every Action $k\in K$ we have$\frac{\partial L\left(\tau ,\mu ,\lambda \right)}{\partial {\tau}_{k}}=1+{\mu}_{k}{r}_{k}^{k}+{\lambda}_{k}=0.$(32)
 2)For Action 0 we have$\frac{\partial L\left(\tau ,\mu ,\lambda \right)}{\partial {\tau}_{0}}=1+\sum _{k=1}^{K}{\mu}_{k}{r}_{k}^{0}+{\lambda}_{0}=0.$(33)
 3)For every Action $k\in K$ it must be true that$\begin{array}{c}{\mu}_{k}\left({\tau}_{k}{r}_{k}^{k}+{\tau}_{0}{r}_{k}^{0}{d}_{k}\right)=0\phantom{\rule{1em}{0ex}}\Rightarrow \\ \phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}{\mu}_{k}\ge 0,\phantom{\rule{1em}{0ex}}{\tau}_{k}{r}_{k}^{k}+{\tau}_{0}{r}_{k}^{0}\phantom{\rule{1em}{0ex}}\ge \phantom{\rule{1em}{0ex}}{d}_{k}.\end{array}$(34)
 4)For all actions i ∈ {0, ...,K} we have${\lambda}_{i}{\tau}_{i}=0\Rightarrow {\lambda}_{i}\ge 0,\phantom{\rule{1em}{0ex}}{\tau}_{i}\ge 0.$(35)
Consider the following cases:
■
Appendix 4: Proof of Lemma 1
Consider now a new ordering ${\left\{{\ell}_{k}\right\}}_{k=1}^{K}$ of the transmitters in decreasing order of the values ${d}_{k}\u2215{r}_{k}^{0}$ for every $k\in K$, i.e.,${d}_{{\ell}_{1}}\u2215{r}_{{\ell}_{1}}^{0}\ge \cdots \ge {d}_{{\ell}_{K}}\u2215{r}_{{\ell}_{K}}^{0}$. Hence, from (45), it follows that there exists a threshold, i.e., a transmitter index value in the new ordering, below which all the transmitters must belong to the set $J$ and above which (that is, the remaining ones), they must belong to the set $K\backslash J$.
■
Appendix 5: Markovian property of the system process
where (47) follows from (3) and (4) and where (48) follows from Assumption 1 on the markovian property of the channel process. This completes the proof.
■
Declarations
Acknowledgements
This work was supported by the Department of Defense under MURI grants W911NF0510246 and W911NF0810238 by the National Science Foundation under the grant CCF0728966 and by Renesas Mobile Corporation.
Authors’ Affiliations
References
 Ephremides A, Truong TV: Distributed algorithms for efficient and interferencefree broadcasting in radio networks. Proceedings of IEEE Infocom 1988.Google Scholar
 Ephremides A, Truong TV: Scheduling broadcasts in multihop radio networks. IEEE Trans Commun 1990., 38(4):Google Scholar
 Hajek B, Sasaki G: Link scheduling in polynomial time. IEEE Trans Inf Theory 1988., 34(5):Google Scholar
 Borbash SA: Design Considerations in Wireless Sensor Networks. PhD thesis, University of Maryland; 2004.Google Scholar
 Borbash SA, Ephremides A: Wireless link scheduling with power control and SINR constraints. IEEE Trans Inf Theory 2006., 52(11):Google Scholar
 Kompella S, Wieselthier JE, Ephremides A: A crosslayer approach to optimal wireless link scheduling with SINR constraints. Proceedings of IEEE Military Communications Conference (MILCOM) 2007.Google Scholar
 Kompella S, Wieselthier JE, Ephremides A: Revisiting the optimal scheduling problem. Proceedings of Conference on Information Sciences and Systems (CISS) 2008.Google Scholar
 Kompella S, Wieselthier JE, Ephremides A, Sherali HD: A crosslayer approach to endtoend routing and SINRbased scheduling in multihop wireless networks. Proceedings of 6th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt) 2008.Google Scholar
 Kompella S, Wieselthier JE, Ephremides A, Sherali HD, Nguyen GD: On optimal SINRbased scheduling in multihop wireless networks. IEEE/ACM Transactions on Networking 2010., 18(6):Google Scholar
 Pantelidou A, Ephremides A: Minimum schedule lengths with rate control in singlehop wireless networks. Proceedings of IEEE Military Communications Conference (MILCOM) 2008.Google Scholar
 Cormen TH, Leiserson CE, Rivest RL, Stein C: Introduction to Algorithms. The MIT Press; 2001.MATHGoogle Scholar
 Bertsimas D, Tsitsiklis JN: Introduction to Linear Optimization. Athena Scientific; 1997.Google Scholar
 Shakkottai S, Rappaport TS, Karlsson P: Crosslayer design for wireless networks. IEEE Commun Mag 2003., 41(10):Google Scholar
 Srivastava V, Motani M: Crosslayer design: a survey and the road ahead. IEEE Commun Mag 2005., 43(12):Google Scholar
 van Der Schaar M, Shankar NS: Crosslayer wireless multimedia transmission: challenges, principles, and new paradigms. IEEE Wirel Commun 2005., 12(4):Google Scholar
 Lin X, Shroff NB, Srikant R: A tutorial on crosslayer optimization in wireless networks. IEEE J Sel Areas Commun 2006., 24(8):Google Scholar
 Kuran MS., Gür G, Tugcu T, Alagöz F: Applications of the crosslayer paradigm for improving the performance of WiMAX. IEEE Wirel Commun 2010., 17(3):Google Scholar
 Pantelidou A, Ephremides A: A crosslayer view of optimal scheduling. IEEE Trans Inf Theory 2010., 56(11):Google Scholar
 Bertsekas DP: Dynamic Programming and Optimal Control. Volume 2. 3rd edition. Athena Scientific; 2007.Google Scholar
 Pantelidou A, Ephremides A: Minimumlength scheduling and rate control for timevarying wireless networks. Proceedings of IEEE Military Communications Conference (MILCOM) 2009.Google Scholar
 Pantelidou A, Ephremides A: Minimumlength scheduling for multicast traffic under channel uncertainty. Proceedings of IEEE Global Communications Conference (GLOBECOM) 2009.Google Scholar
 Goldsmith A: Wireless Communications. Cambridge University Press; 2005.View ArticleGoogle Scholar
 Pantelidou A, Ephremides A: Optimal rate control policies for proportional fairness in wireless networks. Proceedings of Conference on Information Sciences and Systems (CISS) 2008.Google Scholar
 Pantelidou A, Ephremides A: What is optimal scheduling in wireless networks? Proceedings of 4th International Wireless Internet Conference (WICON) 2008.Google Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.