An analytical model for wireless mesh networks with collision-free TDMA and finite queues

Abstract

Wireless mesh networks are a promising technology for connecting sensors and actuators with high flexibility and low investment costs. In industrial applications, however, reliability is essential. Therefore, two time-slotted medium access methods, DSME and TSCH, were added to the IEEE 802.15.4 standard. They allow collision-free communication in multi-hop networks and provide channel hopping for mitigating external interferences. The slot schedule used in these networks is of high importance for the network performance. This paper supports the development of efficient schedules by providing an analytical model for the assessment of such schedules, focused on TSCH. A Markov chain model for the finite queue on every node is introduced that takes the slot distribution into account. The models of all nodes are interconnected to calculate network metrics such as packet delivery ratio, end-to-end delay, and throughput. An evaluation compares the model with a simulation of the Orchestra schedule. The model is applied to Orchestra as well as to two simple distributed scheduling algorithms to demonstrate the importance of traffic-awareness for achieving high throughput.

Introduction

Wireless multi-hop networks are currently on the transition from a popular research topic to the application in real-world scenarios in the industry. Examples include oil refineries [1] and solar tower power plants [2]. However, for such applications, the reliability of state-of-the-art techniques is not sufficient. This can often be traced back to packet collisions due to badly coordinated access to the wireless channel [3]. Therefore, the IEEE has recently extended the IEEE 802.15.4 standard with two approaches for time-division multiple access (TDMA) in multi-hop networks, namely DSME and TSCH [4]. Especially in large networks under heavy load, these approaches are expected to perform much better than the commonly used CSMA/CA. While several implementations of these extensions already exist [5, 6], evaluations of the theoretical performance boundaries have not found much attention in the literature. This paper closes the gap by providing a framework for analytical evaluation of such networks.

The performance evaluation of techniques for wireless networks is possible in multiple ways. Building real-world testbeds provides good insights into the performance of an actual application. However, it is very cost- and time-intensive. Therefore, event-based simulators are developed that try to replicate the reality as closely as possible, while allowing reproducible experiments without the need of specialized hardware. The disadvantage is a complex and extensive implementation and large execution times.

An alternative is the analysis of mathematical models. An analytical model in the sense of this paper is a system of equations that can be solved by numerical means. While such a model usually only describes a small subset of the behavior of a real system and is therefore only an approximation to the real behavior, the calculations are often faster than a simulation by magnitudes and influences from external sources are avoided. The results can also be verified and reproduced quite easily. Last but not least, the comparison of a simulator and an analytical model helps to find inaccuracies and bugs in both approaches and gives new insights into the underlying principles.

The main contributions of the paper are as follows:

• A Markov chain model for a node’s finite queue that considers the slot schedule.

• A multi-hop model for calculating packet delivery ratio, end-to-end delay, and throughput.

• An open-source implementation of the models [7].

• An extensive evaluation, including a comparison with an event-based simulation and the application to two traffic-aware schedules for TSCH.

After presenting the related work in the next section, the requirements of a model for TSCH are outlined in Section 3 followed by the system model in Section 4. We then take the angle of a single node and present and evaluate the queuing model in Section 5. These models are then concatenated in Section 6 to derive global network metrics, and a comparison with a COOJA simulation is presented. In Section 7, two algorithms for building slot schedules are introduced, evaluated, and compared to the Orchestra SBD schedule. Section 8 gives an outlook to possible extensions, and the paper is finally concluded in Section 9.

Related work

Analyzing the performance of wireless networks is a challenging research topic of high value for practical applications [8]. Many evaluations for IEEE 802.15.4 networks are based on simulations [9] or real-world testbeds [10]. Especially for the CSMA/CA technique of IEEE 802.15.4 analytical models exist for single-hop topologies [11] and multi-hop networks [3, 12]. In these publications, packet collisions, especially due to hidden node constellations, are identified as a major reason for bad performance of CSMA/CA.

Since their admission to the standard, TSCH and DSME were analyzed by simulations [13], testbeds [6], and analytical models. The latter includes work focused on special aspects such as network formation [14] or transmission in the contention access period (CAP) [15]. The latter is very similar to the analysis of conventional CSMA/CA because it does not take guaranteed time slots into account. A simple formula for the throughput of DSME networks is given in [16]. Furthermore, much research exists that analyzes TDMA communication on a more general level [17, 18].

A major aspect of network analysis is the influence of the queue filling levels. The fundamentals of queuing theory are well-established [19]. Especially, the M/D/1/K model is of particular interest for TDMA networks, describing Poisson-distributed arrival, deterministic service time, a single transmitter, and a queue of length K. In [20], closed-form expressions are given for the blocking probability and other properties. Many variants were analyzed including service times following a general distribution [21]. In [22], a generalized queuing model for TDMA is developed that is also applicable for traffic that does not follow a Poisson distribution.

In this paper, a queuing model is presented that respects the specific structure of dedicated schedules, in particular, but not limited to, IEEE 802.15.4 TSCH networks. A popular schedule implemented in Contiki [23] is Orchestra [6]. It does not require any management traffic for building up the schedule. Part of the ongoing IETF standardization of 6TiSCH, the link between IPv6 networks and TSCH, is the development of scheduling functions such as SF0 [24] and SF1 [25].

Many other scheduling techniques were proposed that are applicable to TSCH networks or to one of its forerunners, the WirelessHART standard [26]. A centralized schedule for the latter is presented in [27]. Another centralized schedule, this time for TSCH, is the Traffic Aware Scheduling Algorithm (TASA) [28]. Traffic awareness is an important property for improving throughput as we will also see in the evaluation of this paper. However, centralized scheduling algorithms come with a high overhead for building, distributing, and maintaining the schedule, especially in large networks. Therefore, many decentralized approaches were suggested, such as DeTAS [29] and DIS_TSCH [30].

Another promising distributed algorithm to generate a conflict-free schedule is Wave, presented in [31] and extensively analyzed in [32]. DeBraS [33] explicitly targets dense networks where colliding slots are a major problem. All mentioned algorithms implicitly minimize energy consumption by reducing the number of slots assigned to every node. In contrast, [34] explicitly targets minimum energy consumption by formulating an energy efficiency maximization problem.

Requirements

The goal of the paper is to provide a tool for estimating the achievable performance of a network using the IEEE 802.15.4 TSCH data link layer with given multi-hop topology and associated schedule. In this section, an analysis of the potential sources for packet loss is presented. These are compared to the features provided by modern TDMA medium access protocols such as IEEE 802.15.4 TSCH and DSME. The most relevant aspects for a model are finally summarized.

The main reasons for packet loss in wireless multi-hop networks can be coarsely categorized as follows.

• Collisions of transmissions of the same network.

• External interferences, for example between IEEE 802.15.4 and IEEE 802.11 transceivers or with another unsynchronized IEEE 802.15.4 network.

• High path loss, for example due to a large distance or fading.

• Queue drops if the rate of packets that are generated or have to be forwarded is higher than the rate at which packets can be transmitted. This can be permanent or during a burst.

To mitigate these losses, the introduced TDMA data link layers TSCH and DSME provide the following features that allow for reliable multi-hop communication.

• A slot structure with predefined timing to arrange the transmissions in the time domain to avoid collisions within the same network.

• Time synchronization to ensure an aligned timing throughout the multi-hop network.

• Channel adaption (only DSME) and channel hopping (both) to arrange transmission in the frequency domain to increase the number of transmissions per time as well as to mitigate external interferences.

• Procedures for setting up time and frequency schedules in a distributed fashion to ensure conflict-free transmissions. This is only integrated in DSME, but the ongoing IETF standardization of 6TiSCH provides related features for TSCH.

To calculate the performance in steady-state, it can be assumed that all nodes are already associated to the network, are properly synchronized to the global notion of time, and have negotiated a conflict-free and valid multi-hop schedule. In such a schedule, a combination of a time slot and a frequency channel is either free or assigned to exactly one transmitter and one receiver in every neighborhood. By this, packet collisions can be avoided, even in hidden node constellations. In DSME networks, this constraint is inherent, given that no failures happen during the negotiation [35]. For TSCH networks, such a fixed assignment is not necessarily required, but sharing will obviously lead to collisions, even if they might not be as severe as with CSMA/CA. Furthermore, the schedule should avoid using links with high path loss or highly fluctuating channel conditions to avoid occasional packet loss on the physical layer by means of an appropriate neighborhood management [36].

In large industrial plants, the frequencies of the used radio components are usually coordinated and monitored to avoid cross-interferences and as a security measure [37]. However, external interferences can often not be avoided completely, so TSCH and DSME can dynamically use multiple frequency channels. Apart from mitigating external interferences, this can also be used to increase the throughput by assigning the same time slot to multiple transceiver pairs on different frequency channel and is therefore also modeled in this paper. It can either be implemented as channel adaption where a fixed frequency channel is assigned at a given time or channel hopping where the channel to be used is iterated over a given sequence. Yet, in this paper, external interferences are not considered, so channel hopping with non-overlapping hopping sequences is equivalent to a fixed assignment as for channel adaption.

It is therefore concluded that in properly constructed TSCH and DSME networks without a lot of external disturbances, the occurrence of queue drops is the main factor that determines the maximum achievable network performance. Yet, in real-world networks, other losses can still occur, so Sect. 8.2 gives an outlook about how to consider these in the model.

As noted in the previous section, the M/D/1/K model is often used to model queues in the context of TDMA networks. However, it has two major weaknesses in the given scenario: Even if the traffic generation itself is modeled as Poisson distributed, this distribution does not necessarily hold for forwarding nodes, so the M/D/1/K model is not suitable for multi-hop networks. Furthermore, the service times are not necessarily deterministic but can diverge significantly, due to two effects: First, the schedule itself might be irregular, for example if multiple subsequent transmission slots are followed by a large idle phase. Secondly, even if the schedule is regular, the service time of packet that arrives at an empty queue is not constant, but depends on the time left until the next transmission slot. This effect is especially relevant in scenarios with low traffic. A proper model must take these effects into account.

System model

In the presented system model as illustrated in Fig. 1, the queue of every node is modeled as an instance of a Markov chain and they are linked to form a model of the full network. Packets are either forwarded as received or generated at the node. In both cases, the packets are pushed to a queue of fixed length or dropped if the queue is full. If there is at least one packet in the queue at the beginning of a transmission slot, a transmission attempt takes place.

Fig. 1

Illustration of the proposed model annotated with the notation introduced in the following sections

The calculations are based on fixed multi-hop schedules. A schedule can be calculated offline with algorithms such as the ones presented in Section 7. Alternatively, a schedule can be extracted from a real-world deployment or a simulator as presented in Section 6.4. In the following, the notation for such a schedule as used in the model is introduced.

For the presented model, a network of N nodes v₀,…,v_N−1 with a fixed schedule is considered. All slots have an equal time duration T _s . For TSCH, it is usually 10 ms. A slotframe consists of l _S slots and is repeated with an interval of T _s ·l _S . The schedule for node v _n is given as a tuple of transmission slots $\mathcal {T}_{n} = (t_{n,j})_{j =0, \ldots, \left \vert \mathcal {T}_{n}\right \vert -1 }$, sorted in ascending order, where $t_{n,j} \in \mathcal {T}_{n}$ if and only if the slot with the zero-based index t_n,j, that is 0≤t_n,j<l _S , is assigned to n for conflict-free transmission. Correspondingly, $\mathcal {R}_{n} = (r_{n,j})_{j=0,\ldots,\left \vert \mathcal {R}_{n}\right \vert -1}$ describes the reception slots. A slot is never in both $\mathcal {T}_{n}$ and $\mathcal {R}_{n}$, because a node can not transmit and receive at the same time. Furthermore, $\mathcal {K}_{n}(i)$ gives the respective reception node if $i \in \mathcal {T}_{n}$ or the respective transmission node if $i \in \mathcal {R}_{n}$. Figure 2 presents a simple example for these definitions.

Fig. 2

An exemplary schedule corresponding to N=3, l _S =3, $\mathcal {T}_{0} = ()$, $\mathcal {R}_{0} =(0,1)$, $\mathcal {K}_{0} (0) =1$, $\mathcal {K}_{0} (1) =1$, $\mathcal {T}_{1} =(0,1)$, $\mathcal {R}_{1} =(2)$, $\mathcal {K}_{1} (0) =0$, $\mathcal {K}_{1} (1) =0$, $\mathcal {K}_{1} (2) =2$, $\mathcal {T}_{2} =(2)$, $\mathcal {R}_{2} =()$ and $\mathcal {K}_{2} (2) =1$

For modeling frequency diversity, $\mathcal {C}_{v_{n}}(i)$ is introduced that specifies the channel to use in slot i by node v _n . The channel is an element of the set of channels $\mathcal {C}$. For IEEE 802.15.4 in the 2.4 GHz band, it consists of the numbers 11 to 26.

To check that a schedule is conflict-free and valid, the set

$$ \mathbf{L}_{i} = \left\{(n,\mathcal{K}_{n}(i))\,|\, i \in \mathcal{T}_{n}, \forall n\right\} $$

(1)

of all links that are active during slot i has to be considered for all slots. Each node can only be transmitter or receiver during a slot

$$\begin{array}{*{20}l} \left(i \in \mathcal{T}_{n} \Rightarrow i \notin \mathcal{R}_{n}\right) \wedge \left(i \in \mathcal{R}_{n} \Rightarrow i \notin \mathcal{T}_{n}\right). \end{array} $$

(2)

Furthermore, the links have to be unique and consistent, $\forall i \in \mathcal {T}_{n}$ it must hold

$$\begin{array}{*{20}l} i \in \mathcal{R}_{\mathcal{K}_{n}(i)} \wedge \left(\mathcal{K}_{n}(i)=k \Leftrightarrow \mathcal{K}_{k}(i)=n\right). \end{array} $$

(3)

However, this alone is insufficient. Other potential conflicts are shown in Fig. 3. The most obvious constellation is $\mathcal {R_{S}}$: The reception at w₁ can be disturbed by a transmission from v₂ to w₂ if v₂ is in the neighborhood of w₁. If acknowledgments are used, conflicts between acknowledgments and the data packets have to be taken into account, too, represented by the other constellations. For a link (v₁,w₁)∈L _i , the potentially disturbing links are

$$\begin{array}{*{20}l} \begin{aligned} & {\boldsymbol{\mathcal{D}}}_{i,(v_2,w_2)}= \\ & \left\{ (v_2, w_2) \in {\mathbf{L}}_i \,|\, \ensuremath{D}\! \left (v_1, v_{2}\right) \wedge v_1 \neq v_2 \right\}\\ & \cup \left\{ (v_2, w_2) \in {\mathbf{L}}_i \,|\, \ensuremath{D}\! \left (w_1, v_{2}\right) \wedge v_1 \neq v_2 \right\}\\ & \cup \left\{ (v_2, w_2) \in {\mathbf{L}}_i \,|\, \ensuremath{D}\! \left (v_1, w_{2}\right) \wedge v_1 \neq v_2 \right\}\\ & \cup \left\{ (v_2, w_2) \in {\mathbf{L}}_i \,|\, \ensuremath{D}\! \left (w_1, w_{2}\right) \wedge v_1 \neq v_2 \right\}. \end{aligned} \end{array} $$

(4)

Fig. 3

Possible conflicts between two transmissions

Here, D(v,w) denotes that a transmission of v might disturb a reception at w. One possibility to ensure a conflict-free schedule is to have

$$\begin{array}{*{20}l} {\boldsymbol{\mathcal{D}}}_{i, \cup} = \bigcup_{l,\in\, {\mathbf{L}}_i} {\boldsymbol{\mathcal{D}}}_{i, l} \end{array} $$

(5)

empty for all slots. Hence, no concurrent transmissions take place in the neighborhood. By exploiting frequency diversity, it is, however, sufficient to make sure that no concurrent transmissions take place on the same frequency channel at the same time, i.e., one has to make sure that for all slots, 0≤i<l _S holds

$$\begin{array}{*{20}l} \mathcal{C}_{v_{1}}(i) \neq \mathcal{C}_{v_{2}}(i), \forall (v_{2},w_{2}) \in \boldsymbol{\mathcal{D}}_{i,\left(v_{1},w_{1}\right)} \end{array} $$

(6)

Queue model

Before modeling multi-hop communication, this section considers the viewpoint of a single node v _n and its local behavior. In the multi-hop model, every node of the network will get its own instance of this model. The main focus is the finite queue, so after this section, we will be able to calculate packet drops due to a full queue as well as the expected queuing delay. The inputs into this model are the probabilities for incoming traffic and the distribution of the reception slots $\mathcal {R}_{n}$ and transmission slots $\mathcal {T}_{n}$. Beyond the usage in a multi-hop model as introduced in the next section, the model in this section is also applicable in other setups, such as single-hop networks or for evaluating the performance of a single node as presented in Section 5.7.

The following policy is used to model the queue:

• The queue can hold at most K packets.

• The number of packets in the queue at the beginning of a slot is denoted as q.

• New packets can arrive at any time during a slot. At most, K−q packets are accepted during a slot.

• A packet is removed from the queue at the end of a slot if and only if it is a transmission slot and the packet was already in the queue at the beginning of the slot, i.e. q>0, modeling the transmission process itself.

The queue of a node is modeled as a discrete-time Markov chain with the states

$$\begin{array}{*{20}l} \boldsymbol{\Sigma}_{n} = \left\{ \sigma ^{n}_{q,i} \, \middle |\, 0 \leq q \leq K, 0 \leq i < l_{S} \right\}. \end{array} $$

(7)

In order to account for the irregular schedule and for modeling the service time more accurate, the state of the queue $\sigma ^{n}_{q,i}$ does not only account for the number q of packets in the queue as for usual M/D/1/K models, but also the current position within the slot schedule i. As shown in Fig. 4, the model presented here is a super set of the M/D/1/K model for l _S =1 and $\mathcal {T}_{n}=(0)$. Figure 5 depicts a more complex schedule, the corresponding states and the transitions as follows.

Fig. 4

A M/D /1/ K model in the syntax introduced in this paper with K=3, l _S =1 and $\mathcal {T}_{n} = (0)$

Fig. 5

Schedule for l _S =5, $\mathcal {T}_{n} =(1,4)$ and corresponding state diagram for K=2

Traffic model

The number of arriving packets during a time slot i, that is the generated and received traffic, is described by the random variable A_n,i. For the purpose of the queuing model, it can follow an arbitrary distribution that can optionally depend on the current state. A popular option is to model the traffic as Poisson distributed with a mean packet rate λ_n,i. For this, the random variable Π_n,i is introduced with the probability distribution

$$\begin{array}{*{20}l} P\! \left (\boldsymbol{\Pi}_{n,i} = k\right) = \frac{\lambda_{n,i}^ k}{k !} e^{- \lambda_{n, i}}. \end{array} $$

(8)

The probability for at leastk packets is given by

$$\begin{array}{*{20}l} P\! \left (\boldsymbol{\Pi}_{n,i} \geq k\right) = 1 - \sum_{j =0}^{k -1} P\! \left (\boldsymbol{\Pi}_{n,i} = k\right). \end{array} $$

(9)

Since only a single packet can be received per slot, we extend this model by combining it with a Bernoulli distribution with the probability β_n,i that a single packet is arriving in addition to the Poisson traffic in slot i, so we get

$$ \begin{aligned} &P\left (\mathbf{A}_{n,i} = k\right) =\\ &\quad\left\{ \begin{array}{lll} \left (1- \beta_{n, i}\right) \cdot P \left (\boldsymbol{\Pi}_{n,i} = 0 \right)& k = 0\\ \left (1\,-\, \beta_{n, i}\right) \cdot P\left (\boldsymbol{\Pi}_{n,i} = k\right)& \text{otherwise}\\ +\beta_{n, i}\cdot P \left (\boldsymbol{\Pi}_{n,i} = k -1 \right)& & \end{array} \right. \end{aligned} $$

(10)

and

$$ \begin{aligned} &P \left (\mathbf{A}_{n,i} \geq k\right) =\\ &\quad\left\{ \begin{array}{lll} 1& k = 0\\ \left (1- \beta_{n, i}\right) \cdot P \left (\boldsymbol{\Pi}_{n,i} \geq k\right)&\text{otherwise}.\\ +\beta_{n, i}\cdot P\left (\boldsymbol{\Pi}_{n,i} \geq k -1 \right) & \end{array} \right. \end{aligned} $$

(11)

Since all l _S slots occur with the same probability, the expected total number of packets per slotframe is calculated as the sum of the expected number of the A_n,i as derived in Appendix A as

$$\begin{array}{*{20}l} \begin{aligned} \mathcal{A}_{n} &= \sum_{i =0}^{l_{S} -1} E\left [\mathbf{A}_{n, i}\right]\\ &= \sum_{i =0}^{l_{S} -1 }\left (1- \beta_{n, i}\right) \cdot \lambda_{n, i}+ \beta_{n, i} \cdot \left (\lambda_{n, i}+1 \right). \end{aligned} \end{array} $$

(12)

Queuing probability

The random variable Q _n is the number of accepted packets that are inserted into the queue per slot. Since at most K−q packets can be inserted into the queue, its probability distribution, i.e., the probabilities of queuing k packets, is calculated as

$$ \begin{aligned} &P\! \left (\mathbf{Q}_{n} = k \, \middle |\, \sigma ^{n}_{q, i}\right) =\\ &\quad\quad\quad\left\{ \begin{array}{lll} P\! \left (\mathbf{A}_{n,i} = k\right) & k < K - q \\ P\! \left (\mathbf{A}_{n,i} \geq K- q\right) & k = K - q\\ 0 & \text{otherwise.} \\ \end{array} \right. \end{aligned} $$

(13)

Obviously, the difference of A_n,i and Q _n is the number of packets dropped due to a full queue.

Transition probabilities

P(ξ→ζ) with ξ,ζ∈Σ _n is the probability of going from ξ to ζ in one step. Transition probabilities not listed are zero, in particular for transitions with non-consecutive slots. Furthermore, % denotes the modulo operation. So for 0≤q≤K and 0≤i<l _S , all possible transitions are specified by

$$ \begin{aligned} &P\left (\sigma ^{n}_{q,i} \to \sigma ^ n_{\max\left (q- \tau_{n,i},0 \right) + k, \,(i +1) \% l_{S}}\right)\\ &\qquad\qquad\qquad\qquad\!\!\! = P\left (\mathbf{Q}_{n} = k \middle | \sigma ^{n}_{q, i}\right) \end{aligned} $$

(14)

where

$$ \tau_{n,i} =\left\{ \begin{array}{ll} 1 & i \in \mathcal{T}_{n}\\ 0 & i \not\in \mathcal{T}_{n} \end{array} \right. $$

(15)

and k is the number of arriving or newly generated packets during the time slot that iterates from 0 to K−q. While this gives a complete description of the model, two corner cases should be mentioned explicitly for clarity. With an empty queue and A_n,i=0, that is no arriving packets, the model stays in the states with q=0. With a full queue, it is not possible to insert further packets into the queue so Q _n =0, and thus, the single possible transition out of the states with q=K has the probability

$$\begin{array}{*{20}l} P\! \left (\mathbf{Q}_{n} = 0 \, \middle |\, \sigma ^{n}_{K, i}\right) = P\! \left (\mathbf{A}_{n,i} \geq 0 \right) = 1, \end{array} $$

(16)

either to a state with q=K−1 if the current slot is a transmission slot or else to a state with q=K.

As shown in Appendix B, these probabilities can be used to calculate the stationary distribution of this Markov chain. This results in the probability c_n,q,i of being in state $\sigma ^{n}_{q,i}$ in the stationary case.

Transmission probability

From the c_n,q,i, the probability that a successful transmission takes place within a given slot is calculated from the complementary probability of being in a state with empty queue (q=0) as

$$\begin{array}{*{20}l} \mu ^{\textsc{TX}}_{n,i} = \tau_{n,i} \cdot \left (1 - \frac{c_{n,0, i}}{\sum_{q =0}^{K} c_{n,q, i}}\right). \end{array} $$

(17)

Packet acceptance probability

If q packets are in the queue, K−q packets can be inserted into the queue, so the expected number of accepted packets in state $\sigma ^{n}_{q,i}$ is

$$\begin{array}{*{20}l} \begin{aligned} E \left [ \mathbf{Q}_{n} \, \middle |\, \sigma ^{n}_{q, i}\right ]=& P\left (\mathbf{A}_{n, i} \geq K-q \right) \cdot (K-q) \\ &+ \sum_{k =0}^{K -q -1} P \left (\mathbf{A}_{n,i} = k\right) \cdot k. \end{aligned} \end{array} $$

(18)

Since the events of being in a state are mutually exclusive and exhaustive, E[Q _n ] can be calculated from this according to the law of total expectation as

$$\begin{array}{*{20}l} E \left [ \mathbf{Q}_{n}\right ] = \sum_{q =0}^{K} \sum_{i =0}^{l_{S} -1} c_{n,q,i} \cdot E \left [ \mathbf{Q}_{n} \middle | \sigma ^{n}_{q, i}\right ] \end{array} $$

(19)

and the overall expected number of packets per slotframe is thus l _S ·E[ Q _n ]. Together with the overall expected number of packets arriving at the queue, the overall packet acceptance probability is calculated as

$$\begin{array}{*{20}l} P_{\text{accept},{n}} = \frac{l_{S}\cdot E[\!{\boldsymbol{Q}_{n}}]}{\boldsymbol{\mathcal{A}}_{n}}. \end{array} $$

(20)

Queuing delay

The number of time steps it takes until an arriving packet is transmitted is described by the random variable D _n . For convenience, we define

$$\begin{array}{*{20}l} \phi_{n} (i) =\left\{ \begin{array}{ll} \left\vert\mathcal{T}_{n}\right\vert -1 & i \leq t_{n,0} \,\, \vee\\ & i > t_{n, \left\vert\mathcal{T}_{n}\right\vert -1 }\\ g : t_{n,g} < i \leq t_{n,g+1 }\!\!\! & \text{else}, \end{array} \right. \end{array} $$

(21)

as the index of the transmission slot preceding i and

$$\begin{array}{*{20}l} \delta\!\left(i,j\right) =\left\{ \begin{array}{ll} j - i & j \geq i\\ j - i + l_{S} & \textrm{otherwise,} \end{array} \right. \end{array} $$

(22)

as the number of slots when going from i to j. Then, given the event of being in state $\sigma ^{n}_{q,i}$ after accepting a packet and pushing it in the queue, D _n is calculated as

$$\begin{array}{*{20}l} \boldsymbol{D}_{n}\!\left(\sigma^{n}_{q,i}\right) = &f_{q} l_{S} + 1 + \delta\!\left(i,\,\,t_{{n},{\left(\phi_{n}(i) + q\right) \% |\mathcal{T}_{n}|}}\right) \end{array} $$

(23)

where $f_{q} = \left \lceil \tfrac {q}{|\mathcal {T}_{n}|}-1\right \rceil$ is the number of iterations over the full slotframe, the summand 1 accounts for the transmission slot itself and the last summand for the remaining packets after $f_{q}\cdot |\mathcal {T}_{n}|$ packets were transmitted. The expected delay for a packet arriving at an arbitrary point in time can be calculated from this as

$$\begin{array}{*{20}l} \begin{aligned} E [\!{\boldsymbol{D}_{n}}] &= \sum\limits_{q=0}^{K}\sum\limits_{i=0}^{l_{S}-1} c_{n,q,i} \cdot \boldsymbol{D}_{n}\!\left(\sigma^{n}_{g,h}\right)\\ \textrm{with }\sigma^{n}_{g,h} &= \sigma^{n}_{\max\left(q-\tau_{n},0\right)+1,\,\,(i+1) \% l_{S}} \end{aligned} \end{array} $$

(24)

considering the additional transition to account for the arriving packet itself.

Single node evaluation

At first, the results for the queuing model on a single node are analyzed without considering the effects of multi-hop networks. The purpose of this section is to demonstrate the benefit of the refined model in contrast to a simple M/D/1/K model by comparing them with an event-based simulation. For this, an exemplary schedule is shown in Fig. 6 together with the two approximations presented in the following. The schedule is construed in a way that on average the same amount of traffic enters and leaves the system. However, due to a finite queue length of K=5, packet drops can still occur. In Fig. 7, the following four evaluations are compared:

• In the M/D/1/K model, there is no option to specify multiple slots and only an overall packet rate rate can be specified.

  Fig. 6

  

  Exemplary schedule for evaluating the queuing model together with two approximations, all with the same $\boldsymbol {\mathcal {A}}_{n} = 1$

  Fig. 7

  

  Comparing the full presented model with a simulation and simpler models: An M/D/1/K model and a model that considers the TX and Idle slots, but distributes the reception over all slots

• In the Distributed approximation, the transmission and idle slots are modeled correctly, but incoming traffic is modeled to follow a Poisson distribution and not a Bernoulli distribution. Furthermore, it is distributed over all slots. To get the same $\boldsymbol {\mathcal {A}}_{n}$ of 1 as in the actual scenario, all λ_n,i are set to $\tfrac {1}{l_{S}} = 0.2$. This scenario allows to assess the deficiency of not including the β_n,i in the model.

• The Full Model scenario additionally models the reception slot correctly as presented.

• The Simulation is a discrete event simulation of the queue based on SimPy [38] that implements the policy as presented at the beginning of Section 5. The mean and 95% confidence interval are shown of 10 runs with 10,000 packets.

While the full model matches the simulation very well, the probability of dropping packets due to a full queue is higher in the distributed scenario, since the variance of the packet reception is higher, even though the average reception rate is equal. In fact, for g_up=0 and at least as much TX slots as RX slots, the full model will never indicate a packet loss (P_accept,n=100%), while in the simplified models there is a non-zero probability that more packets are received in a slotframe than sent, eventually leading to packet loss.

The M/D/1/K model gives even less accurate results, because the service time is constant and equal to l _S ·T _s , while in the distributed and the full model case, packets might be processed faster if the queue is empty and they arrive during the later slots.

Queue distribution

In this section, we take a deeper look into the distribution of the queue level and its influence on the probability of accepting a packet for different traffic loads.

For this, a scenario with K=10, l _S =5 and a single transmission slot is considered.

In Fig. 8, the probability distribution of the queue level is plotted for different traffic loads $\boldsymbol {\mathcal {A}}_{n}$. For this, the probability of being in a state with q packets in the queue is calculated as

$$\begin{array}{*{20}l} c_{n,q,\star} = \sum\limits_{i=0}^{l_{S}-1} c_{n,q,i}. \end{array} $$

(25)

Fig. 8

Distribution of queue levels for K=10, l _S =5 and a single transmission slot

In the first scenario, the traffic load is distributed over the λ_n,i, corresponding to a node that does only generate but does not forward traffic, and in the second scenario, it is distributed only over the β_n,i, corresponding to a node that only forwards traffic generated by other nodes.

For a low rate of $\boldsymbol {\mathcal {A}}_{n}=0.5$, the queue is empty most of the time with a decaying probability of having more packets in the queue. The probabilities for queue levels larger than 3 are negligible, so the probability of accepting new packets is P_accept,n=1.00 for both scenarios.

For a medium rate of $\boldsymbol {\mathcal {A}}_{n}=1$, where on average one packet is sent and received every slotframe, the distribution is nearly constant. This leads to a packet acceptance probability of P_accept,n=0.95 for the first and P_accept,n=0.96 for the second scenario. Interestingly, the probability for q=K is significantly smaller than the others.

This is due to the policy that in a transmission slot with q=K, no new packets can be pushed to the queue, but certainly, a packet will be sent, i.e., q=K−1 in the next slot. However, for all other queue levels, it is possible to maintain the same queue level by sending and receiving one packet. That is even more apparent for $\boldsymbol {\mathcal {A}}_{n}=1.5$ where a queue level larger than q=5 most of the time leads to a packet acceptance probability of P_accept,n=0.67 for both scenarios. Having q=K more often than other queue levels is only possible for very high rates of $\boldsymbol {\mathcal {A}}_{n} = 2.5$. Here, the node is highly congested and a P_accept,n=0.40 is achieved in both scenarios.

When comparing the scenarios, we see that the difference is not very large, but in the second scenario, the probabilities are shifted to the boundaries. This evaluation also shows that often only a small part of the queue is actually utilized. For $\boldsymbol {\mathcal {A}}_{n} = 0.5$, a maximum queue size of K=3 would be sufficient to prevent packet loss. Also for $\boldsymbol {\mathcal {A}}_{n} = 2.5$, this would not change the packet acceptance probability. The probability distribution would shift to the left, but does not change its overall shape relative to the upper bound. Of course, the queuing delay would be reduced by this. The bottom line is that the maximum queue length K has the highest influence in scenarios where the amount of incoming and outgoing traffic is similar.

Multi-hop model

To model multi-hop communication, every node in the network gets its own instance of the previously presented model. They are then linked to get a full network for allowing to determine network-wide metrics.

Traffic generation and forwarding

For the purpose of this paper, we assume a data-collection scenario with sink v₀. Every node, except v₀, generates packets with exponentially distributed intervals with mean I_up that are to be forwarded to v₀ via a routing tree. The base time unit is the slot length T _s , so the generation rate is

$$\begin{array}{*{20}l} g_{\text{up}} = \frac{T_{s}}{I_{\text{up}}}. \end{array} $$

(26)

In the following, only homogeneous traffic generation is considered, so we set

$$\begin{array}{*{20}l} \lambda_{n,i} =\left\{ \begin{array}{ll} g_{\text{up}} & v_{n} \neq v_{0}\\ 0 & v_{n} = v_{0}, \end{array} \right., \forall 0 \leq i < l_{S}. \end{array} $$

(27)

but other traffic patterns can be obtained by choosing individual values for λ_n,i.

Besides the traffic generation itself, there is a probability $\mu ^{\textsc {RX}}_{n, i}$ of receiving a packet from a neighbor in a reception slot to be forwarded to the sink. This value is calculated from the probability $\mu ^{\text {TX}}_{\mathcal {K}_{n}(i),i}$ that the neighbor $\mathcal {K}_{n}(i)$ is transmitting in the given slot as

$$\begin{array}{*{20}l} \mu ^{\textsc{RX}}_{n, i}=\left\{ \begin{array}{ll} \mu ^{\textsc{TX}}_{\mathcal{K}_{n} (i),i} & i \in \mathcal{R}_{n}\\ 0 & \text{else}. \end{array} \right. \end{array} $$

(28)

The forwarding is modeled by the Bernoulli part of the traffic model since at most one packet can be received per slot, so β_n,i=μn,iRX.

Network throughput

The throughput of the network corresponds to the number of packets arriving at the sink v₀ per time. Since the sink does not generate traffic itself, the throughput can be calculated as

$$\begin{array}{*{20}l} \frac{\mathcal{A}_{0}}{T_{s}} = T_{s}^{-1} \cdot \sum_{i =0}^{l_{S} -1}\mu ^{\textsc{RX}}_{0,i}. \end{array} $$

(29)

End-to-end metrics

The packet delivery ratio (PDR) $\mathfrak {R}_{\text {up},{n}}$, that is the probability that a packet originating from node v _n is finally received by the sink v₀, is given by the product of P_accept,n over the path

$$\begin{array}{*{20}l} \mathfrak{R}_{\text{up},n} =\left\{ \begin{array}{ll} \mathfrak{R}_{\text{up}, p_{n}} \cdot P_{\text{accept},n} & v_{n} \neq v_{0}\\ 1 & v_{n} = v_0, \end{array} \right. \end{array} $$

(30)

where p _n is the parent of v _n in the routing tree.

Similarly, the end-to-end delay $\mathfrak {D}_{\text {up},{n}}$ is calculated as

$$\begin{array}{*{20}l} \mathfrak{D}_{\text{up},n} =\left\{ \begin{array}{ll} \mathfrak{D}_{\text{up}, p_{n}} + E\! \left [ \mathbf{D}_{n}\right ] & v_{n} \neq v_{0}\\ 0 & v_{n} = v_0. \end{array} \right. \end{array} $$

(31)

Comparing model and simulation

After evaluating the model for a single node only, we now build up a full multi-hop network to analyze its performance. An analytical model is very valuable for appreciating the validity of simulation results. Thus, we use the model to compare its results to the Contiki implementation of the Orchestra scheduling [6] for TSCH running in the COOJA simulator [39] with a maximum queue length of K=16. It is evaluated for a data-collection scenario in a concentric topology with 19 nodes as shown in Fig. 9. Sender-based Dedicated (SBD) Orchestra Slots are used, where every node has a dedicated transmission slot and thus, $\boldsymbol {\mathcal {D}}_{i,\cup } = \emptyset$ and so the schedule is conflict-free and valid. Furthermore, the slotframe length is minimized to maximize the throughput under the given constraint.

Fig. 9

A concentric network with 19 nodes

The results in Fig. 10 show the end-to-end packet delivery ratio (PDR) averaged over the nodes in the outer circle as well as the end-to-end delay. In the simulation, every node sends packets to the center with exponentially distributed intervals. After a warm-up phase of 15 minutes, 100 packets are monitored if they arrive and how long they take. The resulting mean and the 95% confidence interval over 5 runs are shown in the plot.

Fig. 10

Comparing a Sender-based Dedicated Orchestra Slot Schedule in the Cooja Simulator and the model with N=19 and K=16. The average end-to-end packet delivery ratio and end-to-end delay are shown for the nodes in the outer circle

While the simulation uses the Routing Protocol for Low power and Lossy Networks (RPL) [40], the analytical model uses a predefined tree routing as in [41]. The PDR $\mathfrak {R}_{\text {up},{n}}$ and the end-to-end delay $\mathfrak {D}_{\text {up},{n}}$ are shown in the plot. Obviously, no confidence intervals are shown because no randomness is included in the calculation. The results show a good conformance between model and simulation and demonstrate the applicability of the analytical model for multi-hop networks as well as the validity of the Contiki simulation for the given scenario. The existing differences can be mainly traced back to the varying routing trees in the simulation runs.

Comparing multi-hop schedules

The analytical model is also very useful to assess new scheduling algorithms without the need to implement them for a full simulation stack. Orchestra is an appropriate schedule for many applications, especially since it requires no management traffic, but its main disadvantage is its inability to handle different traffic loads efficiently. In the given scenario, the inner nodes have to handle the aggregated traffic of all nodes in their sub tree, so they should be able to use more slots than the leafs.

In this section, two traffic-aware schedules are introduced for the data collection scenario presented in the previous section for a given tree. The main idea is that every node has enough slots for the traffic generated at that node as well as for forwarding the traffic of its children, assuming equal traffic distribution. More formally, if γ _n denotes the number of proper descendants of node v _n , every node will get γ _n +1 transmission slots towards the sink. Algorithm 1 describes a distributed algorithm to calculate γ _n for every node and also to make every node aware of the number of proper descendants of their children. For this, every node keeps the variables v _n .γ for the number of proper descendants of this node n and for each child s its number of proper descendants v _n .γ _s . As before, v₀ denotes the sink node and p _n the index of the parent of node v _n .

The algorithm is basically a depth-first search. It is started by sending FORWARD to v₀. After initializing v₀.γ, FORWARD messages are sent until a leaf is reached. After this, a BACKTRACK message is sent back to the parent. The parent increments its counters before continuing the depth-first search. After a subtree is fully handled, the number of nodes in this subtree is sent back to the parent in a BACKTRACK message.

In the presented distributed algorithms, we assume all messages arrive correctly and in order. Otherwise, the algorithms would get a lot more complex. Associated problems and solutions for unreliable message exchange when building schedules are presented in [35].

The schedules presented in this section are meant to demonstrate the applicability of the analytical model and the advantage of traffic-aware schedules. However, they are not directly applicable to real-world applications due to their inability to cope with changes in the network topology and the traffic load without a complete recalculation. Yet, due to their simplicity, they serve as a starting point for more sophisticated traffic-aware schedules.

Traffic-aware schedule (single-channel)

The first traffic-aware scheduling algorithm as shown in Algorithm 2 is based on the scheduling algorithm type III from [42]. As for Orchestra’s Sender-based Dedicated Slots, only one node in the entire network is sending at every given point in time, so again $\boldsymbol {\mathcal {D}}_{i,\cup } = \emptyset$. It is therefore denoted as single-channel scheduling, though, similar to Orchestra, channel hopping could be used to mitigate external interferences.

The algorithm is started by sending TRACK(1) to v₀ after every node has performed the initialization. The algorithm is a depth-first search again, but instead of sending back the number of nodes in the subtree, multiple transmission slots are reserved towards the parent, one for handling the traffic of every proper descendant and one for the traffic generated at that node (γ _n +1). The transmission slot is recorded in the local $\mathcal {T}_{n}$ and a message is sent to the parent $v_{p_{n}}$ to record the reception slot in its $\mathcal {R}_{p_{n}}$. When sending the TRACK message back to the parent, the number of already assigned slots is sent along as an offset for the next slot assignment.

The overall number of slots l _S in a slotframe for this schedule is calculated as

$$\begin{array}{*{20}l} l_{S} = 1 + \sum_{n = 1}^{N-1} \gamma_{n} + 1, \end{array} $$

(32)

since every node (apart from v₀) performs γ _n +1 slot allocations. The additional slot is the first slot of the slotframe that is usually left free for shared communication in most TSCH implementations, for example for management traffic. This also holds for the used Orchestra implementation.

Traffic-aware schedule (multi-channel)

In general, it is not required that only one node is sending at every point in time. Two pairs of nodes that are sufficiently apart can send at the same time without interference. Secondly, nodes can communicate on different channels to avoid interference. This spatial and frequency diversity can be exploited to shorten the slotframe and therefore increase the throughput and lower the latency.

A corresponding distributed algorithm is given in Algorithm 3. In contrast to the previous algorithm, where the child determines the transmission slots towards the parent, in this algorithm, the parent determines the reception slots in which it will expect the child to send. It is started by sending TRACK to v₀. As in the previous algorithm, every node requires γ _n +1 transmission slots. This is also calculated in line 9 for a child u when a node is first handled by its parent. Afterwards, σ=v _n .γ _u +1 slot allocations are performed and the loop is always finished in line 20 if we assume there are always enough channels (see below, otherwise line 14 would fail) and l _S is chosen as follows. For every allocation, a previously unused time slot is searched, recorded at sender and receiver and the used channel is blocked in the 2-hop neighborhood. For this, every node maintains a set of blocked channels for every slot ${\mathcal {B}}_{n}(i)$. This search will always be successful if we choose the slotframe length as

$$\begin{array}{*{20}l} l_{S} = 1 + \max_{n\,=\,0,\ldots,N-1}\left\{ \begin{array}{ll} 2\cdot\gamma_{n} + 1 & v_{n} \neq v_{0}\\ \gamma_{n} & v_{n} = v_{0} \end{array} \right., \end{array} $$

(33)

because the root requires that the slotframe has at least one slot for receiving (potentially forwarded) traffic from every proper descendant. For all other nodes, this holds, too, but in addition, the same number of slots is required for forwarding and one slot for transmitting the traffic generated at that node. Again, the additional slot is the shared first slot in the slotframe. Since we assume enough channels are available to avoid conflicts, the slotframe length corresponds to the requirement of the node with the largest required slotframe length.

In contrast to the other algorithms, transmissions can take place simultaneously without or with very little interference by assigning a dedicated channel or a non-conflicting channel hopping sequence to every conflicting link in a neighborhood. In the presented algorithm, this is ensured by signaling every slot assignment to all neighbors N _n of the sender and the receiver as well as the parents of the neighbors. Thereby, the four interference constellations in Fig. 3 are avoided. This idea is similar to the slot allocation handshake of DSME as well as the DeBraS scheduling algorithm [33] and is especially important for dense networks.

In general, this is a graph coloring problem where L _i is the set of vertices and $\boldsymbol {\mathcal {D}}_{i,\cup }$ is the set of edges. Executing the presented heuristic algorithm for the first slot results in the coloring shown in Fig. 11. Here, only three channels are required, so the 16 channels available for IEEE 802.15.4 in the 2.4 GHz band are more than sufficient. In general, however, no upper bound can be given for the number of required channels, because the conflict graph is not necessarily planar. Figure 12 shows an example that requires five channels and could be extended to an arbitrary number of channels. While in general finding a valid coloring with at most 16 channels is not ensured, it is usually more than sufficient in real-world applications.

Fig. 11

A possible coloring for a set of conflicting links

Fig. 12

A schedule that requires a coloring with five colors

Evaluation

In Fig. 13, the achievable throughput for different schedules is compared for multiple scenarios, i.e., for a network of N=19 and N=37 nodes in a concentric topology and for maximum queue sizes of K=6 (dashed) and K=16 (solid). For this, the number of packets received by the sink v₀ is plotted over the packet generation rate of every node. For low rates, the network is able to handle the complete traffic, so the throughput rises linearly. For high rates, the networks is saturated and increasing the rate does not increase the throughput anymore.

Fig. 13

Throughput for the Sender-based Dedicated Orchestra Schedule and the two presented traffic-aware schedules over the packet sending interval for a network of N=19 and N=37 nodes aligned in concentric circles. The solid lines are for K=16, the dashed for K=6. In addition, the table presents schedule properties at the root node v₀

In general, the traffic-aware schedules provide significantly more throughput than Orchestra, while the multi-channel schedule has an even higher throughput than the single-channel schedule. The higher throughput can be explained by considering that nodes higher in the tree that need to handle more traffic have more slots compared to nodes with low traffic demand.

It is also apparent in Fig. 13 that the maximum queue length K is most significant in the transition section, in conformance to Section 5.8. Compared to the applied schedule, it has a lower impact on the throughput, but the difference is larger for N=37 than for N=19 due to the higher number of hops.

Also, when comparing different network sizes, we see that the saturation is reached for smaller rates. This is expected, because the same packet generation rate applied at more nodes leads to more overall traffic. Secondly, the single-channel schedule and even more Orchestra SBD show a significantly lower throughput, because having more nodes requires a higher number of slots during which the sink is idle. This does not hold for the multi-channel schedule, because spatial reuse is possible.

In the table of Fig. 13, the slotframe lengths are given together with the number and ratio of reception slots at the sink v₀. For Orchestra SBD, every neighbor of v₀ has only one slot, so v₀ has a long phase of inactivity. For the traffic-aware schedules, the inner nodes get more slots because of their higher traffic load. Thus, the sink can receive more packets per slotframe resulting in a higher throughput. The missing slot in the last row is due to the shared slot. In the multi-channel schedule, the sink can even potentially receive data traffic in every slot, apart from the first one. This comparison again explains the difference in throughput for the different network sizes, because comparative to the network with 19 nodes, the RX ratio is significantly lower for N=37 and the Orchestra SBD and single-channel schedules, while it is even larger for the multi-channel schedule where the shared slot has a lower impact due to the longer slotframe length.

Possible extensions

IEEE 802.15.4 DSME

While the focus of this paper is TSCH due to its higher flexibility and broader usage, the model can easily be applied to DSME, too. The main difference between TSCH and DSME is the availability of distributed management procedures for setting up schedules in the latter, but since the presented model analyzes schedules in a steady state, the only major difference is the slot structure. In TSCH, the time slots can be arbitrarily dedicated as contention-free and contention-access slots. DSME has a less flexible structure. It consists of one beacon slot, 8 contention-access slots, and 7 contention-free slots aligned in a fixed, yet configurable, repeated sequence. The contention-access phase is usually used for management, similar to, but longer than the extra slot used in this paper. Secondly, the slots are usually shorter, because TSCH requires some extra time in every slot due to its time synchronization procedure, while DSME uses a dedicated beacon slot (see [43] for details). Therefore, the timing of the model has to be adapted and the schedule has to account for the slots not used for contention free communication.

Furthermore, due to the less flexible structure, a scheduling algorithm such as Algorithm 3 would lead to excess slots that are not utilized. Therefore, a more elaborate algorithm is required to achieve optimal performance by evenly distributing the excess slots.

Other sources of packet loss

As outlined in Section 3, most packet losses in TSCH and DSME networks are queue drops. However, in practice, other sources of packet loss are inevitable. The consideration of packet loss during the transmission can be integrated in the presented model by replacing Eq. (28) with

$$ \mu ^{\textsc{RX}}_{n, i}=\left\{ \begin{array}{ll} \mu ^{\textsc{TX}}_{\mathcal{K}_{n} (i),i} \left (1- \text{PER}_{b,(\mathcal{K}_{n}(i),n)}(t) \right)& i \in \mathcal{R}_{n}\\ 0 & \text{else}, \end{array} \right. $$

(34)

where $\text {PER}_{b,(\mathcal {K}_{n} (i),n)}(t)$ is the probability that a packet transmission with b byte fails at time t.

An example for a time-independent calculation of PER_b,(m,n) can be found in [41] and in [44] a model for Rayleigh-lognormal fading in IEEE 802.15.4 networks is given. Time dependence is required for accurate modeling of external interferences or fading channels, but that is out of the scope of this paper.

Furthermore, since the model assumes no losses on the physical layer, retransmissions are not considered. If other sources of packet loss are considered, retransmissions have to be integrated in the model by adding transitions in the Markov chain for maintaining the queue level after a failed transmission. For the considered scenario, these transitions would never be taken and thus not change the results.

Conclusions

The paper presents an analytical approach for the assessment of wireless mesh networks that use a collision-free TDMA. A queuing model based on a Markov chain is proposed that models forwarding traffic and irregular slot schedules accurately, in contrast to the well-known M/D/1/K model. These models are linked together to build up a multi-hop model of the whole network for calculating packet delivery ratio, end-to-end delay, and throughput.

The results demonstrate the increased accuracy compared to the M/D/1/K model and illustrate the effect of a finite queue by showing the queue level distribution. For evaluating the multi-hop model, a data-collection scenario is applied. The analytical model is compared to a simulation of the Orchestra schedule, showing good conformance. Finally, two distributed traffic-aware scheduling algorithms are presented. The higher throughput achieved by traffic-awareness is demonstrated and the influence of the maximum queue length is shown.

The calculations were conducted by means of the Portable, Extensible Toolkit for Scientific Computation (PETSc) [45, 46], the simulations with COOJA [39] and SimPy [38]. The open source implementation of the models can be accessed at [7].

Overall, the proposed analytical model, together with its software implementation, is a useful tool for testing new ideas while developing new slot schedules. It is also very helpful for practitioners who want to estimate the performance of wireless mesh networks.

Appendix A: Expected value of A _n,i

The expected value of A_n,i is calculated as

$$ \begin{aligned} & E\! \left [ \mathbf{A}_{n, i}\right ] = \sum_{k =0}^ \infty k\cdot P\! \left (\mathbf{A}_{n,i} = k\right) \\ & \begin{aligned} = \sum_{k =1}^ \infty k \left (\left (1\,-\, \beta_{n, i}\right) \cdot P\! \left (\boldsymbol{\Pi}_{n,i} = k\right) \right.\\ \left.+ \, \beta_{n, i}\cdot P\! \left (\boldsymbol{\Pi}_{n,i} = k -1 \right) \right) \end{aligned}\\ & = \left (1- \beta_{n, i}\right) \cdot \left (\sum_{k =1}^ \infty k \cdot P\! \left (\boldsymbol{\Pi}_{n,i} = k\right) \right) \\ & \quad + \beta_{n, i}\cdot\sum_{k =1}^ \infty k \cdot P\! \left (\boldsymbol{\Pi}_{n,i} = k -1 \right) \end{aligned} $$

(35)

so with

$$\begin{aligned} \!\sum_{k=1}^{\infty} k\!\cdot\! P\left({\boldsymbol{\Pi}_{n,i} = k}\right) &\!= \sum\limits_{k=1}^{\infty} k\cdot \frac{\lambda_{n,i}^{k}}{k!} e^{-\lambda_{n,i}}\\ &\!= \lambda_{n,i}\!\cdot\! e^{-\lambda_{n,i}} \sum\limits_{k=1}^{\infty} \frac{\lambda_{n,i}^{k-1}}{(k-1)!} \\ &\!= \lambda_{n,i}\!\cdot\! e^{-\lambda_{n,i}} \sum\limits_{j=0}^{\infty} \frac{\lambda_{n,i}^{j}}{j!}\\ &\!= \lambda_{n,i}\!\cdot\! e^{-\lambda_{n,i}} e^{\lambda_{n,i}} = \lambda_{n,i} \end{aligned} $$

(36)

and

$${}\begin{aligned} \sum_{k=1}^{\infty} k \cdot P\left({\boldsymbol{\Pi}_{n,i} = k-1}\right) &= \sum\limits_{k=1}^{\infty} k\cdot \frac{\lambda_{n,i}^{k-1}}{(k-1)!} e^{-\lambda_{n,i}} \\ &= e^{-\lambda_{n,i}} \!\left(\!\!\left(\sum\limits_{k=1}^{\infty} \frac{\lambda_{n,i}^{k-1}}{(k-1)!}\right)\,+\,\sum\limits_{k=1}^{\infty}(k-1)\frac{\lambda_{n,i}^{k-1}}{(k-1)!}\right) \\ &= e^{-\lambda_{n,i}} \!\left(\!\!\left(\sum\limits_{k=1}^{\infty} \frac{\lambda_{n,i}^{k-1}}{(k-1)!}\right)\,+\,\lambda_{n,i} \sum_{k=2}^{\infty}\frac{\lambda_{n,i}^{k-2}}{(k-2)!}\right)\\ &= e^{-\lambda_{n,i}} \!\left(\!\!\left(\sum\limits_{j=0}^{\infty} \frac{\lambda_{n,i}^{j}}{j!}\right) + \lambda_{n,i} \sum_{j=0}^{\infty}\frac{\lambda_{n,i}^{j}}{j!}\right)\\ &= e^{-\lambda_{n,i}} \left(e^{\lambda_{n,i}} + \lambda_{n,i}\cdot e^{\lambda_{n,i}}\right) = 1 + \lambda_{n,i} \end{aligned} $$

(37)

we finally get

$$\begin{array}{*{20}l} &E\left[{\boldsymbol{A}_{n,i}}\right] = \left(1\,-\,\beta_{n,i}\right) \lambda_{n,i}\,+\,\beta_{n,i} \left(\lambda_{n,i}\,+\,1\right). \end{array} $$

(38)

Appendix B: Stationary distribution

The stationary distribution of the presented Markov chain for node v _n is denoted as

$$\begin{array}{*{20}l} c = \left(c_{n,\left\lfloor\tfrac{j}{l_{S}}\right\rfloor,j\%l_{S}}\right)_{j=0,\ldots,(K+1)\cdot l_{S}-1} \end{array} $$

(39)

where all 0≤c_n,q,i≤1 and c·e=1 with

$$\begin{array}{*{20}l} e = \left(\begin{array}{cccc}1 & 1 & \ldots & 1\end{array}\right)^{T}, \end{array} $$

(40)

that is the normalization criterion that the probabilities have to sum up to 1. The stationary distribution is calculated as the solution of cP=c, where P is the transition probability matrix

$$\begin{array}{*{20}l} \begin{aligned} P &= \left [p_{j, k}\right ]_{((K +1) \cdot l_{S}) \times ((K+1) \cdot l_{S}) }\\ p_{j,k} &= P\! \left (\sigma ^ n_{\left\lfloor\tfrac{j}{l_{S}}\right\rfloor,j\% l_{S}} \to \sigma ^ n_{\left\lfloor\tfrac{k}{l_{S}}\right\rfloor,k\% l_{S}}\right). \end{aligned} \end{array} $$

(41)

This can be rewritten as

$$\begin{array}{*{20}l} c(I - P) = 0 \Leftrightarrow (I - P)^{T} c^{T} = 0, \end{array} $$

(42)

with the identity matrix I. This is a homogeneous system of (K+1)·l _S linear equations and the same number of unknowns.

Irreducibility

Consider the example in Fig. 14 with l _S =3, K=1, g_up=0 and $\mu ^{\text {RX}}_{n,0} > 0$. Since nothing is generated and every packet received in slot 0 is immediately sent out again, the state $\sigma ^{n}_{1,2 }$ is never visited when starting with q=0. It is also possible to construct more complex examples where blocks of states exist that are linked together, but are not reachable from any state with q=0. This property makes the Markov chain reducible and thus a unique c is not guaranteed by the following proof. Finding these states corresponds to finding the states that are not in the strongly connected set that contains $\sigma ^{n}_{0,0 }$.

Fig. 14

A schedule with unreachable states $\sigma ^{n}_{1,2 }$

Since it is meaningless for the application to assign any c_n,q,i>0 to those states, they are set to zero and the corresponding columns and rows are removed from P, making the Markov chain and P irreducible, that is there is no permutation of the rows and columns of P resulting in

$$\begin{array}{*{20}l} \left(\begin{array}{cc} A & B\\ 0 & D \end{array}\right), \end{array} $$

(43)

with the square matrices A and D and the matrix 0 with all elements zero.

Rank of I−P

In the following we prove that the matrix Q=I−P has rank n−1 if P is irreducible. The proof goes along the lines of [47]. Since the outgoing transitions of a state have to sum up to one, that is

$$\begin{array}{*{20}l} \forall \sigma ^ n_{q_1, i_1}\in \boldsymbol{\Sigma}_{n} : \sum_{\sigma ^{n}_{q_2,i_2} \in \boldsymbol{\Sigma}_{n}} P \left (\sigma ^ n_{q_1, i_1} \to \sigma ^ n_{q_2, i_2}\right) = 1, \end{array} $$

(44)

it holds that Pe=1; thus (I−P)e=0, so the matrix Q has at least one zero eigenvalue, is therefore singular and its rank is at most n−1.

Assuming a rank of n−2 or less, then there is at least one vector x with Qx=0 that is orthogonal to e, i.e. x^Te=0. Thus, all linear combinations of x and e are also in the null space of Q. In particular, this holds for d=x−m·e where m is the minimum entry of x. Then at least one, but not all, elements of d are 0 and all others are larger than 0. Note that this would not hold for x parallel to e. However, this is not possible since x^Te=0.

Since permutations do not change the rank, we assume without loss of generality the first u>0 elements of d are positive and the remaining n−u elements are zero. It holds

$$\begin{array}{*{20}l} Q d = 0 \Leftrightarrow (I-P) d = Id - Pd = 0\\ \Leftrightarrow P d = I d \Leftrightarrow Pd = d. \end{array} $$

(45)

This can be partitioned as

$$\begin{array}{*{20}l} P d = \left(\begin{array}{cc} A & B\\ C & D \end{array}\right) \left(\begin{array}{c}d_{1}\\\vdots\\d_{u}\\0\\\vdots\\0\end{array}\right) = \left(\begin{array}{c}d_{1}\\\vdots\\d_{u}\\0\\\vdots\\0\end{array}\right), \end{array} $$

(46)

with C of dimension (n−u)×u. Therefore, it holds

$$\begin{array}{*{20}l} C \left(\begin{array}{c}d_{1}\\\vdots\\d_{u}\end{array}\right) = \left(\begin{array}{c}0\\\vdots\\0\end{array}\right). \end{array} $$

(47)

For d _i >0 ∀1≤i≤u this is only possible if all entries of C are zero or there are negative entries in C. The first one contradicts the irreducibility of P and the second one would require the existence of negative probabilities in P. Therefore, I−P has rank n−1. Since the rank is maintained by transposition, the matrix (I−P)^T has rank n−1, too.

Solution of c P=c

Since (I−P)^T has rank n−1, the homogeneous system of linear equations (I−P)^Tc^T=0 has a solution space of dimension one, so since c·e=1, there is a unique stationary distribution. Furthermore, if we find any vector y≠0 with (I−P)^Ty^T=0, we can get c by normalization.

Though most methods for the numerical solution of systems of linear equations are tailored to the handling of regular matrices, many can be adapted for the singular case as presented in [48]. For our application, the implementation of the generalized minimal residual method (GMRES) in PETSc [46] with SOR preconditioning and initial guess 0.5e turned out to work very well. For more background about why and when GMRES is applicable to Markov chains, see Section 4.4.4 in [48].

Nomenclature

• $\mathcal {A}_{n}$ Total expected number of generated or received packets per slotframe

• A_n,i Random variable for the number of arriving, i.e., generated and received, packets in slot i at node n

• β_n,i Packet probability in the Bernoulli part of the traffic model

• ${\mathcal {B}}_{n}$ Relation of slots to blocked channels

• ${\mathcal {C}}$ Channels

• c ] Vector of the c_n,q,i

• ${\mathcal {C}}_{n}$ Relation of slots to channel

• c_n,q,i Probability of being in state $\sigma ^{n}_{q,i}$ in the stationary distribution

• D(v,w) Predicate that describes potential collisions between nodes

• ${{\boldsymbol {\mathcal {D}}}}_{i,l}$ The set of possibly disturbing links during slot i

• D _n Random variable for the queuing delay

• ${\mathfrak {D}}_{\text {up},{n}}$ End-to-end packet delay

• γ _n Number of proper descendants of node n

• g_up Packet generation rate in upstream

• I _up Mean packet generation interval

• ${\mathcal {K}}_{n}$ Relation of slots to counterpart

• L _i The set of active links during slot i

• λ_n,i Packet rate in the Poisson part of the traffic model

• l _S Overall number of slots in the slotframe

• ${\mu ^{\textsc {RX}}_{n,i}}$ Probability of receiving a packet in slot i

• ${\mu ^{\textsc {TX}}_{n,i}}$ Probability of sending a packet in slot i

• N _n Neighbors of node n

• P_accept,n Packet accepting probability

• PER_b,l Packet error rate for a transmission of b bytes

• p _n Parent of node n

• Π_n,i Random variable for the number of arriving packets according to the Poisson part of the traffic model

• Q _n Random variable for the number packets inserted into the queue

• ${\mathcal {R}_{n}}$ Reception slots

• ${\mathfrak {R}}_{\text {up},n}$ Reliability that a packet sent by n arrives at the sink

• r_n,i Reception slot.

• ${\left \vert \mathcal {T}_{n}\right \vert }$ Number of transmission slots in the schedule for node n

• Σ _n The set of states for the queuing model

• $\sigma ^{n}_{q,i}$ State in the queuing model

• ${\mathcal {T}_{n}}$ Transmission slots

• T _s Duration of a slot

• t_n,i Transmission slot

• v₀ Root node

• v _n Node with index n