# Handoff management in communication-based train control networks using stream control transmission protocol and IEEE 802.11p WLANs

- Li Zhu
^{1}Email author, - Fei Richard Yu
^{2}, - Bin Ning
^{1}and - Tao Tang
^{1}

**2012**:211

https://doi.org/10.1186/1687-1499-2012-211

© Zhu et al.; licensee Springer. 2012

**Received: **16 November 2011

**Accepted: **25 May 2012

**Published: **9 July 2012

## Abstract

Communication-based train control (CBTC) network is an automated control network for railways that ensures the safe operation of rail vehicles using data communications. CBTC networks have stringent requirements for communication availability and latency. Wireless local area network (WLAN) is a popular choice in CBTC networks due to the available commercial-off-the-shelf WLAN equipments. However, handoffs in WLANs may result in communication interruption and long latency in WLANs-based CBTC networks. In this article, we propose a handoff management scheme for CBTC networks using stream control transmission protocol (SCTP) and IEEE 802.11p WLANs to provide high communication availability and low latency in CBTC networks. We formulate the handoff decision problem as a stochastic Semi-Markov Decision Process (SMDP) with the objectives of minimizing the handoff latency and maximizing the SCTP throughput. Simulation results are presented to show that the proposed scheme can significantly improve the handoff performance in CBTC networks.

### Keywords

Communication-based train control (CBTC) Handoff Stream control transmission protocol (SCTP) IEEE 802.11p wireless local area networks (WLANs)## Introduction

Communication-based train control (CBTC) network is an automated control network for railways that ensures the safe operation of rail vehicles using data communications[1]. CBTC is based on two important technologies that marked profoundly the development of our society in the last century: railways and communication technologies. It is a modern successor of traditional railway signaling systems that provide a limited control using track circuits, interlockings and signals. In most CBTC networks, data between trains and trackside equipments are transferred bidirectionally by wireless communication networks, such as global system for mobile communications-railway (GSM-R) and wireless local area network (WLAN). For urban mass transit systems, WLAN is a better choice due to the available commercial-off-the-shelf equipments[2]. WLAN-based CBTC has been deployed in many real systems, such as New York City Canarsie Line, Beijing Metro Line 10 from Siemens[3], and Las Vegas Monorail from Alcatel[4]. We will focus on WLAN-based CBTC networks in this article.

Communication-based train control networks have stringent requirements for wireless communication availability and latency[5]. Whereas in commercial wireless networks, less service availability and long latency mean less revenues or/and poor quality of services (QoSs)[6]; in CBTC networks, less service availability could cause train derailment, collision or even catastrophic loss of life or assets. Therefore, it is important to ensure the wireless communications are available when they are needed, and the latency is minimized in CBTC networks. Furthermore, in recent years, there have been significant developments of high speed train systems around the world (e.g., China railway high-speed (CRH) systems with the maximum speed of 352 km/h[7]), which introduce new non-trivial challenges to the CBTC designs in the high speed environment.

Most existing WLAN-based CBTC networks are using traditional IEEE 802.11 technologies[8], such as 802.11a/b/g. However, IEEE 802.11a/b/g WLANs were not originally designed for high speed environments. Particularly, when a train moves away from the coverage of a WLAN access point (AP) and enters the coverage of another AP along the railway, a handoff procedure occurs, and this handoff process may result in communication interruption and long latency. The handoff procedure can be divided into four steps, namely probing (also referred to as scanning), channel switching, authentication and association. This whole procedure may take up to several hundreds milliseconds[9].

There are several schemes proposed in the literature to decrease WLAN handoff latency. Fitzmaurice[9] and Mishra et al.[10] have shown that over 90% of the time in the handoff process is spent in the scanning stage. Therefore, most of previous works in optimizing WLAN handoffs focus on making the scanning process more efficient. A SyncScan technique is proposed in[11], in which appropriate time synchronization is required between APs and clients. A topology inferencing technique in both clients and APs is proposed in[12] to improve the scanning process. A cooperative handoff framework is proposed in[13] to utilize mechanism for information sharing to reduce the delays during the scanning/probe phases. In[14], a fast handoff scheme that skips all mentioned stages is proposed, where handoff is controlled and prepared by the access network and is triggered by sending a hop request message to the mobile station (STA). There are some schemes using multi-radio in mobile clients trying to reduce the WLAN handoff latency. Adya et al.[15] proposed a protocol to allow multi-radio mobile nodes in a mesh network to potentially establish two separate wireless links between a pair of nodes. This work primarily focuses on improving efficiency of wireless mesh networks, which is different from the CBTC networks considered in this article.

It is necessary to look at the handoff management at multiple layers of the protocol stack, not just at the data link layer as considered in the past[16]. Indeed, the handoff management problem can be solved at transport layer[17–20]. For example, stream control transmission protocol (SCTP)[21, 22], a new IETF-standardized transport layer protocol in addition to transmission control protocol (TCP) and User Datagram Protocol (UDP), can be used to solve the handoff management problem. The multi-homing, multi-stream and partial reliable[23] data transmission features of SCTP are especially attractive for applications that have stringent performance and high reliability requirements. Compared to other handoff management approaches, transport layer schemes have many advantages, such as improved throughput and latency performance. Moreover, no third party other than the endpoints participates in handoff process, and no modification or addition of network components is required, which makes transport layer approaches attractive in WLAN-based CBTC networks, where commercial-off-the-shelf equipments are widely used.

Although some works have been done for the handoff management in CBTC networks, most of them are focused on handoff protocols and network architectures, and handoff decision policy issues (i.e., when to execute handoff) are largely ignored in CBTC networks. However, due to the high mobility environment, as well as the high availability and low handoff latency requirements, handoff decision policy issues are very important in designing CBTC networks, which will significantly affect the overall system performance.

- (1)
We propose a handoff management scheme based on SCTP and IEEE 802.11p WLANs to provide high communication availability and low latency in CBTC networks.

- (2)
We formulate the handoff decision problem as a stochastic Semi-Markov decision process (SMDP)[25], which has been successfully used to solve finance[26] and admission control[27] problems, among others. This article focuses on the application of SMDP to the handoff decision problem in CBTC networks.

- (3)
Minimizing the handoff latency is one of the objectives in the proposed scheme. In addition, since multimedia information, such as train schedule, weather forecast, live news, sports and finance, is more and more popular in railway communication networks[28], we also consider maximizing the SCTP throughput in our scheme.

- (4)
Extensive simulation results are presented. It is illustrated that the proposed scheme can significantly decrease the handoff latency and improve SCTP throughput in CBTC networks.

The rest of this article is organized as follows. The 802.11p and SCTP based CBTC network with the corresponding handoff procedure is presented in Section “The proposed CBTC network based on SCTP and IEEE 802.11p”. The SMDP based handoff decision model, optimality equation, and value iteration algorithm are described in Section “Optimal Handoff the CBTC network using SCTP and IEEE 802.11p WLANs”. Some implementation issues are given in Section “Implementation issues”. Simulation results are presented and discussed in Section “Simulation results and discussions”. Finally, we conclude our study in Section “Conclusions and future work”.

## The proposed CBTC network based on SCTP and IEEE 802.11p

In this section, we first present an overview of CBTC. Then, we introduce IEEE 802.11p WLANs and SCTP. The proposed CBTC network based on SCTP and IEEE 802.11p WLANs is also presented in this section.

### Overview of CBTC

When a train moves away from the coverage of an AP and enters the coverage of another AP along the railway, the handoff procedure may result in communication interruption and long latency. In CBTC networks, it is important to maintain communication link availability in order to guarantee train operation safety and efficiency. To this end, we present a handoff management scheme based on SCTP at transport layer and IEEE 802.11p at data link and physical layers to provide high link availability in CBTC networks. Brief introductions of IEEE 802.11p and SCTP are given in the next two subsections.

### IEEE 802.11p

IEEE Std 802.11p-2010, also known as Wireless access in vehicular environment (WAVE), is an amendment to the IEEE Std 802.11-2007 standard that adds applications to fast changing vehicular networks[24].

It deals essentially with the data link and physical layers of the OSI model. The medium access control (MAC) protocol in IEEE 802.11p uses the enhanced distributed channel access (EDCA) mechanism originally provided by IEEE Std 802.11e-2005[29], while the physical layer is a variation of the OFDM based IEEE Std 802.11a-1999 standard, with a 10 MHz wide channel instead of the 20 MHz one usually used by IEEE 802.11a devices.

The focus of IEEE 802.11p lies on fast adaptation to rapid changes occurring in a highly mobile vehicular network, sacrificing identification and authentication procedures that are usually part of the IEEE 802.11 WLAN standards. For more efficient data exchange between high speed vehicles or between a vehicle and a Road Side Unit (RSU), IEEE 802.11p specifies a minimized set of parameters for the execution phase of the handoff process[30].

### Stream control transmission protocol

Any of the IP addresses at either host can be used as the corresponding source or destination address in an IP packet sent by one host to the other. Before data can be exchanged, the two SCTP hosts must exchange the set of available IP addresses in the association establishment stage. The mobile extension of SCTP (mSCTP) enables the endpoints to dynamically add, delete, or change the IP addresses during an active SCTP association[17]. The multi-homing mechanism was originally designed for fault-resilient communications between two SCTP endpoints over wired networks. This powerful feature can be used to design a handoff management scheme in CBTC networks.

### The proposed CBTC network based on SCTP and IEEE 802.11p WLANs

A critical issue in the above network is the handoff decision policy, i.e., when to perform handoff. In high speed environments, wireless channels are changing dynamically in CBTC networks. The communication QoS is not simply determined by the geo-location of the train and AP. If the handoff decision policy is not designed carefully, communication interruption, long latency and low throughput may occur, which will significantly affect the performance of a CBTC network. Therefore, an efficient handoff decision policy is needed to decide at what time the second path should be established and when to cut off the primary path, which will be studied in the following sections.

## Optimal Handoff decision policy in the CBTC network using SCTP and IEEE 802.11p WLANs

In this section, we first present an overview of SMDP modeling. Then, the states, actions, reward functions, state transition probability, constraints, optimality equations, and value iteration algorithm in the CBTC system are presented.

### SMDP modeling

In this article, the handoff decision problem in the CBTC network presented above is formulated as an SMDP[25]. Markov decision process (MDP) provides a mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker. MDP has been successfully used in heterogeneous wireless networks[31]. Besides the basic features, an SMDP generalizes a MDP by allowing decision maker to choose actions whenever the system state changes and allowing the time spent in a particular state to follow an arbitrary probability distribution. In this article, we use SMDP to solve the handoff decision problem in CBTC networks using SCTP and IEEE 802.11p. The optimal handoff decision policy can be obtained from the value iteration algorithm in this formulation. In our proposed CBTC network, the SA on the train makes handoff decisions at specific time instances according to the current state *s*(*t*), and the system moves into a new state based on the current state *s*(*t*) as well as the chosen decision *a*(*t*). Given *s*(*t*) and *a*(*t*), the next state is conditionally independent of all previous states and actions. This Markov property of state transition process makes it possible to model the handoff problem as an SMDP.

An SMDP model consists of the following five elements: (1) decision epochs, (2) states, (3) actions, (4) rewards, and (5) transition probabilities, which will be described in the following.

The SA on a train has to make a decision whenever a certain time period has elapsed. The instant times are called *decision epochs*.

Let *S* and *A* be the state space and action space, respectively. Given the current state *s*(*t*)∈*S* and the chosen action *a*(*t*)∈*A*, the state transition probability function for the next state *s*(*t* + 1) is denoted as *P*[*s*(*t* + 1)|*s*(*t*),*a*(*t*)]. This function is Markovian because the state transition probability depends on the current state and action but not on the previous states.

A decision rule prescribes a procedure for action selection in each state at a specified decision epoch. Markov decision rules are functions *δ*(*t*) : *S* → *A*, which specify the action choice *a*(*t*) when the system occupies state *s*(*t*) at decision epoch *t*. A policy *Π* = (*δ*(1),*δ*(2),…,*δ*(*t*)) is a sequence of decision rules to be used at all decision epochs.

*υ*

^{ Π }(

*s*(0)) denote the expected total reward from the first decision epoch until the handoff decision period elapses, given that the policy

*Π*is used with an initial state

*s*(

*0*). We have

where *r*(*s*(*t*),*a*(*t*)) is the reward function,${E}_{s\left(0\right)}^{\Pi}$ denotes the expectation with respect to policy *Π* and initial state *s*(*0*), and *E*_{
N
}denotes the expectation with respect to random number *N*. The sequence *T* = {1,2,…,*N*} represents the times of successive decision epochs. Let the time between two successive epochs be *τ*. The product of random variables *N* and *τ* denotes the time that the train stays between two successive APs. We refer that time as the handoff decision period. The random variable *N*, which depends on the AP deployment space, train speed and the time between successive decision epochs, is assumed to be geometrically distributed with mean 1/(1 − *λ*).

*N*geometrically distributed with mean 1/(1 −

*λ*), according to[25], (1) can be rewritten as

where *λ* can also be interpreted as the discount factor of the model, and 0 ≤ *λ* ≤ 1,${E}_{s\left(0\right)}^{\Pi}$ denotes the expectation with respect to policy *Π* and initial state *s*(*0*).

Our optimization problem is to maximize the expected total discounted reward. We define a policy *Π*^{∗} to be optimal if${\upsilon}^{{\Pi}^{\ast}}\ge {\upsilon}^{\Pi}$. A stationary policy has the form$\Pi =(\delta ,\delta ,\dots \phantom{\rule{0.3em}{0ex}})$. For convenience, we simply denote *Π* by *δ*. Our objective is to determine an optimal stationary policy *δ*^{∗}, which maximizes the expected total discounted reward given by (2).

In order to obtain the optimal solution, it is necessary to identify the states, actions, reward functions, state transition probability and constrains in our SMDP model.

### Action and state

*M*be the AP that covers the SA, then the other one is

*M*+ 1. According to our handoff scheme, we define the action space as

*A*= {

*M*,

*M*+ 1,

*M*∗(

*M*+ 1)}. An action

*a*(

*t*) ∈

*A*is defined as follows.

- (1)
If

*a*(*t*) =*M*, the SA communicates with AP*M*; - (2)
If

*a*(*t*) =*M*+ 1, the SA communicates with AP*M*+ 1; - (3)
If

*a*(*t*) =*M*∗ (*M*+ 1), the SA communicates with both AP*M*and AP*M*+ 1.

*s*(

*t*) = [

*γ*

_{1},

*γ*

_{2},

*η*,

*ξ*] ∈

*S*has the following information:

- (1)
The measured signal-to-noise ratio (SNR) from two APs,

*γ*_{1}and*γ*_{2}; - (2)
The current SCTP congestion window,

*η*; - (3)
The path(s) currently used by the SA,

*ξ*;

### Reward function

where *f*(*s*(*t*)) reflects the QoS provided by the chosen path(s) at epoch *t*, and *g*(*s*(*t*),*a*(*t*)) captures the cost under state *s*(*t*) and action *a*(*t*).

*s*(

*t*) and the chosen action

*a*(

*t*),

*f*(

*s*(

*t*)) is defined as

where *f*_{
b
}(*s*(*t*)) is the SCTP throughput, *f*_{
d
}(*s*(*t*)) is the SCTP packet delay, *α* and *β* are two independent dimension weight factors to make the SCTP throughput and packet delay comparable, *ϕ* and (1 − *ϕ*) are importance weight factors to indicate the importance of SCTP throughput and packet delay. In (4), we combine SCTP throughput and delay into a single function. This is a common approach used in the optimization literature, which is called aggregate objective function (AOF), to solve an optimization problem with multiple objectives[32, 33]. In reality, different CBTC networks have different throughput and packet delay requirements. By adjusting the parameters in (4), the proposed scheme is generic enough to accommodate different requirements in real CBTC networks.

Stream control transmission protocol throughput and packet delay will be derived later in Section “SCTP throughput and packet delay”

*g*(

*s*(

*t*),

*a*(

*t*)) under the current state

*s*(

*t*) and the chosen action

*a*(

*t*) is defined as

where *ξ* is the currently used path, *K* is the multi-path penalty when the SA is working in the multi-path mode, which is mostly caused by the interference of two wireless links when they are working simultaneously.

When the action is a single path action and no handoff happens, the penalty is 0. The penalty will change to be *K* when the current action is a multi-path policy. As shown in Figure3, there are information exchanges between the SA and AP when handoff is triggered and finished. We assume the reward for these actions to be zero since most of the bandwidth is occupied by the communication overhead. The cost for these actions is *f*(*s*(*t*)).

#### SCTP throughput and packet delay

In this section, we derive SCTP throughput and packet delay, which are used in (4).

where *C* is the achievable data rate, *W* is the OFDM sub-channel bandwidth in 802.11p, BER is the bit error rate.

We choose this capacity formula in our formulation because it has been commonly used in the literature, and has reasonable accuracy[35, 36]. Nevertheless, our proposed SMDP model is not dependent on a particular capacity formula. If there is a new physical layer technology available for WLANs in the future, a different capacity formula for this new physical layer technology can be used in our formulation as well.

where *L*_{fr} is the frame length in bits which can be considered approximately equal to the SCTP chunk size.

As we introduced in section “The proposed CBTC network based on SCTP and IEEE 802.11p”, the MAC protocol in IEEE 802.11p uses the EDCA mechanism originally developed in IEEE 802.11e. In EDCA, a window based backoff mechanism is used such that a node willing to transmit will sense the medium first, and if the medium is not free it will choose a backoff time uniformly at random from the interval [0,*CW* + 1], where *CW* is Contention Window and the initial value equals to$C{W}_{min}$. The *CW* will be doubled if the subsequent transmission attempt fails until it reaches$C{W}_{max}$.

*n*times at MAC layer, the corresponding packet delay

*T*

_{delay11p}(

*n*) can be ****calculated as follows[37].

where *T*_{aifs} is the Arbitration Inter Frame Space (AIFS), *T*_{data} is the time needed to transmit a data frame, *T*_{sifs} is the Short Inter Frame Space (SIFS), *T*_{ack} is the time needed to transmit the ACK frame, *T*_{backoff(i)} is the backoff time of the retransmission at *i* times, and *T*_{transfer} is the propagation time of the data. We need to point out that, in (9), an acknowledgment is actually sent only during the last *T*_{ack}, while no acknowledgement is sent in the previous *T*_{ack}. The client will wait for *T*_{ack} even there is no acknowledgment actually sent when a transmission fails.

*T*

_{11p}with maximum retransmission time

*R*is given by

*T*

_{wired}in the wired network, finally we get the SCTP packet delay

The wireless delay *T*_{11p}, which is described in (10), is determined by the SNR in the state parameter *s*(*t*), as well as the current action *a*(*t*).

where *η* is the current congestion window in state parameter in *s*(*t*), and RTT is the round-trip time which is approximately 2∗*f*_{
d
}(*s*(*t*)).

### State transition probability

*s*(

*t*) = [

*γ*

_{1},

*γ*

_{2},

*η*,

*ξ*] and the chosen action

*a*(

*t*), the probability function of the next state$s(t+1)=[{\gamma}_{1}^{\prime},{\gamma}_{2}^{\prime},{\eta}^{\prime},{\xi}^{\prime}]$ is given by

*γ*

_{1},

*γ*

_{2}, SCTP congestion window

*η*and currently used path

*ξ*are independent. This assumption is reasonable in practice, because the two wireless channels from two APs are independent, and the currently used path is solely determined by the last action. Moreover, the channels change much faster than the SCTP congestion window size, which makes it reasonable to assume that the SCTP congestion window size and the channels are independent. Then, we have

where$P\left[{\gamma}_{1}^{\prime}\right|{\gamma}_{1}]$ and$P\left[{\gamma}_{2}^{\prime}\right|{\gamma}_{2}]$ are the channel state transition probabilities for the two wireless links, respectively, *P*[*η*^{
′
} | *η*] is the SCTP congestion window state transition probability, and *P*[*ξ*^{
′
} | *ξ*,*a*(*t*)] is the currently used path transition probability. These state transition probabilities will be derived in the following.

#### Channel state transition probability

*P*

_{loss}is the path loss,

*h*

_{ t }and

*h*

_{ r }are the heights of the transmitting and receiving antennas, respectively, and

*d*is distance between the train SA and the AP. Combined with large-scale path loss and small-scale fading (Rayleigh distribution is used to describe the fading envelope), we get the received SNR

*γ*as

where *P*_{
t
} is the transmitted power, *ϑ* is a Gaussian random variable with a variance of *ς* and a mean of 0, *χ* is a Rayleigh random variable with a mean of 1, *G*_{
t
} and *G*_{
r
}are the antenna gains for the transmitter and receiver, respectively, and *P*_{noise} is the noise power.

In this article, we use finite-state Markov channel (FSMC) models in CBTC networks. FSMC models have been widely accepted in the literature as an effective approach to characterize the correlation structure of the fading process, including satellite, indoor, Rayleigh fading, Ricean fading, and Nakagami fading channels[39–45]. Considering FSMC models may enable substantial performance improvement over the schemes with memoryless channel models[46].

In FSMC models, the range of the received SNR can be partitioned into discrete levels. Each level corresponds to a state in the Markov chain.

Assume there are *L* states in the model. Let *i* and *κ* denote the instantaneous channel state and SNR, respectively. When the channel is in state *i*, the corresponding SNR is *κ*_{
i
}. Then we have *κ*_{
i
} < *κ* < *κ*_{i + 1}, 0 ≤ *i* ≤ *L* − 1. The probability of transition from state *i* to state *j* in the Markov model is channel transition probability.

In real networks, the values of the above transition probability can be obtained from the history observation of the CBTC network.

#### SCTP congestion window transition probability

In order to derive the congestion window transition probability, we refer to the SCTP behavior model in[18]. The SCTP behavior is modeled in terms of “rounds”, where a round starts when the sender begins the transmission of a window of chunks and ends when the sender receives the last acknowledgment for chunks in this window. SCTP doubles its congestion window size in the slow-start stage when the current congestion window, denoted by *η*, is less than a threshold, denoted by _{η th}. SCTP increases the congestion window linearly in the congestion avoidance stage if none of the chunks in the previous window is lost during the previous RTT. If one or more chunks in the previous window are lost, the congestion window is set to half of the current window. When the maximum congestion window, denoted by${\eta}_{max}$, is reached, the congestion window will not increase if no chunks get lost in the last round. The congestion window will not change until all the chunks in the window are sent out.

*η*, when SCTP is in slow-start, the congestion window in the next epoch can be

*η*,

*η*/2, or 2

*η*. For the congestion avoidance stage, the congestion window can be changed to

*η*,

*η*/2, or

*η*+ 1, which depends on the decision epoch and packet losses. We derive the congestion window state transition probability as

*τ*is the time between two successive decision epochs,

*q*is the probability that a chunk is successfully received, which can be calculated as,

*q*= 1 − FER

^{ n }, where FER is the FER obtained in (8), and

*n*is the number of WLAN retransmissions. Particularly, when the current congestion window size is one, it will not change even if the packet is lost. We then derive the probability that the congestion window size transitioning from one to one as follows.

#### Currently used path transition probability

### Constraint

*ξ*, when

*ξ*is in a single path, the chosen action should not be the other single path action.

### Optimality equations and value iteration algorithm

*υ*(

*s*) denotes the maximum expected total reward, given the initial state

*s*, and

*s*

^{ ′ }represents next state. That is

The solutions of the optimality equation correspond to the maximum expected total reward *υ*(*s*) and the SMDP optimal policy *δ*^{∗}. Note that the SMDP optimal policy *δ*^{∗} indicates the decision as to which action to choose.

- (1)
Set

*?*^{0}(*s*)?=?0 for each state*s*. Specify*e*?>?0, and set*k*?=?0; - (2)For each state
*s*, compute*?*^{k + 1}(*s*) by$?\left(s\right)=\underset{a?A}{\text{max}}\left\{r(s,a)+\underset{{s}^{\text{'}}?S}{?}\mathrm{?P}[{s}^{\text{'}}|s,a]?\left({s}^{\text{'}}\right)\right\}.$(23) - (3)
If ||

*?*^{k + 1}(*s*)?-?*?*^{ k }(*s*)||?<?*e*(1?-?*?*)/2*?*, go to step 4). Otherwise, increase*k*by 1 and return to step 2). - (4)For each
*s*???*S*, compute the stationary optimal policy$d\left(s\right)=\text{arg}\underset{\mathrm{aeA}}{\text{max}}\left\{r(s,a)+\underset{{s}^{\text{'}}\mathrm{eS}}{?}\mathrm{?P}\left[{s}^{\text{'}}\right|s,a]?\left({s}^{\text{'}}\right)\right\}.$(24)

The value iteration algorithm is proved to be an efficient and stable iteration algorithm[25]. The algorithm operates by calculating successive approximation to the value function *υ*(*s*). The computation complexity of the algorithm is *O*(|*A*||*S*|^{2}), where *A* is the action set, and *S* is the state space[25].

## Implementation issues

In this section, we briefly explain how to implement our proposed handoff decision algorithm.

In order to determine the optimal handoff decision policy *δ*^{∗}, we need to measure and estimate the parameters in the SMDP model. In constructing these parameters, we assume that most properties in the network can be made known, which should be realistic particularly for CBTC networks where initial planning and network management is a crucial priori requirement. The multi-path penalty *K* in (5) can be determined by the wireless interface performance. We can measure this parameter by comparing the multi-radio performance with the single radio performance. The weight factor *ϕ* in (4) can be set according to the CBTC networks’ requirements. The handoff decision time space (i.e., discount factor *λ* in (2)) can be estimated based on the AP locations and train moving speeds. The channel state transition probability can be estimated by field measurements. Given the values of all the parameters, the value iteration algorithm described above can be used to derive the optimal handoff decision policy*δ*^{∗}. The calculation of the optimal policy is performed offline and should be updated whenever the system parameters are changed.

Once the optimal policy is obtained, it can be stored in a table format. Each entry of the table specifies the optimal action (handoff decision), given the current state (i.e., channel state, currently used path and SCTP window size). For the on-line process, at each decision epoch, each SA on the train lookups the table to find out the optimal action corresponding to its current state, and then executes the handoff decision. On-line looking up tables can be designed with little computational complexity in practice.

## Simulation results and discussions

*T*

_{aifs}= 9

*us*,$C{W}_{min}=15$ and$C{W}_{max}=1023$. We assume the IEEE 802.11p operates in service channel 174 and 175, and the combined channel provides a date rate of 18 Mbps. The parameters used in the simulations are shown in Table1.

Simulation parameters

Notation | Definition | Value |
---|---|---|

| Time between successive epochs | 50 ms |

| Channel data rate | 18 Mbits/s |

| Arbitration inter frame space | 9 |

| Short inter frame space | 32 |

| Time required to send an ACK | 20 |

$C{W}_{min}$ | Minimum contention window | 15 |

$C{W}_{max}$ | Maximum contention window | 1023 |

| Wired transmission delay | 100 ms |

| Average AP space | 600 m |

| Train speed | 80 km/h |

| Shadowing fading standard deviation | 8 |

| Noise power | −100 dbm |

| SCTP chunk size | 400 Bytes |

We need to point out that our handoff scheme is not limited to train speed. The optimal handoff policy can be calculated only if we can measure the channel transition probability under specific train speed from field tests.

We compare the performance of the proposed scheme with that of the existing handoff scheme based on UDP and traditional IEEE 802.11a and two other heuristic handoff decision policies. The results show that the proposed scheme can significantly decrease the handoff latency, as well as improve SCTP throughput and the expected total reward in CBTC networks.

respectively.

### Handoff delay improvement

By contrast, we can observe from Figure5 that the proposed handoff scheme supports handoff between adjacent APs. During the handoff process, the SA associates to the new AP with the other radio, obtains a new IP address for the new path, communicates with the new address before the old path is terminated. No packet is lost during this handoff procedure. The small delay in the handoff process is caused by the information exchange before the establishment of the new path.

In addition, we illustrate the behaviour of normal SCTP in Figure5. Instead of multicasting data over two paths during the handoff process, the lost packet is retransmitted on the second path for the normal SCTP behaviour. As shown in Figure5, compared to our proposed handoff scheme, extra transmission latency occurs in the normal SCTP behaviour during the handoff process. This is because it takes time for the SCTP sender to get to know the packet is lost, and then to retransmit it.

### Expected total reward and SCTP throughput improvement

In this section, we compare the performance of our proposed handoff decision policy with two other heuristic handoff decision policies. For the first heuristic policy, the path to be selected in each decision epoch is the one that always has the better SNR. For the second heuristic policy, we also model the handoff decision as an SMDP, but SCTP congestion window variation is not considered in this model.

*K*defined in (5). In this figure, the

*Y*axis represents the current path:

*Y*= 1 means that the SA is communicating with AP

*M*;

*Y*= 2 means that the SA is communicating with AP

*M*+ 1;

*Y*= 3 means that the SA is communicating with both AP

*M*and AP

*M*+ 1. With the increase of the multi-path penalty, the proposed SMDP policy considering SCTP congestion window changes. When the multi-path penalty is very small, the SA tends to stay in the multi-path state to get the best expected total reward. The multi-path state duration decreases rapidly with the increase of the multi-path penalty. When the multi-path penalty

*K*= 0.05, the multi-path state duration is very short during the handoff process. For the SMDP policy without considering SCTP congestion window, the policy is not so sensitive to the multi-path penalty, because it does not have enough information to make an appropriate decision to get the best expected total reward. For the policy without using SMDP, as it does not consider the long term total expected reward, the ping-pong handoff shows up in the handoff procedure, and the policy does not change when the handoff parameters change.

*ϕ*, defined in (4), for the three policies. The SMDP policy considering SCTP congestion window gives the highest expected total reward for all different values of

*ϕ*. As we can see, the reward improvement is more obvious with the increase of weight factor. This is because when

*ϕ*increases, the SCTP throughput reward in (12) becomes more important than the SCTP packet delay reward in (11), the two heuristic handoff decision policies do not care about the SCTP congestion window, which is a very important factor for SCTP throughput.

### Structure of the optimal policy

*δ*

^{∗}is numerically computed by implementing the value iteration algorithm. In our scheme, the state space

*S*has 4 dimensions. To present the optimal policy clearly, one dimension needs to be fixed to a specific value. Therefore, in the following example, we fix the currently used path to be

*M*, which means the SA is currently communicating with AP

*M*. The structure of the optimal policy is shown in Figure10. The cubes represent the handoff policy: When the cube’s height is 1, the SA does not trigger a handoff; When the cube’s height is 3, the SA makes a handoff decision and begins to work in the multi-path mode.

We can observe from Figure10 that when the congestion window changes from 1 to 32, the SA tends not to execute a handoff. This is because when the congestion window increases, even if the SNR is low, the throughput is acceptable. Compared to the multi-path penalty caused by a handoff, making a handoff decision would not improve the expected total reward.

## Conclusions and future work

Communication-based train control networks using WLANs have stringent requirements for wireless communication availability and latency. In this article, we studied the handoff management issues in CBTC networks. We presented a CBTC network based on SCTP and IEEE 802.11p WLANs to provide high communication availability and low latency in CBTC networks. The handoff decision problem was modeled as a SMDP with the objectives of minimizing the handoff latency and maximizing the SCTP throughput. In simulation results, we showed that the handoff delay is very close to zero in our proposed handoff management scheme, and the proposed SMDP based handoff decision algorithm can significantly improve SCTP throughput. We also observed that both SCTP congestion window and the measured SNR from APs are important in making the handoff decisions.

We are currently implementing the proposed scheme in a real testbed to further evaluate the performance. We have a project “Channel modeling in metro tunnels and its applications” supported by the National Science Foundation of China (Project NO. 61132003). The testbed is in Beijing YiZhuang Line, which is a part of Beijing Urban Rail system. Moreover, we plan to extend the proposed model to multiple-input multiple-output (MIMO) systems, where the tradeoff between multiplex and diversity will be considered in our model. Moreover, in our SMDP model, only transport layer, data link layer and physical layer were considered as cross-layer information. Extending our model to application layer is also our future work.

## Declarations

### Acknowledgements

We thank the reviewers for their detailed reviews and constructive comments, which have helped to improve the quality of this article.

This work was in part supported by Beijing Jiaotong University Technology Funding Project (NO.KWJB11082536, and NO.2011JBZ014), National Science Foundation of China (NO.61132003), Key Projects in State Key Lab. of Rail Traffic Control and Safety (No. RCS2011ZT007, NO. RCS2010ZT007, NO. RCS2010ZZ003), State Key Laboratory of Rail Traffic Control and Safety of Beijing Jiaotong University within the frame of the project (No. I11K00060), Beijing Science and Technology Star Project (2010B015), and by the Natural Sciences and Engineering Research Council (NSERC) of Canada and industrial and government partners, through NSERC-DIVA Strategic Research Network.

## Authors’ Affiliations

## References

- Pascoe RD, Eichorn TN: What is communication-based train control.
*IEEE Veh. Tech. Mag*2009, 4(4):16-21.View ArticleGoogle Scholar - Aquado M, Jacob E, Saiz P, Unzilla JJ, Hiquero MV, Matias J: Railway signaling systems and new trends in wireless data communication.
*Proc. IEEE VTC’2005-Fall,*(Dallas, TX, Sept 2005), pp. 1333–1336Google Scholar - Lardennois R: Wireless communication for signaling in mass transit.
*Siemens Transportation Systems*2003.Google Scholar - Kuun E: Open standards for CBTC and CCTV radio based communication.
*Technical Forums of Alcatel*2004.Google Scholar - Whitwam F: Integration of wireless network technology with signaling in the rail transit industry.
*Alcatel Telecommun. Rev*2003, 1(1):43-48.Google Scholar - Wei W-F, Sheu T-L: A channel preemption model for vertical handoff in a WLAN-embedded cellular network.
*Wirel. Netw*2010, 16(4):929-941. 10.1007/s11276-009-0178-xView ArticleGoogle Scholar - Dong H, Ning B, Cai B, Hou Z: Automatic train control system development and simulation for high-speed railways.
*IEEE Circ. Syst. Mag*2010, 10(2):6-18.View ArticleGoogle Scholar - Fitzmaurice M: Use of wireless local area networks in rail and urban transit environments.
*Transport. Res. Record J. Transport. Res. Board*2005, 1916(1):42-46. 10.3141/1916-07MathSciNetView ArticleGoogle Scholar - Mishra A, Shin M, Arbaugh W: An empirical analysis of the IEEE 802.11 MAC layer handoff process.
*SIGCOMM Comput. Commun. Rev*2003, 33: 93-102. 10.1145/956981.956990View ArticleGoogle Scholar - Shin S, Forte AG, Rawat AS, Schulzrinne H: Reducing MAC layer handoff latency in IEEE 802.11 wireless LANs.
*Proceedings of the, Second International Workshop on Mobility Management and Wireless Access Protocols*(Philadelphia, USA, 2004), pp. 19–26View ArticleGoogle Scholar - Ramani I, Ramani I: Syncscan: Practical fast handoff for 802.11 infrastructure networks.
*Proc. IEEE Infocom’05,*(Seoul, Korea, Mar 2005), pp. 675–684Google Scholar - Shin M, Mishra A, Arbaugh WA: Improving the latency of 802.11 hand-offs using neighbor graphs.
*Proc.MobiSys’04,*(Boston, USA, June 2004), pp. 70–83Google Scholar - Athanasiou G, Korakis T, Tassiulas L: An 802.11k compliant framework for cooperative handoff in wireless networks. EURASIP J. Wirel. Commun. Netw 2009: 14. (Article ID 350643) (2009) 10.1155/2009/350643Google Scholar
- Jooris B, Schoutteet A, Vermeulen F, Moerman I: Access network controlled fast handoff for streaming multimedia in WLAN.
*Proc. 16th IST Mobile and Wireless Communications Summit,*(Budapest,Hungary, July 2007), pp.1–5Google Scholar - Adya A, Bahl P, Padhye J, Wolman A, Zhou L: A multi-radio unification protocol for IEEE 802.11 wireless networks.
*Proc. BROADNETS’04,*(San Jose, USA, Oct 2004), pp. 344–354Google Scholar - Yu FR, Wong VWS, Song J-H, Leung VCM, Chan HCB: Next generation mobility management: an introduction.
*Wiley Wirel. Commun. Mobile Comput*2011, 11(4):446-458. 10.1002/wcm.904View ArticleGoogle Scholar - Ma L, Yu F, Leung V, Randhawa T: A new method to support UMTS/WLAN vertical handover using SCTP.
*IEEE Wirel. Commun*2004, 11(4):44-51. 10.1109/MWC.2004.1325890View ArticleGoogle Scholar - Ma L, Yu FR, Leung VCM: Performance improvements of mobile SCTP in integrated heterogeneous wireless networks.
*IEEE Trans. Wirel. Commun*2007, 6(10):3567-3578.View ArticleGoogle Scholar - Pareit D, Gheysens N, Van Leeuwen T, Moerman I, Van Brussel W, Torfs W, De Cleyn P, Blondia C: QoS-enabled Internet-on-train network architecture: inter-working by MMP-SCTP versus MIP.
*Proc. ITST’07,*Jun 2007.Google Scholar - Pareit D, Moerman I, Demeester P, Torfs W, De Cleyn P, Blondia C: SCTP as mobility protocol for enhancing Internet on the train.
*Proc. ITST’08,*Oct 2008.Google Scholar - Stewart R: Stream control transport protocol.
*IETF RFC4960,*2007.Google Scholar - Stewart R, Xie Q, Tuexen M, Maruyama S, Kozuka M: Stream control transport protocol (SCTP) dynamic address reconfiguration.
*IETF RFC5061*2007.Google Scholar - Stewart R, Ramalho M, Xie MTQ, Conrad P: Stream control transport protocol.
*IETF RFC3758,*2004.Google Scholar - IEEE: IEEE standard for information technology–telecommunications and information exchange between systems–local and metropolitan area networks–specific requirements part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) specifications amendment 6: Wireless access in vehicular environments. IEEE Std 802.11p-2010, (2010), pp. 1–51Google Scholar
- Puterman M:
*Markov Decision Processes: Discrete Stochastic Dynamic Programming*. (Wiley, New York, 1994)View ArticleGoogle Scholar - Feinberg EA, Schwartz A, (eds.), (eds.):
*Handbook of Markov Decision Processes*. ch. Markov Decision Processes in Finance and Dynamic Options (Kluwer, Boston, 2002)Google Scholar - Yu F, Krishnamurthy V, Leung V: Cross-layer optimal connection admission control for variable bit rate multimedia traffic in packet wireless CDMA networks.
*IEEE Trans. Signal Process*2006, 54(2):542-555.View ArticleGoogle Scholar - Zografos K, Androutsopoulos K, Spitadakis V: Design and assessment of an online passenger information system for integrated multimodal trip planning.
*IEEE Trans. Intell. Transport. Syst*2009, 10(2):311-323.View ArticleGoogle Scholar - IEEE , Wireless medium access control (MAC) and physical layer (PHY) specification: Medium access control (MAC) enhancement for quality of service (QoS), July 2003Google Scholar
- Bohm A, Jonsson MHandover in IEEE 802.11p-based delay-sensitive vehicle-to-infrastructure communication(Research Report, Halmstad University, Sweden, 2007)Google Scholar
- Stevens-Navarro E, Lin Y, Wong VWS: An MDP-based vertical handoff decision algorithm for heterogeneous wireless networks.
*IEEE Trans. Veh. Tech*2008, 57(2):1243-1254.View ArticleGoogle Scholar - Messac A, Melachrinoudis E, Sukam CP: Aggregate objective functions and pareto frontiers: Required relationships and practical implications.
*Optim. Eng*2000, 1(2):171-188. 10.1023/A:1010035730904MathSciNetView ArticleGoogle Scholar - Gadallah MH: On multi-objective optimization problem: modeling issues and numerical verification.
*Proc. 34th Int’l, Conf. Computers and Industrial Eng.*(San Francisco, USA, Nov 2004), pp. 635–640Google Scholar - Qiu X, Chawla K: On the performance of adaptive modulation in cellular systems.
*IEEE Trans. Commun*1999, 47(6):884-895. 10.1109/26.771345View ArticleGoogle Scholar - Ali S, Leung V: Dynamic frequency allocation in fractional frequency reused ofdma networks.
*IEEE Trans. Wirel. Commun*2009, 8(8):4286-4295.View ArticleGoogle Scholar - Song G, Li Y: Cross-layer optimization for ofdm wireless networks-part i: theoretical framework.
*IEEE Trans. Wirel. Commun*2005, 4(2):614-624.MathSciNetView ArticleGoogle Scholar - Li Z, Yan Z: Train-ground communication in CBTC based on 802.11b: Design and performance research.
*Proc. WRI Int’l Conf. Commun. and Mobile Computing*(Kunming, China, Jan 2009), pp. 368–372Google Scholar - Green DB, Obaidat MS: An accurate line of sight propagation performance model for ad-hoc 802.11 wireless LAN (WLAN) devices.
*Proc. IEEE ICC’02*(New York, USA, Apr 2002), pp. 3424–3428Google Scholar - Babich F, Lombardi G, Valentinuzzi E: Variable order Markov modeling for LEO mobile satellite channels.
*Electron. Lett*1999, 35(8):621-623. 10.1049/el:19990455View ArticleGoogle Scholar - Si P, Yu FR, Ji H, Leung V: Distributed sender scheduling for multimedia transmission in wireless mobile peer-to-peer networks.
*IEEE Trans. Wirel. Commun*2009, 8: 4594-4603.View ArticleGoogle Scholar - Babich F, Lombardi G: A measurement based Markov model for the indoor propagation channel.
*Proc. IEEE VTC’97, vol. 1*(Phoenix, AZ, May 1997), pp. 77–81Google Scholar - Wang HS, Chang P-C: On verifying the first-order Markovian assumption for a rayleigh fading channel model.
*IEEE Trans. Veh. Tech*1996, 45(2):353-357. 10.1109/25.492909View ArticleGoogle Scholar - Yu FR, Sun B, Krishnamurthy V, Ali S: Application layer qos optimization for multimedia transmission over cognitive radio networks.
*Wirel. Netw*2011, 17: 371-383. 10.1007/s11276-010-0285-8View ArticleGoogle Scholar - Pimentel C, Falk TH, Lisbôa L: Finite-state Markov modeling of correlated Rician-fading channels.
*IEEE Trans. Veh. Tech*2004, 53(5):1491-1501. 10.1109/TVT.2004.832413View ArticleGoogle Scholar - Guan YL, Turner LF: Generalized FSMC model for radio channels with correlated fading.
*IEE Proc. Commun*1999, 146(2):133-137. 10.1049/ip-com:19990130View ArticleGoogle Scholar - Wei Y, Yu FR, Song M: Distributed optimal relay selection in wireless cooperative networks with finite-state markov channels.
*IEEE Trans. Veh. Tech*2010, 59: 2149-2158.View ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.