A cognitive control approach to interference mitigation in communications-based train control (CBTC) co-existing with passenger information systems (PISs)

Abstract

As a key component of urban rail transit systems, communications-based train control (CBTC) is an automated train control system using train-ground communications to ensure efficient operation of rail vehicles. In addition to CBTC systems, passenger information systems (PISs) are adopted in urban rail transit systems to improve quality of service (QoS) offered to customers. The interference between CBTC systems and PISs is an important factor impacting QoS of both CBTC systems and PISs. With recent advances in cognitive dynamic systems, in this paper, we take a cognitive control approach to interference mitigation considering the co-existence of CBTC systems and PISs. In our cognitive control approach, the notion of information gap is adopted to quantitatively describe effects of interference on CBTC. The wireless channel is modeled as a finite-state Markov chain with multiple state transition probability matrices, which are derived from real field measurements. Simulation results show that the proposed cognitive control approach can significantly improve performance of CBTC train-ground communications under interference from co-existing PISs.

1 Introduction

Recently, urban rail transit systems are developing rapidly around the world. Due to the huge urban traffic pressure, improving efficiency of urban rail transit systems is in high demand. As a key component, communications-based train control (CBTC) is an automated train control system using train-ground communications to ensure the efficient operation of rail vehicles [1]. CBTC can improve utilization of railway network infrastructure and enhance the level of service offered to customers [2].

Designing a train control system based on WLANs is a challenging task. Due to unreliable wireless communications and complicated wave propagation environment, train control performance can be significantly affected by the WLAN-based train-ground communications [3]. Performance issues in CBTC systems have attracted a lot of interests recently. A handoff mechanism designed for high-speed railway systems is proposed in [4] by building up a neighboring cell list to improve handoff performance. In [5], a new handoff algorithm of GSM-R is developed based on on-vehicle for railway systems. The cross-layer approach is also introduced in CBTC system to enhance performance of train control [6]. In [7], energy-efficient train control schemes are studied in CBTC systems. In order to deal with the packet drops in CBTC system, the network controlled system (NCS) is adopted in [8]. Similarly, communication latency is also considered as the main factor which could bring reduction of train control performance, and cognitive control is successfully applied to CBTC systems to help the train run at the optimal trajectory curve [9].

In addition to CBTC systems, passenger information systems (PISs) are adopted in urban rail transit systems to improve the quality of service (QoS) offered to customers. Based on computer technologies, multimedia networks and wireless communications, PISs can provide real-time services (e.g., video) information to passengers at the station or onboard, including arriving and departure time, news, sport games, and advertisements. When emergencies (e.g., fire and terrorist attacks) happen, PISs can supply signs of emergency evacuation.

Both CBTC systems and PISs use wireless local area networks (WLANs) as the main method for train-ground communications due to available commercial-off-the-shelf (COTS) WLAs equipment with low costs. As a result, interference between CBTC systems and PISs is an important factor impacting QoS of both CBTC systems and PISs. Generally, interference can have significant impacts on signal-to-interference-noise-ratio (SINR), packet loss rate, and latency of train-ground communications, and consequently interference can severely affect train control performance, train operation efficiency, and utilization of railway [10–12].

Although interference is an important issue in CBTC systems and PISs, this issue has not been well studied in literatures. This is not very surprising, as most of existing research works focus on individual CBTC systems or PISs, and interference between them is largely ignored. To fill this gap, we study interference issues in co-existing CBTC systems and PISs. In addition, compared to PISs, CBTC systems have more stringent requirements for service availability and latency, and less service availability in CBTC could cause train emergency braking that reduces efficiency of train operation. Therefore, we study to improve performance of CBTC train-ground communications under interference from co-existing PISs.

The rest of this paper is organized as follows. Overview of CBTC systems and PISs is presented in Section 2. Section 3 presents real-field measurements on co-existing CBTC systems and PISs. Section 4 proposes a cognitive control approach to reduce impacts of interference from the PIS. Section 5 presents formulations of the cognitive control approach. Simulation results are discussed in Section 6. Finally, we conclude the study in Section 7.

2 Overview of CBTC systems and PISs

In this section, we describe CBTC systems and PISs in urban rail transit systems. Figure 1 shows a scenario where a CBTC system and a PIS coexist. For a CBTC system, train-ground communications play an important role. There are a wired backbone network and a wireless network for train-ground communciations. The wayside equipment, including automatic train supervisor (ATS), zone controller (ZC), computer interlocking (CI), data storage unit (DSU) is connected together through the wired backbone network. Through the wireless network, the wayside equipment can communicate with the onboard equipment, which is also called vehicle onboard controller (VOBC) including automatic train operator (ATO), automatic train protection (ATP), and the mobile station (MS). Continuous bidirectional wireless communications between the MS and access points (APs) are adopted in modern CBTC systems to guarantee high efficiency and reliability of train operation. Through the CBTC wireless link, the real-time state of a train is updated to ZC while the movement authority (MA) from ZC is received by the train, where MA is generated based on the state of the front train. According to the real-time state of a train and the practical state of railway lines, ATS and CI will set routes for the train.

Fig. 1

Coexistence of a CBTC system and a PIS

Similarly to CBTC systems, a PIS consists of onboard devices, ground equipment, and a network. Onboard devices include cameras, screens, and PIS MSs, while ground equipment includes a video server, a monitor center and PIS APs. PIS train-ground communications ensure passengers onboard can get real-time information from the video server, and the monitor center can real-timely receive monitoring pictures of the train interior and the station.

Theoretically, based on the control mechanism of CBTC systems, when each train runs at the same speed and has the same braking capability, distance between adjacent trains could be a few meters. However, the coverage of one AP is usually several hundred meters (e.g., in tunnels), when a train moves away from the coverage of an AP and enters the coverage of another AP, the handoff procedure happens. Unreliable wireless communications and handoffs can result in communication interruption and long latency. When the PIS wireless link exists, the CBTC wireless link may be affected. We performed measurements in Beijing Subway Lines to quantitatively determine the interference effects on CBTC wireless links brought by PIS wireless links, which will be described in the next section.

3 Real-field measurements on co-existing CBTC systems and PISs

In this section, we describe real-field measurements on co-existing CBTC systems and PISs. Then, we discuss effects of interference from PISs on CBTC wireless links according to measurement results.

Measurements were performed in a section of Beijing Subway Line 1 tunnels, where there are no other interference sources. The network topology in measurements is shown in Fig. 1. CBTC APs and PIS APs are located on tunnel walls along tracks and deployed alternately, which means there is a PIS AP between two adjacent CBTC APs in order to decrease impacts of interference.

3.1 Measurement methodology

The goal of measurements is to determine impacts of interference from a PIS on a CBTC system. According to characteristics of CBTC, we used a server at the station to simulate actual control information sent from the control center, while a PC was set in the train to receive data packets generated by the server at the station through the CBTC wireless link. The PIS server sent encoded entertainment video streams to receivers in the train through the PIS wireless link. Similarly, onboard cameras captured real-time images that can be sent to the monitor center at the station through the PIS wireless link. The CBTC wireless link occupied the channel 1, while the PIS wireless link worked at the channel 6. The basic configuration of measurements is shown in Table 1 and test principles are shown as Fig. 2.

Fig. 2

Measurement principles

Table 1 Configurations of measurements

The train ran at the speed of 30 km/h. We captured signal-to-noise ratio (SNR) samples and got statistical results of packet loss at each communication cycle. The measurement equipment is shown in Fig. 3.

Fig. 3

Measurement equipment

3.2 Measurement results

Measurement results demonstrate variations of SNR and packet losses in CBTC systems due to interference from the PIS.

3.2.1 SNR

Statistical results of SNR with or without interference from the PIS are shown in Table 2. We can see interference from the PIS significantly increases variations of SNR, while mean value of SNR almost keeps the same. SNR data was captured by the onboard PC through the simple network management protocol (SNMP) from the wireless device. Figure 4 shows SNR frequently reaches 0 when interference exists. Existence of interference leads to communication interruptions, then the onboard PC cannot get the response from wireless devices after an SNMP request is sent and value of SNR is set to be 0.

Fig. 4

SNR of the CBTC wireless link with/without interference from the PIS

Table 2 Mean and variance of SNR

3.2.2 Packet losses

As shown in Table 1, the transmission rate of simulated CBTC service is 200 k b p s and the packet size is 200 b y t e s. Therefore, the number of packets transmitted in a second is 125. Figure 5 shows percentage of packet losses without interference is much less than that with interference. Mean percentage of packet losses in a communication cycle without interference is about 0.0795 while mean value of that with interference is about 0.2724, which means existence of interference from the PIS significantly affects the packet transmission in CBTC systems.

Fig. 5

Percentage of packet losses at each communication cycle when interference does or does not exist

Figure 6 shows the relationship between percentage of packet losses and ID of the associated AP, which indicates that packet losses without interference happens when the associated AP changes. However, Fig. 7 illustrates packet losses with interference happens even if the associated AP does not change. As we know, changes of the associated AP mean handoffs happen, which can bring packet losses. Comparing Figs. 6 and 7, we can observe that, when a handoff happens, the percentage of packet losses obviously increases. In other words, the interference from the PIS leads to the decrease of the handoff performance.

Fig. 6

Relationships between percentage of packet losses in a communication cycle and the associated AP at each communication cycle when interference does not exist

Fig. 7

Relationships between percentage of packet losses in a communication cycle and the associated AP at each communication cycle when interference exists

In our experiment, PIS wireless links and CBTC wireless links work at channels 6 and 1, respectively. Theoretically, there is no overlapping frequency band [13]. However, according to measurement results, SNR is not obviously affected while packet losses increase. Based on 802.11 protocols, a WLAN station applies the clear channel assessment (CCA) mechanism to detect the channel state (busy or idle). Due to the high traffic of the PIS (e.g., 2–4 Mbps), in the view of frequency domain, the frequency band of channel 6 is almost occupied completely and it is possible that the power of the PIS wireless link “leaks” to the adjacent channel (i.e., channel 1), which can cause the CBTC WLAN equipment working at channel 1 to misjudge the channel state according to the CCA mechanism. Therefore, the interfered equipment could perform unnecessary backoff procedures or suffered from collisions, which can lead to packet losses. However, due to propagation environment of CBTC scenarios, the CBTC wireless link could also be affected by fading, such as multipath fading, Doppler frequency shift, which can also bring packet losses.

4 A cognitive control approach to reduce impacts of interference from the PIS

Based on above observations, we propose a cognitive control approach to reduce impacts of interference from the PIS on the CBTC system. The proposed approach dynamically makes decisions, such as the packet size, the transmission interval, handoff decisions, and physical layer parameters, in order to guarantee the data transmission between the train and the control center in the CBTC system.

In this section, we describe cognitive control and its application in CBTC systems. Then, the cost function is defined, and a reinforcement learning algorithm to derive the optimal policy in cognitive control is also illustrated.

4.1 Overview of cognitive control

Cognitive control was originally developed in neuro-science and psychology (e.g., [14]). Recently, it has emerged as a new engineering discipline [15]. The application of cognitive control was proposed by Professor Simon Haykin in 2012, and at present, cognitive control is still in an early stage of development [16]. However, the concept of cognitive control has been successfully applied on the cognitive radio and the cognitive radar [17]. For cognitive radio networks, the concept of cognitive control is widely used for multiuser sequential channel sensing and access [18]. For cognitive radar systems, the application of cognitive control shows good environmental interactivity and helps systems perceive the environment in an intelligent way and adaptively reacts [19]. Cognitive control can be adopted to provide intelligent strategies for reconfiguration of wireless networks for wireless systems and its surrounding environment [20]; it can use computational intelligence (CI) to deal with limitations of cross-layer design such as being limited to interactions between two or three layers and having to be completely redesigned if network requirements change. In robot engineering, cognitive control is proposed to develop more robust and better adapted robots, and it will be treated as a design tool to obtain the appropriate dynamics for bio-inspired real time systems [21]. For intelligent video surveillance systems, the cognitive control approach is applied to support human decisions through analysis of a huge amount of information which can help to detect densely populated areas [22]. As mentioned above, cognitive control can be applied in a variety of fields. In this paper, we take a cognitive control approach to interference mitigation considering the co-existence of CBTC systems and PISs.

Compared with other control methods, such as adaptive control [23] and neuro-control [24], cognitive control has the following advantages. There is no memory block in the adaptive controller, which reduces the ability of learning. The neuro-controller lacks intelligence, which is distributed throughout the cognitive dynamic system and can make the system in an orderly fashion. The feedback information plays the key role in a cognitive dynamic system and cognitive control describes a control system from the information flow perspective. The feedback information obtained by the perceptual part is partitioned into relevant information and redundant information. However, the required information used to make correct decisions is called sufficient information in cognitive control. The information gap is defined as the difference between sufficient information and relevant information obtained from measurements. The goal of cognitive control is to decrease the information gap.

Cognitive control can be applied in the train-ground communication of CBTC systems to decrease effects of interference and increase the channel quality. The information gap of the CBTC system can be defined according to performance of train-ground wireless communications. As CBTC systems are safety-critical, the requirements of reliability for train-ground communications are high, which means time delay and packet losses should be minimized to guarantee data transmission between the train and the control center.

4.2 Brief descriptions of cognitive control

For a cognitive control system, a cognitive controller makes corresponding decisions based on the knowledge and selects the optimal action which has influence on the system itself or the environment, where the goal is to decrease the information gap. For example, when there is a quadratic optimal controller, the cost function is

$$ J=(s-\tilde{s})^{T}W(s-\tilde{s})+u^{T}Ru $$

(1)

where $\tilde {s}$ is the the desired state of the system, s is the actual state of the system, u is the physical control vector, and matrices W and R are applied as desired weights for systems’s state and control. The objective is to minimize the cost function.

Moreover, cognitive control adds another term about information gap to (1). The resulting cost function can be formulated as [15]

$$ J=\left(s-\tilde{s}\right)^{T}W\left(s-\tilde{s}\right)+u^{T}Ru+\beta G $$

(2)

where G is the information gap and β is a scalar.

In cognitive control, cognitive actions are the most concerned. Actions are determined through the implementation of reinforcement learning (RL). Reinforcement learning is the process by which the agent learns an approximately optimal policy through trial and error interactions with the environment. At each communication cycle, RL can determine a cognitive action to decrease the information gap according to the reward. As a result, the objective of RL is to find a policy that is updated by rewards provided by the environment, which means minimizing the cumulative amount of cost over a long run [25].

5 Formulation of the cognitive control approach

In this section, we formulate the cognitive control approach to reduce impacts of interference from the PIS, where the MAC layer model and the physical layer model are also shown.

The general structure of cognitive control consists of an executive part including a cognitive controller, perceptual part and the practical environment. In our cognitive control approach, the CBTC MS is considered as the perceptual part that can calculate the information gap according to QoS indexes. Based on the information gap, the cognitive controller can make decisions that should be performed on the CBTC MS. The cognitive decisions are applied at the MAC layer and the physical layer. As a result, we need to model the MAC layer and the physical layer. In fact, the model of the MAC layer is a service model including the packet size and the packet sending frequency, where it is a periodic service for CBTC systems. For the PIS, the service model is related to video. Next, we will determine the MAC layer model and the interference caused by the adjacent channel.

5.1 The MAC layer model

The queueing model can be used to describe characteristics of the WLAN MAC layer, where the key is to model the backoff procedure. According to the random access scheme of CSMA/CA protocols, a random packet is transmitted after the DIFS time and the backoff time, which is uniformly distributed in the range [ 0,C W−1]. CW is the value of contention window which has a lower and upper bounds denoted as C W _min and C W _max . For the first backoff, CW is initialized as C W _min . Then, each backoff will double CW until it reaches C W _max . In 802.11 standards, C W={16,32,64,128,256,512,1024} and C W _min =16, C W _max =1024. Then the general expression of CW can be denoted as

$$ {CW}_{\alpha}=\left \{ \begin{array}{l} 2^{4+\alpha},0 \leq \alpha \leq 6\\ 2^{10},\alpha>6 \end{array} \right. $$

(3)

where α is times of backoff. As a result, the backoff time can be defined as

$$ {BackoffTime}_{\alpha}=Random\left(\left[0,{CW}_{\alpha}-1\right]\right)\times aSlotTime $$

(4)

where aSlotTime is a constant time corresponding to IEEE 802.11g standards [13].

Consider the collision probability denoted by p, we can get

$$ Pr\{CW=W\}=\left \{ \begin{array}{ll} p^{\alpha-1}(1-p), for\quad W=2^{\alpha+3}{CW}_{min}\\ p^{m}, for\quad W={CW}_{max} \end{array} \right. $$

(5)

The parameter m is defined in the protocol which means the contention window is kept as C W _max when times of backoff are no less than m. According to the definition in (4), the probability B a c k o f f T i m e∈ [ 0,C W _min ) is highest. As a result, the probability B a c k o f f T i m e=i could be derived based on (3).

$$ \begin{aligned} &P(BackoffTime=i)=\\ &\left \{ \begin{array}{ll} \sum^{m-1}_{\alpha=0}\frac{p^{\alpha}(1-p)}{2^{\alpha} {CW}_{min}}+\frac{p^{m}}{{CW}_{max}}& 1 \leq i \leq {CW}_{min}\\ & \\ \sum^{m-1}_{\alpha=\lfloor \log_{2}^{i} \rfloor}\frac{p^{\alpha}(1-p)}{2^{\alpha} {CW}_{min}}+\frac{p^{m}}{{CW}_{max}} & 2^{\lfloor \log_{2}^{i} \rfloor-1}{CW}_{min}+1 \leq \\ &i \leq 2^{\lfloor \log_{2}^{i} \rfloor}{CW}_{min}\\ \frac{p^{m}}{{CW}_{max}}\quad & 2^{m-1}{CW}_{min}-1\leq \\ &i \leq {CW}_{max} \end{array} \right. \end{aligned} $$

(6)

where ⌊·⌋ is a function to round numbers, C W _min =16 and C W _max =1024.

However, considering the CBTC scenarios, collision is not the only factor that causes unsuccessful packet transmissions, and the channel is indeed also an important factor which can impact the usual packet transmissions. As a result, symbol p mentioned in Eqs. (5) and (6) is the probability of unsuccessful transmissions, which includes both the collision of the MAC layer and the channel fading of the physical layer. For a successful transmission, the probability is 1−p. Therefore, we can calculate the expected value of backoff time.

$$ \begin{aligned} &\overline{BackoffTime} = \\ &(1-p)\frac{{CW}_{min}}{2}+{\ldots}+p^{m}(1-p)\frac{\Sigma^{m}_{i=0}2^{i}{CW}_{min}}{2}+\\ &{\ldots}+p^{m_{r}}\frac{\Sigma^{m}_{i=0}2^{i}{CW}_{min}+(m_{r}-m)2^{m}{CW}_{min}}{2} \end{aligned} $$

(7)

The PIS service is a video stream. For simplicity, we consider both the CBTC service and the PIS service as periodic services, where the packet interval and the packet size are different. For the PIS service, the uplink is the Closed Circuit Television (CCTV) while the downlink is the PIS video stream, and related parameters are shown in Table 3. Packet sizes of PIS and CCTV are 20,480 B y t e s and 10,240 B y t e s, respectively. However, according to 802.11 MAC protocols [13], the maximum MPDU (MAC protocol data unit) length is 4096 B y t e s. As a result, the PIS frame and CCTV frame should be fragmented, and the maximum fragmentation of a MPDU is 3000 B y t e s. A PIS frame could be divided into $\frac {VideoFrameLength}{3000\times 8}$ fragments.

Table 3 Parameters of a PIS

The transmission procedure of a multiple-fragment MSDU¹ is shown in Fig. 8. Figure 9 illustrates the basic procedure of a 802.11 node accessing the channel.

Fig. 8

Transmission procedure of a multiple-fragment MSDU

Fig. 9

Transmission procedure of a normal MSDU

According to the fragmented MPDU transmission procedure in Fig. 8, the duration for a successful transmission of a PIS frame is calculated as follows:

$$ T_{video}=\frac{L_{v}}{R} + (aSIFSTime + anACKTime)\times \lceil\frac{L_{v}}{l_{m}}\rceil $$

(8)

where aSIFSTime is the period of short interframe space (SIFS), R is the transmission rate, anACKTime is the time for the transmitter to send an acknowledge frame, ⌈·⌉ is the top integral function, L _v is the length of a video frame and l _m is the maximum length of MPDU stated in 802.11 protocols. Therefore, through the comparison of the CBTC service and the PIS service, it is possible that PIS nodes will occupy the channel for a long time as the size of MPDU is large, which can affect the periodic transmission of other wireless link. When the PIS link and the CBTC link work at adjacent channels, impacts on the CBTC link are from the side lobe of PIS link frequency spectrum which will cause the CBTC nodes misjudge the channel is busy. As a result, the channel model is needed to calculate the probability that the adjacent channel is affected by the PIS link, which is introduced in the next section.

5.2 The channel model

As the unsuccessful packet transmission could be caused by the channel contention with other WLAN nodes and the channel variations, it is necessary that the channel model illustrates characteristics of the CBTC link and interference effects from the PIS link. As a result, in the CBTC scenarios, the part of interference signals could work on the adjacent channel which could affect the CBTC link from the PIS. Therefore, the multi-matrices finite state Markov channel model (FSMC), as described in [9] has been employed in the paper which can also be used to describe characteristics of interference signals.

5.2.1 The multi-matrices finite state Markov channel model

In order to completely quantify effects of interference on CBTC systems, we build a finite state Markov channel (FSMC) model based on real field measurements. FSMC models have been widely accepted in literatures as an effective approach to characterize wireless channels including high-speed railway channels [26], and Rayleigh fading channels [27]. In FSMC models, the SNR range of the received signal can be partitioned into non-overlapping levels. Then, received SNR can be modeled as a random variable evolving according to a finite state Markov chain with state transition probabilities, which can be obtained from real field channel measurements.

Due to effects of large scale fading, the amplitude of SNR depends on distance between the transmitter and the receiver. It is obvious that SNR is usually high when the receiver is close to the transmitter; while it is low when the receiver is far away from the transmitter. As a result, the transition probability from the high channel state to the low channel state is different when the receiver is near or far away from the transmitter, which means that the Markov state transition probability is related to the location of the receiver. Therefore, only one state transition probability matrix, which is independent of the location of the receiver, may not accurately model the channels.

As our previous work in [28], we divide the communication coverage of one AP into L intervals. For each interval, we use the LLOYD-MAX method to partition the SNR amplitude into several levels, which are non-uniformed distributed. And, the non-uniformed partitioning can be useful to obtain more accurate estimates of system performance measures [29].

5.2.2 The interference from the PIS working at the adjacent channel

We can use the FSMC model to describe characteristics of the PIS link and the CBTC link. Although the power leaked into the adjacent channel is indeed quite low compared with the transmitted signal power (at the adjacent channel), it is sufficient to cause the adjacent channel interference according to the 802.11 protocols [30]. Based on the CCA mechanism [13], the transmission of a valid packet which is received with a level equal to or greater than sensitivity (−82 d B m for 20 M H z channel spacing, −85 d B m for 10 M H z channel spacing, and −88 d B m for 5 M H z channel spacing) shall cause CCA to indicate busy with a probability >90% within 4 μ s for 20 M H z channel spacing, 8 μ s for 10 M H z channel spacing, and 16 μ s for 5 M H z channel spacing). As both the CBTC wireless link and the PIS wireless link adopt 802.11 g as the main method which uses 20 M H z channel spacing, if the interference signal leaked into the adjacent channel is higher than −82 d B m, it is possible that WLAN nodes working at the adjacent channel will judge the channel is busy which could cause packet dropping.

Based on the multi-matrices FSMC model, we can obtain strength of interference signals. As the receiver is equipped with a band-width filter, we could apply the filter’s frequency response to calculate the leaked interference signal. Assume the filter’s frequency response function is F(f) and the power spectral density is P(f). According to the channel spacing of 802.11 g, the interference factor can be derived as follows

$$ IF=\frac{\int^{\frac{w_{a}}{2}}_{-\frac{w_{a}}{2}}P(f)F(f)df}{\int^{\frac{w}{2}}_{-\frac{w}{2}}P(f)df} $$

(9)

where w _a is the bandwidth of the adjacent channel, and w is the bandwidth of the interference signal. Therefore, IF can be used to calculate the interference signal leaked into the adjacent channel based on the multi-matrices FSMC model.

Based on the time axis and the location of a train, the FSMC model generates channel samples. With the IF function, it is possible to obtain the adjacent interference element of the generated samples. Then, compared with the sensitivity value, if the interference element is larger, wireless nodes will judge the channel is busy with certain probability.

As a result, for CBTC systems, we can calculate the packet loss rate through performance of wireless communications with the FSMC model, and consider both the retransmission probability and time delay.

5.3 Q-learning Model

As the wireless channel is modeled as the FSMC, each channel state can only transit to adjacent channel states. In order to utilize the Q-learning algorithm, system states, actions, and rewards should be identified.

5.3.1 Systems states and actions

For the CBTC wireless link, a cognitive controller on the train should help the CBTC MS to make some decisions in order to improve communication performance. As mentioned above, actions could be implemented at the application layer, the MAC layer and the physical layer. Generally, we assume that the place occupied by a train can only be covered by two adjacent APs, as a result, a better handoff decision strategy could be generated through cognitive control. As we consider the CBTC system is based on the MIMO-WLAN method in the paper, the multiplexing gain needs to be determined in order to improve the channel quality. The packet interval and the packet size could also be controlled to reduce impacts caused by PIS interference. Therefore, the action at the kth communication cycle could be denoted as $a_{k} = \left \{a^{s}_{k},a^{i}_{k},a^{h}_{k},a^{m}_{k}\right \}$, where $a^{s}_{k}$ is the packet size action, $a^{i}_{k}$ is the packet interval action, $a^{h}_{k}$ is the handoff action and $a^{m}_{k}$ is the multiplexing gain action.

Corresponding to actions, current states should indicate the physical layer, the MAC layer and the application layer of wireless communications. As the multi-matrices FSMC model is applied in the paper, the channel state is related to distance between the TX and the RX and defined by ${\gamma _{1}^{l}}_{k}$ and ${\gamma _{2}^{l}}_{k}$, which are SNR levels of two successive APs, and l is the number of intervals between two successive APs. As a result, the state s _k is denoted as $s_{k}=\left \{{\gamma _{1}^{l}}_{k}, {\gamma _{2}^{l}}_{k}, ID, PLR, PI, PS\right \}$, where ID is the identification number of the current associated AP, PLR is the packet loss rate, PI is the packet interval, and PS is the packet size. When ID changes, the handoff procedure happens.

5.3.2 Cost function

According to the definition of cost function for cognitive control in (2), there are three parts: the system state, the control vector, and the information gap. We should determine the control vector and the information gap of CBTC wireless communications. The physical control vector u _k in (1) could be considered as the actions at each communication cycle, which is a _k . However, the information gap is the key of cognitive control, which is adopted to quantitatively describe effects of interference. For the cognitive radar and the cognitive radio, the information gap is defined as the difference between relevant information and sufficient information representing the information needed for achieving minimal risk [15]. According to principles of wireless communications, the information gap can be defined based on the QoS. QoS indexes could be given as D _k ,L _k , where D _k and L _k are the time delay and the number of packet loss at the kth communication cycle, and G _k ={D _k ,L _k }

Therefore, the cost function of cognitive control in the paper is shown as follows:

$$ J = \left(s_{k}-\hat{s_{k}}\right)Q\left(s_{k}-\hat{s_{k}}\right)^{T}+a_{k}Ra_{k}^{T}+\beta G_{k}G_{k}^{T} $$

(10)

where $\hat {s_{k}}$ is the desirable state of the system. As the system state is defined as the channel state, we take the desirable state as the channel state at the previous communication cycle, which means $\hat {s_{k}}=s_{k-1}$.

6 Simulation results and discussions

In the section, we present simulation results to show performance of the proposed cognitive control approach in CBTC systems. We first present details of simulations. Next, simulation results are discussed.

We implement simulations using Matlab. According to the multi-matrices FSMC model, we generate signal samples of the PIS wireless link, as the PIS system and the CBTC system work at the same environment. At each communication cycle, the cognitive controller can decide to transmit packets based on the state of the channel where signals of the PIS wireless link can lead to the misjudge of CBTC systems according to the CCA mechanism. Through the cognitive control, the multiplexing gain, handoff decisions, packet intervals and packet sizes are changed dynamically according to the reward function and the corresponding Q value generated by Q learning. As a result, we can obtain the optimal strategy that can reduce effects of interference caused by the PIS and improve performance of CBTC systems. In simulations, there are 35 APs and the coverage of each AP is 500 m according to [28].

Figure 10 shows the convergence of Q learning, where x label is learning steps and y label is mean value of the cost function for one learning process. Value of the cost function vibrates greatly at the beginning. After about 600 steps, the cost function reaches a stable value, which means optimal polices have been obtained. The packet loss and time delay are compared with experimental results. According to Fig. 5, with the existence of interference, the number of packet losses increases both at the handoff procedure and the normal transmission. In addition, with the learning process is being implemented, mean value of time latency is becoming lower and lower as shown in Fig. 11, where it is 5 ms when cognitive control converges to a stable state. The reinforcement learning algorithm can obtain the probability of misjudging the state of the channel in the cognitive control approach according to characteristics of information transmission of the PIS including the transmission interval and the packet length. Therefore, based on the multi-matrices FSMC channel model, the cognitive control approach can determine the optimal strategy that can help the CBTC wireless link avoid interference from the PIS wireless link. Figure 12 demonstrates the packet loss rate decreases with the increase of learning times and the total number of packet losses for one entire learning process. The entire learning process includes 35 APs and the total number of packet loss is less than 150. As a result, for each handoff procedure, there are no more than 5 packet losses. Compared with packet losses for each communication cycle shown in Fig. 5, performance is improved with the application of cognitive control. In the manuscript, we take 0.2 s as a learning step, and the algorithm reaches the convergence after 600 steps. Similarly, Fig. 13 demonstrates the time delay versus the location of the train, where maximum value of time delay is no more than 20 m s.

Fig. 10

Convergence of the Q learning algorithm

Fig. 11

Mean value of time latency between consecutive instants of handoffs

Fig. 12

Packet losses versus learning steps

Fig. 13

Time delay versus the location of the train

Generally, handoff latency is much larger than the basic time delay, which means performance of handoffs is the key factor affecting CBTC wireless communications. In current CBTC systems, the handoff mechanism is triggered when signal strength of optional APs is “X” dBm higher than that of the current associated AP, and the AP with the highest signal strength will be selected as the next associated AP, which could be called “Greedy policies”. In order to verify advantages of the cognitive control approach, we compare simulation results of cognitive control policies and greedy policies of handoffs. In Fig. 14, handoff decisions based on greedy policies and cognitive control policies are illustrated. Obviously, reedy policies bring frequent Ping-Pong handoffs while few Ping-Pong handoffs happen under cognitive control polices. Details of handoff decisions are shown in the enlarged part in Fig. 14. Performance of handoffs is demonstrated in Fig. 15, where handoff latency under greedy policies is much higher than that under cognitive control policies. We find that greedy handoff decisions can bring delay larger than 1s, which may make the MA discarded and may lead to the useless braking of trains. However, handoff latency under cognitive control policies is dozens of milliseconds, which cannot basically affect the transmission of control commands. As shown in Fig. 16, handoffs occur at different SNR levels with the application of cognitive control compared with greedy policies. Obviously, cognitive control can make handoff happen at an appropriate SNR level, which is based on the reinforcement learning. However, greedy policies make some handoffs happen at the higher SNR level (>=50 d B).

Fig. 14

Handoff decisions under greedy policies and cognitive control policies

Fig. 15

Handoff latency under greedy policies and cognitive control policies

Fig. 16

SNR levels handoffs happen under different polices

Considering frequencies of handoffs and value of handoff latency, we can derive the availability of train-ground communications as follows [31].

$$ A_{av}=\frac{MTTF}{MTTF+MTTR} $$

(11)

where MTTF (mean time to failure) denotes mean time between adjacent handoffs, and MTIR (mean time to repair) denotes mean value of handoff latency. The availability of different policies for the whole subway line is shown as Table 4.

Table 4 The availability of train-ground communications under different policies

For CBTC scenarios, packet losses could be caused by the varying channel, handoff procedures and interference. However, the application of MIMO can improve the transmission performance, and the appropriate handoff decision can reduce time delay. As the wireless link is broken during the handoff procedure, which can directly determine the number of packet losses. A successful handoff procedure consists of several management frames, which can be impacted by interference. According to learning of the channel state, the handoff decision determined by the cognitive control can bring the optimal handoff performance, reducing handoff delay and corresponding packet losses, as shown in Fig. 13. Handoff latency is demonstrated in Fig. 15, where the handoff latency keeps at a low level. We can also observe that the cognitive control approach can reduce the Ping-Pong handoffs, which happens frequently in experimental results shown in Fig. 14.

7 Conclusions

In this paper, we studied to improve performance of CBTC train-ground communications under interference from co-existing PISs. We proposed a cognitive control approach to interference mitigation considering co-existence of CBTC systems and PISs. In the proposed cognitive control approach, we introduced the information gap, which is used to describe effects of interference on CBTC. We discussed effects of interference on the CBTC wireless link from the PIS through experimental results. Based on the cognitive control formulation, reinforcement learning was used to get the optimal strategy. Moreover, wireless channels of both the CBTC system and the PIS are modeled as the multi-matrices finite state Markov chains, which can bring accurate channel samples and enhance the learning speed of the proposed cognitive control approach. Simulation results were presented to show that the cognitive control approach can significantly improve performance of CBTC train-ground communications under interference from the co-existing PIS. In the future, we will develop a test bed to implement our scheme in CBTC systems.

8 Endnote

¹ MSDU is a unit of data that has been passed down from an OSI layer to a lower layer and that has not yet been encapsulated into a MPDU by a lower layer. The MSDU architecture is shown in Fig. 6-1 of the IEEE standard [13], which describes the relationship between MSDU and MPDU. For 802.11 protocols, MSDU could be considered as the Ethernet message while MPDU is the encapsulated MSDU through 802.11 protocols. As the frame header and footer could be ignored compared with the length of MSDU, in the paper, we take the MSDU the same as the MPDU.