MAC protocol for underwater acoustic sensor network based on belied state space

Abstract

This study aims to solve time-space uncertainties due to the narrow network channel bandwidth and long transmission delay of an underwater acoustic sensor network when a node is using a channel. This study proposes a MAC protocol (BSPMDP-MAC) for an underwater acoustic sensor network based on the belief state space. This protocol can averagely divide the time axis of a sensor’s receiving nodes into n slots. The action state information of a sensor’s transmission node was divided by the grades of link quality and the residual energy of each node. The receiving nodes would obtain the decision strategy sequence of the usage rights of the competitive channels of the sensor’s transmission nodes according to the joint probability distributions of historical observations and action information of channel occupancy. The transmission nodes will transmit data packets to the receiving nodes in turns in allocated slots, according to the decision strategy sequence, and the receiving nodes will predict the channel occupancy and perceive the belief states and access actions in the next cycle, according to the present belief states and actions. These experimental simulation results show that this protocol can reduce the collision rate of data packets, improve the network throughput and transmission success rate of data packets, and reduce the energy overhead of the network.

1 Introduction

The medium access control (MAC) protocol for underwater acoustic sensor network protocol is the bottom tier of the data link tier, which mainly allocates the underwater channel resources in a reasonable and effective manner for multiple underground nodes, and is the key protocol to ensure efficiency communication. At present, the research achievements on the MAC protocol for underwater acoustic sensor networks are divided into MAC protocol for resource allocation and MAC protocol for resource competition [1].

The representative MAC protocols for resource allocation include FDMA [2], TDMA [3], CDMA [4], and the corresponding improved protocols. These protocols include problems such as limited channel bandwidth for division, low-precision clock synchronization, and coding difficulties. Therefore, these cannot become the main research objects of MAC protocols for an underwater acoustic sensor network.

The MAC protocols for the resource competition are mainly divided into handshake protocol, channel reservation protocol, and the corresponding improved protocols. The typical handshake MAC protocols include the MACA-MN [5], RIPT [6], and S-FAMA [7] protocol. These protocols can obtain channels via the control packet handshake negotiation, allowing it to effectively alleviate the hiding terminal problem and reduce waste time caused by transmission delay. When data packets conflict, the backoff algorithm is used for backoff or retransmission, in order to reduce the conflictions of data packets and improve the channel reuse rate. Multiple handshakes of control packets of such protocol would lead to some overhead, which not only reduces network throughput, but also consumes the limited energy of nodes and affects the survival cycle of the whole network. The typical channel reservation MAC protocols mainly include the T-Lohi [8] protocol. The T-Lohi protocol proposes three channel reservation mechanisms, including ST-Lohi, CUT-Lohi, and AUT-Lohi. These three channel reservation mechanisms can better solve time-space uncertainties when nodes use a channel, but some problems exist. For example, the time synchronization is difficult in the communication slot for ST-Lohi. CUT-Lohi shall avoid difficulties in complicated clock synchronization, extend channel competition time, and reduce network throughput based on ST-Lohi. AUT-Lohi combines the strengths of the above two mechanisms and reduces channel competition time. For low-probability, packet tone collision is controlled, and the collision backoff algorithm is used to reduce the data packet collision of channels, reduce the loss rate of AUT-Lohi data packets, and improve the success rate of data packets. However, it cannot effectively solve the hiding terminal problem, which leads to data packet confliction and data retransmission, and waste limited bandwidth and node energy. Based on this, for the severity of time-time uncertainties, Li et al. proposed one underwater MAC protocol (PC-MAC) for presetting receiving and transmission time. This protocol mainly considers the possible position changes of transmission nodes. The receiving nodes predefine the time window of the receiving packets. Furthermore, the transmission nodes calculate the transmission time of packets based on the receiving window, in order to prevent packet conflictions [9]. QianLiangfang et al. [10] proposed the MAC protocol for underwater wireless sensor networks based on the reserved data queue. This protocol sets the channel reservation cycle and reservation confirmation cycle in data transmission and completes channel reservation by using RTS-CTS handshake. Furthermore, this method can reduce the average reservation time of nodes, decrease waiting time in the data transmission of nodes, and effectively improve network throughput [11].

These protocols consider that the whole network node state cannot be fully observed. When some new nodes join in or individual nodes are invalid due to too low energy, it will affect the normal operation of the MAC protocol. Based on analysis on massive data, the author will describe the influences of factors, such as time slot division and control packet joining on channel occupancy and data arrival rate, using the state transition probability and Markov algorithm, and model an underwater channel as the belief state space using the allocation process, predict channel occupancy according to the present belief state and action, and reasonably perceive the belief state and connection actions of the next cycle. A series of decisions are made to realize the optimal target, effectively reduce the confliction rate of data packets, improve the success rate of data packet transmission and network throughput, and reduce the energy consumption of the network.

2 Methodology

2.1 Partial observable Markov decision model

Affected by the water flow, the underwater sensor node monitors dynamic and uncertain targets. The partial observable Markov decision process (POMDP) [12] is the mathematic model to solve dynamic and uncertain problems, which has been applied in MAC protocols for underwater acoustic sensor networks [13]. The sensor nodes perceive partial and incomplete information in POMDP. The history actions of the sensor node are independent from the observed channel states. Thus, these have no non-Markov characteristics. The belief state space is introduced to make it only associated with the present belief state space, and not related to the history belief state space. The POMDP is transformed to the Markov decision process (MDP) [14] based on the belief state space.

$$ {b}^t=P\left({s}^t|{a}^t,{z}^t,{a}^{t-1},{z}^{t-1},\cdots, {a}^0,{z}^0,{s}^0\right) $$

(1)

The joint probability distribution of history observations and actions is:

$$ {b}^t\left({s}_i\right)=P\left({s}_i|{z}^t,{a}^t,{s}^{t-1}\right)=\eta O\left({s}_i,{a}^t,{z}^t\right)\sum \limits_{j=1}^{\mid S\mid }T\left({s}_j,{a}^t,{s}_i\right)P\left({s}^{t-1}={s}_j\right) $$

(2)

In this equation, η is the normalization factors:

The joint probability distribution is called as the belief state. A set of whole belief states is called as the belief state space [13], which is marked as B.

Definition 1 Assume that the POMDP model structure is described by a six-component tuple < S,A,T,R,Z,O> [9], the MDP model under the belief state is defined. The six-component tuple is defined as follows:

• (1) State set S = {s_1,s_2,…,s _n } is the limited state set of the sensor node.

• (2) The action set A = {a₁,a₂,…,a _n } is the limited action set of the sensor nodes;

• (3) The observation set Z = {z₁,z₂,…,z _n } is the limited set of the observed channel states of sensor nodes;

• (4) The transition function of the belief state is described as follows:

$$ T\left(b,a,{b}^{\hbox{'}}\right)=P\left({b}^{\hbox{'}}|a,b\right)\sum \limits_{z\in Z}P\left({b}^{\hbox{'}}|a,b,z\right)P\left(z|a,b\right) $$

(3)

In this equation, a∈A, b, b′∈B, Eq. (3) describes the action a taken under the state b and transitions to the next state b′.

• (5) The observation function is:

$$ P\left({b}_j|{b}_j,a\right)=\sum \limits_{k=1}^{\mid {z}_{b_j}\mid }P\left({z}_k|{b}_a\right)=\sum \limits_{k=1}^{\mid {z}_{b_j}\mid}\sum \limits_l^{\mid S\mid }Z\left({s}_l,a,{z}_k\right)\sum \limits_{m=1}^{\mid S\mid }T\left({s}_m,a,{s}_l\right){b}_j\left({s}_m\right) $$

(4)

In this equation, ${z}_{b_j}\subseteq Z,{z}_k\subseteq {z}_{b_j},$ ${z}_{b_j}$ is a set of observed channel states under the state bj. Equation (4) describes the probability of transition to state bj when the action a is taken under the state bi.

• (6) The belief state award function is described as follows:

$$ \rho \left(b,a\right)=\sum \limits_{s_i\in s}b(s)R\left(s,a\right) $$

(5)

Observe the channel state via b and the award value returned to the system when the action a is taken under the state s.

2.2 Partial observable Markov decision process

For easy description, the superscript indicates the time and subscript indicates the set state. s _i indicates ith state of the set, and s^t indicates the state at t time. P(s^t = s _i ) indicates the probability of the state s _i at t time. No superscript and subscript indicates the variant at the present time. The superscript ^′ indicates the variant at the subsequent time. For interactions between the MDP model based on the belief state and underwater acoustic channel, refer to Fig. 1. As presented in Fig. 1 and Eqs. (3–4), the belief state space can transform the POMDP problem to the Markov chain problem based on the state space. The MDP decision process based on the state space indicates the transformation of the state in POMDP to the belief state. The transition function is replaced with T(b, a, b′). The influences of states on the observation is changed to the influence of observations on the belief state.

Fig. 1

The interaction between MDP model and underwater acoustic channel based on internal belief state. Legend: The MDP decision process based on the state space indicates the transformation of the state in POMDP to the belief state

Definition 2 The decision strategy Π=(π₁,π₂,…,π _n ) indicates that the sensor node maps the belief state to the action set A, and the selected action can obtain the maximal accumulated value awarded by the channel:

$$ {\omega}_{\mathrm{max}}=E\left[\sum \limits_{t=0}^{\infty }{\gamma}^t{\rho}_t\right]\kern0.5em \gamma \in \left(0,1\right] $$

(6)

In this equation, γ is the discount factor and can make the expected target value converge.

The uncertainty of the underwater sensor network environment and the extended sensor network life shall be considered in the MDP decision of the belief state space. A function shall be constructed between the strategy Π and the obtained accumulative value of the channel award to make a decision. When the node is under the belief state b and the decision strategy Π, the function constructed through the Bellman principle and decision strategy value is described as follows:

$$ {\prod}_t^{\ast }(b)=\underset{\alpha }{\arg \max}\left[\sum \limits_{s\in S}b\left(s,a\right)R\left(s,a\right)+\gamma \sum \limits_{z\in Z}P\left(z|b,a\right){\Gamma}_t^{\ast}\left({b}^{\hbox{'}}\right)\right] $$

(7)

$$ {\Gamma}_t^{\ast}\left({b}^{\hbox{'}}\right)=\underset{\alpha }{\arg \max}\left[\sum \limits_{s\in S}b(s)R\left(s,a\right)+\gamma \sum \limits_{z\in Z}P\left(z|b,a\right){\Gamma}_{t-1}^{\ast}\left({b}^{\hbox{'}}\right)\right] $$

(8)

In Eq. (7), an optimal strategy was searched to map the present belief state to actions in the node competition channel. Channel occupancy is predicted according to the present belief state and actions, in order to reasonably perceive the belief state and connection action in the next cycle.

3 Results and discussion

For easy description, the underwater acoustic sensor network is comprised of n randomly distributed underwater acoustic sensor nodes, which is represented as G(V,E). V indicates the set of sensor nodes, and E represents the edge set. Each underwater acoustic sensor node corresponds to one 3D coordinate (x, y, z) in the 3D space, and the communication radius of these nodes is R.

Definition 3 The distance between two nodes, s _u and s _v , in the Euclidean space is described as follows:

$$ d\left(u,v\right)=\sqrt{{\left({u}_x-{v}_x\right)}^2+{\left({u}_y-{v}_y\right)}^2+{\left({u}_z-{v}_z\right)}^2} $$

(9)

For d(u,v) < R, the s _u node is adjacent to the s _v node, and there is one communication link between s _u and s _v . The underwater acoustic sensor node is under the half duplex operation mode, namely, the node is only the transmission or receiving node at the time t. In the communication radius R, if multiple transmission nodes send data to one receiving node, the receiving node shall reasonably schedule the channel slot to reduce channel access conflictions.

3.1 Description of node state

The time axis of the receiving nodes is averagely divided into n slots. The transmission node transmits data packets in the time slot allocated under the obtained channel use privilege. The action state a _n,t of the transmission node s _n is divided into the silent state and busy state.

$$ {a}_n=\left\{\begin{array}{c}1,\kern0.5em \mathrm{The}\ \mathrm{node}\ {s}_n\ i\mathrm{s}\ \mathrm{under}\ \mathrm{the}\ \mathrm{busy}\ \mathrm{state}\ \mathrm{at}\ t+1\ \mathrm{time}\\ {}0,\mathrm{The}\ \mathrm{node}\ {s}_n\ \mathrm{is}\ \mathrm{under}\ \mathrm{the}\ \mathrm{silent}\ \mathrm{state}\ \mathrm{at}\ t+1\ \mathrm{time}\end{array}\right. $$

(10)

Assume that the sensor node s _n is under the state ${x}_t^n=\left({l}_t^n,{e}_t^n\right)\kern0.5em \left(n=1,2,\cdots M\right)$, at the time t, ${l}_t^n$ is the link quality of the node s _n and ${e}_t^n$ is the residual energy of the node s _n . The link quality state Q = {l ₁ , l ₂ , …l _m } and residual energy state E = (e ₁ , e ₂ , …e _L ) of each node are respectively divided into M and L discrete grades.

The state transition probability matrix of the state ${l}_k^n$ and ${e}_t^n$ are as follows:

At the time t, when the node s _n is silent, its state ${x}_t^n=\left({l}_k^n,{e}_t^n\right)$ remains unchanged, namely, ${l}_{t+1}^n={l}_t^{n=i}$ and ${e}_{t+1}^n={e}_t^n$. The action state transition matrix T _n of the transmission node s _n is calculated from Uⁿ and Xⁿ is (a _ij ) _m×n , a _ij ∈{0,1}. The row vector is the M × 1 binary vector.

The transmission node s _n receives the confirmation message (ACK) and no confirmation message (NAK) observation values for free or busy receiving channels. It is assumed that the observation set is Z ₀  = {ACK,ANK} = {σ₁,σ₂}, σ₁ = 1,σ₂ = 0 and each observation value is ${z}_t^n\in {z}_0$. The node states are partially observable. Therefore, the channel access conditions at a slot depends on the present state of each node and the historical observation value. It is assumed that the size of each data packet is r bits. In Eqs. (3–4), the probability of the observation value NCK is:

$$ P\left({z}_t^n=\mathrm{NCK}|{x}_t^n,{a}_{n,t}\right)=\sum \limits_{t=1}^{\mid {z}_{b_j}\mid}\sum \limits_l^{\mid S\mid }Z\left({s}_l,\mathrm{NCK},{z}_t\right)\sum \limits_{m=1}^{\mid S\mid }T\left({s}_m,\mathrm{NCK},{s}_l\right){b}_j\left({s}_m\right)\times \kern0.5em \sum \limits_{l=t+1}^r{C}_r^l{P}_e{\left({x}_t^n\right)}^l\left(1-{P}_e{\left({x}_t^n\right)}^{r-l}\right) $$

(11)

The ${p}_e\left({x}_t^n\right)$ is the data transmission error rate of the sensor transmission node s _n .

3.2 Channel scheduling strategy

The state transition probability depends on the final observation values and present node state in the belief state space MDP system. Assuming that the sensor node s _n is under the state ${x}_t^n$ at the time t and under the state ${x}_{t+1}^n$ at the time t + 1, the state transition probability could be calculated from Eqs. (10–11).

$$ P\left({x}_t^n,{x}_{t+1}^n,{a}_{n,t}\right)=\sum \limits_{\delta_t\in {z}_0}\sum \limits_{d_t}P\left({d}_t\right){P}_U\left({l}_t,{l}_{t+1}\right)P\left({z}_t^n|{x}_t^n,{a}_{n,t}\right)\times \left(\Phi \left({e}_{t+1}^n\right)-\min \left(\Phi \left({e}_t^n\right)+{d}_t,r\right)\right) $$

(12)

The buffer size of the node S _n is d _t , and the transition probability of the link quality is P _U (l _t ,l_t + 1). The quantity of the data packet received at the time t and t + 1 under the corresponding energy state is $\Phi \left({e}_t^n\right)$ and $\Phi \left({e}_{t+1}^n\right)$.

It is assumed that the link quality award function of the node S _n is ${R}_1\left({l}_t^n,{a}_{n,t}\right)$ at the time t and the residual energy award function is${R}_e\left({e}_t^n,{a}_{n,t}\right)$. In order to improve the network channel utilization rate and extend network survival life, the transmission node with the high link quality shall be first selected for the transmission data packets. The nodes with higher residual energy shall be selected to forward data, in order to avoid the excessive consumption of some nodes. The belief state award function ρ(b,a) of the node calculated by Eq. (5) is:

$$ \rho \left(b,a\right)=\sum \limits_{s_i\in S}b(s)R\left({x}_t^n,{a}_{n,t}\right)=\sum \limits_{s_i\in S}b(s)\left(\lambda {R}_1\left({l}_t^m,{a}_{n,t}\right)+\left(1-\lambda \right){R}_e\left({e}_t^n,{a}_{n,t}\right)\right) $$

(13)

0 < λ < 1 is the weight of two awards.

One target for optimizing channel scheduling is to obtain the maximal award scheduling strategy. The channel scheduling target function calculated by Eqs. (5–6) and Eq. (12) is described as follows:

$$ {J}^{\pi }=E\left[\sum \limits_{k=0}^{\infty }{\omega}^k\sum \limits_{n=1}^M\rho \left(b,a\right),{a}_{n,t}|{x}_0^n\right] $$

(14)

In this equation, ω^k(0 < ω^k < 1) is the adjustment factor.

The maximal expected award is:

$$ {\psi}_n\left({x}^n\right)=\underset{k>0}{\max}\frac{E\left[\sum \limits_{k=0}^{r-1}{w}^k\rho \left(b,a\right)|{x}_0^n\right]}{E\left[\sum \limits_{k=0}^{r-1}{\omega}^k\right]} $$

(15)

3.3 Steps of channel scheduling algorithm

The steps of channel scheduling algorithm are as follows:

• (1) The set of transmission node s _i is Sset(), and the receiving node is s _j . Sset() comprises of the adjacent nodes of s _i .

• (2) Read s _i in Sset() in order to set the reading node as s _n . When the receiving node s _j is free, s _n sends control packet RT to s _j .

• (3) Calculate Ψ _n (xⁿ) from Eq. (14) and place the node (s _n ) of the maximal Ψ _n (xⁿ) into q _i .

• (4) s _j transmits the control packet CTS to s _n .

• (5) Remove s _n from Sset().

• (6) Traverse all transmission nodes in Sset().

• (7) Output the queue q _i , which is the scheduling sequence of the channels used by the transmission node.

Obtain the scheduling sequence q _i of the channel, according to the transmission node and transmit data packets in the allocated slots. After transmission, the state of the channel used by the node is predicted according to link quality, residual energy, and belief state space. The steps (1–7) are repeated to form the MAC protocol for s underwater acoustic sensor networks based on the belief state space (BSPMDP-MAC).

3.4 Simulation analysis

NS3 was simulated and analyzed. The performance of the MAC protocol was evaluated using Aqua-sim as the simulator, and the sensor node was made to work under half duplex mode. The sensor node can move at random in any direction. The movement speed v changes within the interval of 1–5 m/s. For parameter setting, refer to the reference [14]. The network throughput, energy consumption and success rate of transmission data were used as the reference parameters in the simulation analysis. The performance of the BSPMDP-MAC protocol was compared with the performance of the typical AUT-Lohi [15] and ST-Lohi [16], based on channel reservation.

• (1) The correlation among the network throughput, data packet transmission rate, and node quantity of the three different MAC protocols was compared. The network throughput of the three MAC protocols is proportional to the transmission rate of the data packet (refer to Fig. 2a). When the transmission rate of the data packet reaches 0.07/s, the network throughput will be prone to saturation. The average network throughput of the BSPMDP-MAC is 28.7% higher than that of the AUT-Lohi protocol, and is 34.8% higher than that of the ST-Lohi protocol. In the AUT-Lohi protocol, the competitive node only intercepts one T_max, and the tone frame may collide with the data packets in next cycle. In the ST-Lohi protocol, two-way epicophosis led to the collision between tone frames and data packets, and between data packets. The network throughput of the BSPMDP-MAC protocol was higher than that of the other two protocols under similar conditions.

Fig. 2

The relationship between network throughput and packet delivery rate and number of nodes. Legend: The correlation among the network throughput, a data packet transmission rate, and b node quantity of the three different MAC protocols was compared

The network throughput of the three MAC protocols was proportional to the quantity of sensor nodes (refer to Fig. 2b). When the node quantity reached 80, the network throughput would gradually approximate to its saturation. The average network throughput of the BSPMDP-MAC protocol was 18.4% higher than that of the AUT-Lohi protocol and was 20.8% higher than that of the ST-Lohi protocol. The link quality was considered in the channel scheduling of the BSPMDP-MAC protocol. Nodes with better link quality could transmit data reliably with each other. The network throughput of the BSPMDP-MAC protocol was higher than that of the other two protocols under the same condition.

• (2) For the relationship among the data packet transmission success rate of sensor nodes, the data packet transmission rate and node transmission radius of three different MAC protocols were compared. The data packet transmission success rate of sensor nodes of three different MAC protocols was disproportional to the transmission rate (refer to Fig. 3a). When this was proportional to the data packet transmission rate, the data packet transmission success rate of the BSPMDP-MAC protocol became higher than that of other two protocols. The competitive nodes obtained channel access privilege via the tone frame competition in the ST-Lohi protocol. When collision occurred, the random collision backoff was used to affect the data packet transmission success rate. One low-probability tone frame collision as used in the AUT-Lohi protocol. When collision occurred, the confliction backoff algorithm was used to reduce competition time and decrease the loss rate of packets in AUT-Lohi. When the data packet transmission rate was lower, this method was effective. When the data packet transmission rate increased, the transmission success rate of data packets quickly decreased.

Fig. 3

The relationship between packet transmission rate and packet transmission rate and node. Legend: For the relationship among the data packet transmission success rate of sensor nodes, the a data packet transmission rate and b node transmission radius of three different MAC protocols were compared

The data packet transmission success rate of sensor nodes is disproportional to the transmission radius for the three MAC protocols (refer to Fig. 3b). When the transmission radius of the sensor node increased, the adjacent nodes of each sensor node also increased. Therefore, the nodes fiercely competed for channel use privilege, the channel collision probability increased, and the data packet transmission success rate was significantly reduced. When the node transmission radius was proportional, the average data packet transmission success rate of the BSPMDP-MAC protocol was 26.8% higher than that of the AUT-Lohi protocol and 89.5% higher than that of the ST-Lohi protocol. When the nodes inside the communication radius increased, the AUT-Lohi and ST-Lohi protocols called the backoff mechanism. The BSPMDP-MAC protocol transmits the data packets by time slots according to the scheduling sequence without the need to frequently call the backoff mechanism. Thus, the data packet transmission success rate of the BSPMDP-MAC protocol was higher than it of the other two protocols.

• (3) The correlation between the energy consumption of the sensor node and node quality for three different MAC protocols:

The energy consumption of the sensor nodes was disproportional to the node quantity for the three MAC protocols (refer to Fig. 4). When the sensor nodes increase, the communication distance between sensor nodes would become shorter. The BSPMDP-MAC protocol first regards transmission nodes with better link quality as nodes that transmit data packets. Nodes with higher residual energy are used as data forwarding nodes to avoid excessive energy consumption in some nodes. Although the AUT-Lohi and ST-Lohi protocols use a corresponding wakeup mechanism to ensure that the sensor nodes maintain its low energy consumption under the monitoring state, the excessive energy consumption problem of individual nodes was not considered. When the sensor nodes are proportional, the energy consumption of the BSPMDP-MAC protocol was 45.1% less than that of the AUT-Lohi protocol and 19.3% less than that of the ST-Lohi protocol. The whole network survival cycle is thereby extended.

Fig. 4

The relationship between energy consumption and node quantity. Legend: The energy consumption of the sensor nodes was disproportional to the node quantity for the three MAC protocols

4 Conclusions

The BSPMDP-MAC protocol can obtain the channel decision strategy of nodes according to the joint probability distribution of the historical observations and actions of the sensor’s receiving node channel. When the transmission nodes transmit data packets to receiving nodes in the allocated time slots, it reduces the collision rate of data packets according to the scheduling sequence of the decision strategy. The receiving nodes predict the channel occupancy according to the present belied state and actions. In order to perceive the belief state and access actions in the next cycle reasonably, time-space uncertainties and fairness problems in the channel use of nodes are solved and the utilization rate of the whole channel is improved. This protocol regards transmission nodes with better link quality as nodes for data packet transmission firstly. Then, nodes with higher residual energy were used as data forwarding nodes. In order to effectively reduce the confliction rate of data packets, data packet transmission success rate and network throughput are improved. Moreover, the network energy consumption is reduced and the lifecycle of the whole network is extended.