The energy harvesting multi-way relay channel with intermittent data: the impact of buffer sizes

Abstract

In this paper, a multi-way relay channel is considered with energy harvesting nodes. The harvested energy at each node can be stored in a battery of finite capacity. Additionally, each node employs a data buffer of limited size to store data prior to transmission. Data packets to be exchanged between the nodes arrive at the source nodes in an intermittent fashion. In this setup, the offline sum throughput maximization problem, which turns out to be a convex optimization problem, is solved. The corresponding online problem is formulated, and a backward induction-based optimal solution is proposed. In addition, a near-optimum low complexity solution is presented. It is noted that the solutions for the offline and online throughput maximization problems for the multi-way relay channel also provide solutions for its special cases, e.g., the multi-pair two-way relay channel, the two-way channel, and others. Numerical results are presented to demonstrate the resulting optimal transmission policies for various channel setups, comparing the sum throughput to upper and lower bounds, and demonstrating the impact of buffer sizes at the relay.

1 Introduction

Energy harvesting wireless networks rely on efficient allocation of transmit power for perpetual operation. Such networks consist of nodes that do not have a constant source of energy for transmission but harvest energy in an intermittent fashion from external sources, such as solar cells, piezoelectric devices, and others [1,2]. In this paper, we consider such a network with many anycasting nodes which can communicate with the help of a shared relay which is also energy harvesting.

In recent years, various channel models have been studied in an energy harvesting setup. Reference [3] has proposed the framework for optimization of the transmission schedule of an energy harvesting node by solving the transmission completion time minimization problem in a single-user channel. The energy harvesting source node is assumed to receive packets of data in a stochastic fashion. The same setup is studied in [4] without data arrivals, and dynamic programming is used to find throughput maximizing policies. Reference [5] has solved the throughput maximization problem in a single user channel with a source node that has a finite-capacity battery. The same setup is solved for fading channels in [6] providing a directional water-filling interpretation of the optimal power schedule. References [7-9] have quantified the impact of imperfect energy storage in an energy harvesting setup. References [10] and [11] have studied a single-user communication system with an energy harvesting receiver.

Multi-user channel models have also been studied with energy harvesting including the multiple access, broadcast, relay, and interference channels [12-21]. The two-hop channel is considered in [22] where the solution of the throughput maximization problem is found for two energy harvests at the source and the relay. This result is extended to multiple energy arrivals at the relay in [23], and the solution for the general two-hop channel is found in [24]. Reference [25] also considers the two-hop channel and analyzes how the buffer size at the relay affects the achieved throughput. The two-way relay channel is studied in reference [26] where the optimal policy is found to maximize throughput with a decode-and-forward relay and also in [27] where the generalized directional water-filling solution is found, and it is shown that hybrid relaying strategies result in higher throughput. Reference [28] studies the energy harvesting two-way relay channel in a more general setup - the relay node is given finite-capacity buffers to store data, and the source nodes are allowed to receive data packets in a stochastic fashion. Therefore, the relay node does not have to forward incoming data immediately after it is received, and the data in the relay buffers must also be optimally allocated, which introduces a new optimization dimension.

The main focus of this paper is on the multi-way relay channel which is introduced in [29] where achievable rates are found with various relaying schemes including amplify-and-forward, decode-and-forward, compress-and-forward, and lattice forwarding. In [30], the functional-decode-and-forward strategy is shown to outperform decode-and-forward at a high signal-to-noise ratio. In [31], functional-decode-and-forward is shown to achieve the cut-set upper bound on the capacity region of the multi-way relay channel where the channel inputs and outputs lie in a finite field. Reference [32] studies the degrees of freedom of the multi-way relay channel with three users in each cluster and then states conditions under which the results can be generalized. An interesting special case of the multi-way relay channel is the multi-pair relay channel which is studied in [33] where the capacity region is first investigated in a deterministic setting, and then, the resulting insights are used for the Gaussian setting. The multi-pair relay channel is also studied in [34,35] with single antenna terminals and in [36,37] with multiple antennas at the relay.

Our focus is on identifying the throughput optimal operation for the energy harvesting multi-way relay channel. Reference [29] illustrates that decode-and-forward relaying outperforms other relaying strategies at low transmit power and with a large number of users. Since the nodes in an energy harvesting network are expected to be energy deficient, we find decode-and-forward relaying suitable for the energy harvesting multi-way relay channel and use this strategy throughout the paper.

We solve the offline and online throughput maximization problems for an energy harvesting multi-way relay channel with a half-duplex relay and L clusters, each composed of K source nodes. Our model also has stochastic data arrivals at the source nodes. We impose finite-battery and finite-buffer constraints at all nodes in order to gain more realistic insights and to observe how these size constraints affect the optimal policy and the achieved throughput. To this end, our system model captures all of the physical resource constraints of energy efficient and energy harvesting communications and generalizes various relay-aided communication models. We show that the offline throughput maximization problem can be expressed as a convex optimization problem. First, with the assumption that all energy and data arrivals are known beforehand, i.e., the offline setup, we identify the optimal transmission policies. Next, we remove this assumption and find online policies with causal energy and data arrivals using dynamic programming.

The multi-way relay channel model reduces to many simpler channel models of interest as its special cases. Therefore, we present numerical examples that demonstrate how optimality is achieved both for some of these special cases and the general model of the multi-way relay channel. Our results show that the relay node spends most of its resources for clusters of users that have more data to transmit and more energy to spend which is intuitively pleasing for the throughput maximizing operation of the network.

2 System model

We consider a Gaussian multi-way relay channel with a half-duplex decode-and-forward relay and L≥1 clusters of users or source nodes with K≥2 users in each cluster. Every node in the model is assumed to have one antenna. We denote the relay node by T ₀ and T _j,i denotes the ith node in the jth cluster for $i \in I_{K} \triangleq \{ 1,2, \dots, K \}$ and $j \in I_{L} \triangleq \{ 1,2, \dots, L \}$, following the notation of [29]. Every source node wishes to decode the messages of all the other source nodes belonging to its cluster. The relay node enables communication between the source nodes as it is assumed that the source nodes cannot hear each other directly, i.e., they are separated.

We have two-phase communication. Phase I refers to the uplink phase where the relay listens to the source nodes and decodes all the messages. Phase II refers to the downlink phase where the relay transmits functions of the decoded messages to the L K source nodes. In phase I, while the relay listens, the power gain from T _j,i to T ₀ is denoted by $\tilde {h}_{j,i}^{I}$. Similarly, in phase II, while the relay transmits, the power gain from T ₀ to T _j,i is denoted by $\tilde {h}_{j,i}^{II}$. The additive white Gaussian noise (AWGN) at node T _j,i is assumed to have zero mean and variance $\sigma _{j,i}^{2}$. Similarly, the AWGN at the relay node T ₀ has zero mean and variance ${\sigma _{0}^{2}}$. We normalize the power gains with respect to the noise variance at the corresponding receiver so that effectively the additive noise at each node has zero mean and unit variance. This normalization is done as follows. For phase I, we calculate the effective power gain from T _j,i to T ₀ as $h_{j,i}^{I} = \tilde {h}_{j,i}^{I}/{\sigma _{0}^{2}}$, and for phase II, we calculate the effective power gain from T ₀ to T _j,i as $h_{j,i}^{II} = \tilde {h}_{j,i}^{II}/\sigma _{j,i}^{2}$, for i∈I _K and j∈I _L . The system model is depicted in Figure 1.

Figure 1

The energy harvesting multi-way relay channel with L clusters and K users per cluster.

All nodes in the model are energy harvesting; they harvest random amounts of energy at random times. The source nodes T _j,i also receive data packets at random times. We refer to the time duration between any two consecutive energy or data arrivals as an epoch. N denotes the total number of epochs by the deadline, D. s _n denotes the beginning of the nth epoch, and we set s _N+1=D. The length of the nth epoch is denoted by l _n =s _n+1−s _n for all $n \in I_{N} \triangleq \{1,2,\dots,N\}$. Node T _j,i harvests $E_{j,i}^{(n)}$ units of energy and $B_{j,i}^{(n)}$ bits of data at the beginning of the nth epoch, s _n , for i∈I _K and j∈I _L . The relay node T ₀ harvests $E_{0}^{(n)}$ units of energy at s _n , but it does not have any messages of its own that it wishes to transmit. If node T _j,i (resp. T ₀) does not harvest any energy at s _n for some j,i,n, then we set $E_{j,i}^{(n)} = 0$ (resp. $E_{0}^{(n)} = 0$), and similarly, we set $B_{j,i}^{(n)} = 0$ if node T _j,i does not receive any data at s _n . Thus, the epoch construction procedure is done over the union of all arrivals, energy, or data, at all nodes. An example is shown in Figure 2. (Note that, in the figure zero energy or data arrivals are not shown and some source nodes are omitted for clarity). Each node is equipped with a finite-capacity battery. $E_{j,i}^{(\rm max)}$ and $E_{0}^{(\rm max)}$ denote the maximum amount of energy that the battery at node T _j,i and T ₀ can store, respectively.

Figure 2

An example of epoch construction.

The source nodes receive packets of data to transmit during the communication session until D. Each data arrival is assumed to take an infinitesimal amount of time; therefore, the data arrivals at the source nodes do not interfere with the multi-way communication through the relay. Each source node has a finite-capacity buffer that it can use to store data packets until the packets can be transmitted. The maximum amount of data that the buffer at source node T _j,i can store is denoted by $B_{j,i}^{(\rm max)}$, i∈I _K and j∈I _L . Similarly, the relay node is given L K finite-capacity buffers to store incoming data from the source nodes. Since the relaying operation is decode-and-forward, the relay has to decode all L K messages coming from L K source nodes. We assume that it uses buffer (j,i), which has finite capacity $\bar {B}_{j,i}^{(\rm max)}$, to store the messages of node T _j,i ,i∈I _K and j∈I _L . We will first assume that all arrival instants s _n , energy amounts $E_{j,i}^{(n)}$ and $E_{0}^{(n)}$, and data amounts $B_{j,i}^{(n)}$ are known non-causally as in references [3,5-8,12-20,22-28]. We will then consider the online version where all these quantities are known to the transmitter in a causal manner.

Remark 1.

While for clarity of exposition, we focus on a Gaussian channel, we observe that the formulation and the solution readily extend to fading channels, see also [6]. Thus, we implement the offline and online solutions with fading as well and present our findings in the Numerical results section.

We define $\Delta _{0}^{(n)} \in {}[\!0,1]$ which denotes the fraction of the nth epoch during which the sources transmit and the relay listens (phase I). Naturally, $1-\Delta _{0}^{(n)}$ denotes the fraction of the nth epoch during which the relay forwards the messages it has received (phase II). Then, the durations of phase I and phase II in the nth epoch are given by $\Delta _{0}^{(n)} l_{n}$ and $\left (1-\Delta _{0}^{(n)}\right) l_{n}$, respectively.

The first phase of the multi-way communication scheme amounts to a multiple access channel with L K transmitters, T _j,i , and a receiver, T ₀. Therefore, the rates achieved in phase I have to satisfy the L K-user multiple access channel capacity region given in [29] and [38] by:

$$ \sum_{j \in S_{L}} \sum_{i \in S_{K}} R_{j,i} \leq C \left(\sum_{j \in S_{L}} \sum_{i \in S_{K}} h_{j,i}^{I} P_{j,i} \right), $$

((1))

for all S _L ⊂I _L and S _K ⊂I _K where $C(x) \triangleq \frac {1}{2} \log (1+x)$, R _j,i denotes the rate achieved from T _j,i to T ₀, and P _j,i denotes the transmit power at node T _j,i , i∈I _K and j∈I _L .

The second phase is a broadcast channel with side information [29]. Phase II in any epoch is divided into L time slots proportional to τ _j ≥0 where $\sum _{j=1}^{L} \tau _{j} = 1$. The relay power P ₀ is also divided into L fractions, P _0,j , j∈I _L . In the jth time slot, the relay node broadcasts the messages of all users in cluster j to all users in the cluster with transmit power P _0,j . The details of the coding scheme are given in [29]. The messages can be decoded by all the users if the rates R _j,i satisfy:

$$ \sum\limits_{\substack{i \in I_{K}\\i \neq l}} R_{j,i} \leq \tau_{j} C\left(h_{j,l}^{II} P_{0,j} \right), $$

((2))

for all j∈I _L and l∈I _K . We denote the fraction of the time slot in phase II of the nth epoch in which the relay node broadcasts to cluster j by $\Delta _{j}^{(n)}, j \in I_{L}$. Thus, we have $\Delta _{j}^{(n)} = \tau _{j} \left (1-\Delta _{0}^{(n)}\right)$ and $\sum _{j=0}^{L} \Delta _{j}^{(n)} = 1$. The length of the time slot reserved for broadcast to cluster j in epoch n is then given by $\Delta _{j}^{(n)} l_{n} = \tau _{j} \left (1-\Delta _{0}^{(n)}\right) l_{n}$.

Since the achievable rates in Equations 1 and 2 are concave in transmit powers, by a similar argument to that in ([3], Lemma 2) we can conclude that the transmit power at each node should remain constant throughout an epoch while the node is transmitting. We denote the average transmit power at source node T _j,i in the nth epoch by $p_{j,i}^{(n)}, j \in I_{L}, i \in I_{K}$, and n∈I _N . Similarly, the transmit power at the relay node T ₀ for cluster j in the nth epoch is denoted by $p_{0,j}^{(n)}, j \in I_{L}$ and n∈I _N . The transmit powers are averaged over the duration of the corresponding epoch, i.e., node T _j,i transmits with power $p_{j,i}^{(n)}/\Delta _{0}^{(n)}$ for a duration of $\Delta _{0}^{(n)} l_{n}$ seconds and the relay node T ₀ transmits with power $p_{0,j}^{(n)}/\Delta _{j}^{(n)}$ for a duration of $\Delta _{j}^{(n)} l_{n}$ seconds while it broadcasts to cluster j. In addition, we denote by $r_{j,i}^{I,n}$ the average rate achieved from T _j,i to T ₀ in phase I of the nth epoch for j∈I _L and i∈I _K . Similarly, $r_{j,i}^{II,n}$ denotes the rate of node T _j,i ’s message that the relay forwards in phase II of the nth epoch for j∈I _L and i∈I _K . Hence, the amount of data transmitted by node T _j,i in phase I of the nth epoch is $l_{n} r_{j,i}^{I,n}$, and similarly, the amount of data forwarded by node T ₀ in phase II of the nth epoch is $l_{n} \sum _{j=1}^{L} \sum _{i=1}^{K} r_{j,i}^{II,n}$.

3 Throughput maximization for the multi-way relay channel

In this section, we consider offline throughput maximization for the multi-way relay channel. The end-to-end throughput maximization is equivalent to maximizing the total amount of data that the relay forwards subject to reliable decoding and resource constraints, stated as follows.

$$\begin{array}{*{20}l} & \max_{\substack{p_{j,i}^{(n)}, p_{0,j}^{(n)}, r_{j,i}^{I,n},\\ r_{j,i}^{II,n}, \Delta_{0}^{(n)}, \Delta_{j}^{(n)}}} ~ \sum\limits_{n=1}^{N} \sum\limits_{j=1}^{L} \sum\limits_{i=1}^{K} l_{n} r_{j,i}^{II,n} \end{array} $$

((3a))

$$\begin{array}{*{20}l} &\text{s.t.}~ \sum\limits_{n=1}^{{\bar n}} l_{n} p_{j,i}^{(n)} \leq \sum\limits_{n=1}^{\bar n} E_{j,i}^{(n)}, ~ \forall {\bar n} \in I_{N},~ j \in I_{L},~ i \in I_{K}, \end{array} $$

((3b))

$$\begin{array}{*{20}l} &\sum\limits_{n=1}^{\bar{n}} \sum\limits_{j=1}^{L} l_{n} p_{0,j}^{(n)} \leq \sum\limits_{n=1}^{\bar n} E_{0}^{(n)}, ~ \forall {\bar n} \in I_{N}, \end{array} $$

((3c))

$$\begin{array}{*{20}l} &\sum\limits_{n=1}^{\bar n} E_{j,i}^{(n)} - \sum_{n=1}^{{\bar n}-1} l_{n} p_{j,i}^{(n)} \leq E_{j,i}^{(\rm max)}, ~ \forall {\bar n} \in I_{N},~ j \in I_{L},~ i \in I_{K}, \end{array} $$

((3d))

$$\begin{array}{*{20}l} &\sum\limits_{n=1}^{\bar n} E_{0}^{(n)}-\sum\limits_{n=1}^{\bar{n}-1} \sum\limits_{j=1}^{L} l_{n} p_{0,j}^{(n)} \leq E_{0}^{(\rm max)}, ~ \forall {\bar n} \in I_{N}, \end{array} $$

((3e))

$$\begin{array}{*{20}l} &\sum\limits_{n=1}^{{\bar n}} l_{n} r_{j,i}^{I,n} \leq \sum\limits_{n=1}^{\bar n} B_{j,i}^{(n)}, ~ \forall {\bar n} \in I_{N},~ j \in I_{L},~ i \in I_{K}, \end{array} $$

((3f))

$$\begin{array}{*{20}l} &\sum_{n=1}^{\bar n} B_{j,i}^{(n)} - \sum\limits_{n=1}^{{\bar n}-1} l_{n} r_{j,i}^{I,n} \leq B_{j,i}^{(\rm max)}, ~ \forall {\bar n} \in I_{N},~ j \in I_{L},~ i \in I_{K}, \end{array} $$

((3g))

$$\begin{array}{*{20}l} &0 \leq \sum\limits_{n=1}^{\bar n} l_{n} (r_{j,i}^{I,n} - r_{j,i}^{II,n}) \leq {\bar B}_{j,i}^{(\rm max)}, ~ \forall {\bar n} \in I_{N},~ j \in I_{L},~ i \in I_{K}, \end{array} $$

((3h))

$$\begin{array}{*{20}l} &\sum_{j \in S_{L}} \sum_{i \in S_{K}} r_{j,i}^{I,n} \leq \Delta_{0}^{(n)} C \left(\frac{\sum_{j \in S_{L}} \sum_{i \in S_{K}} h_{j,i}^{I} p_{j,i}^{(n)}}{\Delta_{0}^{(n)}} \right), ~ \forall n \in I_{N}, \\ & S_{L} \subset I_{L},~ S_{K} \subset I_{K}, \end{array} $$

((3i))

$$\begin{array}{*{20}l} &\sum_{\substack{i \in I_{K}\\i \neq l}} r_{j,i}^{II,n} \leq \Delta_{j}^{(n)} C\left(\frac{h_{j,l}^{II} p_{0,j}^{(n)}}{\Delta_{j}^{(n)}} \right), ~ \forall n \in I_{N},~ j \in I_{L},~ l \in I_{K}, \end{array} $$

((3j))

$$\begin{array}{*{20}l} &0\leq \Delta_{0}^{(n)} \leq 1,~ 0\leq \Delta_{j}^{(n)} \leq 1,~ \forall j \in I_{L}, ~ \sum_{j=0}^{L} \Delta_{j}^{(n)} = 1, \end{array} $$

((3k))

$$\begin{array}{*{20}l} &p_{j,i}^{(n)}, p_{0,j}^{(n)}, r_{j,i}^{I,n}, r_{j,i}^{II,n}, \geq 0, ~ \forall n \in I_{N},~ j \in I_{L},~ i \in I_{K}. \end{array} $$

((3l))

The constraints in Equations 3b and 3c are the so-called energy causality constraints [3,5,6]. These ensure that the nodes do not spend any energy that has not yet been harvested. The battery constraints are given in Equations 3d and 3e which limit how much energy can be stored in the batteries during the communication session. The constraints in Equations 3f and 3g are the data causality and buffer constraints for the source nodes, respectively. These constraints ensure that the source nodes only transmit data that has arrived and do not store more data than the size of their buffers. These constraints are given in Equation 3h for the relay. Lastly, the average rates for each epoch are constrained by the rate regions in Equations 1 and 2 for reliable decoding, which translate to the problem as Equation 3i for phase I and Equation 3j for phase II.

Due to data storage limitations at all source nodes, some of the received data may need to be dropped. In order to address this, we introduce a slack variable $d_{j,i}^{(n)} \geq 0$ which denotes the amount of data dropped at node T _j,i in the nth epoch for all j∈I _L and i∈I _K . As a result of this modification, Equation 3f has to be restated as:

$$ \sum\limits_{n=1}^{\bar{n}} \left(l_{n} r_{j,i}^{I,n} + d_{j,i}^{(n)}\right) \leq \sum\limits_{n=1}^{\bar{n}} B_{j,i}^{(n)}, $$

((4))

and Equation 3g as:

$$ \sum_{n=1}^{\bar n} B_{j,i}^{(n)} - \sum_{n=1}^{{\bar n}-1} \left(l_{n} r_{j,i}^{I,n} + d_{j,i}^{(n)}\right) \leq B_{j,i}^{(\rm max)}. $$

((5))

Note that there is no need to introduce a slack variable for the relay node. This follows from the fact that the source nodes can adjust their rates so as not to transmit more data than the relay can store. Therefore, the slack variable for the source nodes $d_{j,i}^{(n)}$ effectively covers the impact of the finite data buffers at the relay as well.

The objective function in Equation 3a and all constraints except Equations 3i and 3j are linear in the optimization parameters. The constraints in Equations 3i and 3j do not violate convexity of the feasible region. This follows from the fact that the right hand sides in Equations 3i and 3j are of the form y C(x/y) for some x and y, which is jointly concave in x and y since it is the perspective of a concave function, C(x) ([39] §3.2.6). Then, Equation 3 is a convex optimization problem and can be solved using available efficient algorithms for convex programming [39,40].

It is worth noting that it is necessary to leave the rates $r_{j,i}^{I,n}$ and $r_{j,i}^{II,n}$ as optimization parameters in Equation 3 instead of expressing them in terms of $p_{j,i}^{(n)}, p_{0,j}^{(n)}, \Delta _{0}^{(n)}$, and $\Delta _{j}^{(n)}$, contrary to previous work on energy harvesting, e.g., [3,5,12,26]. Due to the multi-way nature of the channel model, the nodes may not always be able to fully exploit their transmit powers, i.e., they may have to operate at a lower rate than their transmit power allows. As an example, consider the case where the relay has data only from node T _1,1 in the last epoch. Then, $r_{j,i}^{II,n}$ has to be zero for j∈I _L ∖{1} and i∈I _K since the relay does not have any data from clusters 2,3,…,L to forward. However, it still has to spend some energy to forward node T _1,1’s message to cluster 1. Thus, it uses its energy to forward only one message to cluster 1, although it would have been possible to forward messages from T _1,i , i∈I _K ∖{1} to cluster 1 had the relay in any data from these nodes. Therefore, the relay is unable to fully exploit its power in this case, and in order to allow this, we must include $r_{j,i}^{I,n}$ and $r_{j,i}^{II,n}$ as optimization parameters.

We use the method of steepest descent to solve the offline throughput maximization problem in Equation 3. Steepest descent is guaranteed to converge to a local optimum of Equation 3. Although Equation 3 may have multiple local optima since its objective is concave (as opposed to strictly concave), each of these solutions is equally good and produces the globally optimum sum throughput value. The algorithm is provided in Algorithm 1 where $\mathbf {u} = \left (p_{j,i}^{(n)}, p_{0,j}^{(n)}, r_{j,i}^{I,n}, r_{j,i}^{II,n}, d_{j,i}^{(n)}, \Delta _{0}^{(n)}, \Delta _{j}^{(n)} \right)_{\forall i,j,n}$ is a solution, u ^[k] is the solution at the kth iteration, $g(\mathbf {x}) = \sum _{n=1}^{N} \sum _{j=1}^{L} \sum _{i=1}^{K} l_{n} r_{j,i}^{II,n}$ is the objective to be maximized, and Π(·) is a projection operator which projects its argument onto the feasible space in Equation 3.

In this section, we have solved the throughput maximization problem for the multi-way relay channel with the assumption that all energy harvests and data arrivals are known to all nodes non-causally. The optimal policies found with this offline approach can be used in setups where the arrival processes for the energy and data are predictable. With rapidly varying, hard to predict energy and data arrivals, these offline policies can still be used as a benchmark since the sum throughput achieved by an offline policy is an upper bound on the sum throughput achieved by the online policy for the same setup. However, from a network operation point of view, we need to find transmission policies that rely on causal energy and data arrival information only. We shall next consider this setup, i.e., the online policies.

4 Online policies for the multi-way relay channel

In this section, we consider online policies which assume causal knowledge of energy harvests and data arrivals at the terminals. For simplicity, we assume that all epochs are of duration l, i.e., l _n =l for n∈I _N . We formulate the online throughput maximization problem as a dynamic program where the state variables are the energy and data amounts in all batteries and buffers, the control variables are the power, rate, and phase fraction values chosen by the nodes, and the randomness is due to the absence of the non-causal knowledge of all energy and data arrivals. Formally, we denote the state at the beginning of the nth epoch by x _n and express it as:

$$ \begin{aligned} x_{n} &= \left(\left(e_{j,i}^{(n)}\right)_{\forall i,j}, e_{0}^{(n)}, \left(b_{j,i}^{(n)}\right)_{\forall i,j}, \left(\bar{b}_{j,i}^{(n)}\right)_{\forall i,j} \right) \\ &\in \mathcal{E}_{1,1} \times \cdots \times \mathcal{E}_{L,K} \times \mathcal{E}_{0} \times \mathcal{B}_{1,1} \times \cdots \times \mathcal{B}_{L,K}\\ &\quad \times \bar{\mathcal{B}}_{1,1} \times \cdots \times \bar{\mathcal{B}}_{L,K} = \mathcal{X} \end{aligned} $$

((6))

where $E_{j,i}^{(n)}$ is the amount of energy in T _j,i ’s battery, $E_{0}^{(n)}$ is the amount of energy in T ₀’s battery, $B_{j,i}^{(n)}$ is the amount of data in T _j,i ’s buffer, and $\bar {b}_{j,i}^{(n)}$ is the amount of data in buffer (j,i) at T ₀ at the beginning of the nth epoch. The state space depends on the battery and buffer sizes and can be expressed using:

$$\begin{array}{*{20}l} \mathcal{E}_{j,i} &= \left[0, E_{j,i}^{(\rm max)}\right], \end{array} $$

((7))

$$\begin{array}{*{20}l} \mathcal{E}_{0} &= \left[0, E_{0}^{(\rm max)}\right], \end{array} $$

((8))

$$\begin{array}{*{20}l} \mathcal{B}_{j,i} &= \left[0, B_{j,i}^{(\rm max)}\right], \end{array} $$

((9))

$$\begin{array}{*{20}l} \bar{\mathcal{B}}_{j,i} &= \left[0, \bar{B}_{j,i}^{(\rm max)}\right], \end{array} $$

((10))

for all j∈I _L and i∈I _K . Although for now, we consider to be continuous, in practice, will have to be quantized. Similarly to the state, we denote the random input at the beginning of the nth epoch by w _n and express it as:

$$ w_{n} = \left(\left(E_{j,i}^{(n)}\right)_{\forall i,j}, E_{0}^{(n)}, \left(B_{j,i}^{(n)}\right)_{\forall i,j} \right) $$

((11))

which depends on energy and data arrival processes.

At the beginning of the nth epoch, the nodes have to decide on the transmission variables $p_{j,i}^{(n)}$, $p_{0,j}^{(n)}$, $r_{j,i}^{I,n}$, $r_{j,i}^{II,n}$, $\Delta _{0}^{(n)}$, and $\Delta _{j}^{(n)}$ depending on the current state x _n and the current and past arrivals $(w_{1},\dots,w_{n}) \triangleq w^{n}$ where we use the superscript notation to denote the causal knowledge of energy and data arrivals. Note that the past states need not be included in the decision of the transmission variables as they can be computed using x _n and w ⁿ. We define an action for each transmission variable in each epoch which outputs the transmission variable as a function of the state and the causal knowledge of arrivals. We denote these actions by $\phi _{p_{j,i}^{(n)}}$, $\phi _{p_{0,j}^{(n)}}$, $\phi _{r_{j,i}^{I,n}}$, $\phi _{r_{j,i}^{II,n}}$, $\phi _{\Delta _{0}^{(n)}}$, and $\phi _{\Delta _{j}^{(n)}}$ each of which is used to find the transmission variable given in the subscript. For convenience, we also define a collective action for each epoch as:

$$\begin{array}{*{20}l} \phi_{n} \colon \left(x_{n},w^{n}\right) \mapsto \left(p_{j,i}^{(n)}, p_{0,j}^{(n)}, r_{j,i}^{I,n}, r_{j,i}^{II,n}, \Delta_{0}^{(n)}, \Delta_{j}^{(n)}\right). \end{array} $$

((12))

Using these actions and the current state, the next state can be calculated as:

$$\begin{array}{*{20}l} x_{n+1} &= \left(\left(e_{j,i}^{(n+1)}\right)_{\forall i,j}, e_{0}^{(n+1)}, \left(b_{j,i}^{(n+1)}\right)_{\forall i,j}, \left(\bar{b}_{j,i}^{(n+1)}\right)_{\forall i,j} \right) \end{array} $$

((13))

where:

$$\begin{array}{*{20}l} & e_{j,i}^{(n+1)} = \min \left\{ E_{j,i}^{(\rm max)}, e_{j,i}^{(n)}-l \phi_{p_{j,i}^{(n)}}\left(x_{n},w^{n}\right)+E_{j,i}^{(n+1)} \right\} \end{array} $$

((14))

$$\begin{array}{*{20}l} &e_{0}^{(n+1)} = \min \left\{ E_{0}^{(\rm max)}, e_{0}^{(n)}- \sum_{j=1}^{L} l \phi_{p_{0,j}^{(n)}}\left(x_{n},w^{n}\right)+E_{0}^{(n+1)} \right\} \end{array} $$

((15))

$$\begin{array}{*{20}l} &b_{j,i}^{(n+1)} = \min \left\{ B_{j,i}^{(\rm max)}, b_{j,i}^{(n)}-l \phi_{r_{j,i}^{I,n}}\left(x_{n},w^{n}\right)+B_{j,i}^{(n+1)} \right\} \end{array} $$

((16))

$$\begin{array}{*{20}l} &\bar{b}_{j,i}^{(n+1)} = \min \!\left\{ \bar{B}_{j,i}^{(\rm max)}, \bar{b}_{j,i}^{(n)}\,+\,l \phi_{r_{j,i}^{I,n}}\left(x_{n},w^{n}\right) \,-\,l \phi_{r_{j,i}^{II,n}}\left(x_{n},w^{n}\right) \right\} \end{array} $$

((17))

for all n=2,3,…,N, j∈I _L , i∈I _K . Note that, the slack variable denoting the dropped packages, $d_{j,i}^{(n)}$, does not need to be included in the online solution since the dropped packages are already modeled by Equation 16.

The initial state x ₁ only depends on the arrivals at the beginning of transmission and can be found using:

$$\begin{array}{*{20}l} e_{j,i}^{(1)} &= \min \left\{ E_{j,i}^{(\rm max)}, E_{j,i}^{(1)} \right\} \end{array} $$

((18))

$$\begin{array}{*{20}l} e_{0}^{(1)} &= \min \left\{ E_{0}^{(\rm max)}, E_{0}^{(1)} \right\} \end{array} $$

((19))

$$\begin{array}{*{20}l} b_{j,i}^{(1)} &= \min \left\{ B_{j,i}^{(\rm max)}, B_{j,i}^{(1)} \right\} \end{array} $$

((20))

$$\begin{array}{*{20}l} \bar{b}_{j,i}^{(1)} &= 0, \end{array} $$

((21))

for all j∈I _L and i∈I _K .

Lastly, we denote the reward in the nth epoch by R _n and define it to be the amount of data that departs the relay in the nth epoch as:

$$\begin{array}{*{20}l} R_{n} \left(\phi_{n}\left(x_{n},w^{n}\right)\right) = \sum_{j=1}^{L} \sum_{i=1}^{K} l \phi_{r_{j,i}^{II,n}}\left(x_{n},w^{n}\right). \end{array} $$

((22))

The Bellman equations for this system can be given as [41]:

$$\begin{array}{*{20}l} J_{N}\left(x_{N},w^{N}\right) &= \max_{\phi_{N}} R_{N}\left(\phi_{N}\left(x_{N},w^{N}\right)\right), \end{array} $$

((23))

$$\begin{array}{*{20}l} J_{n}\left(x_{n},w^{n}\right) &= \max_{\phi_{n}} \left(R_{n}\left(\phi_{n}\left(x_{n},w^{n}\right)\right) \right.\\ &\quad+\left. \mathbb{E}_{w_{n+1} \mid w^{n}} \left[ J_{n+1}\left(x_{n+1},w^{n+1}\right) \right] \right), \end{array} $$

((24))

for all n=1,2,…,N−1 where $\mathbb {E}_{w_{n+1} \mid w^{n}} [\cdot ]$ denotes expectation with respect to the distribution of w _n+1 given w ⁿ. The optimal online policy $(\phi ^{\ast }_{n})_{n=1,2,\dots,N}$ consists of maximizers of the right hand sides of Equations 23 and 24 and can be found with backward induction, starting from Equation 23 and going backwards using Equation 24. The optimal online policy has to be calculated for all realizations of w ^N, so that when the transmission starts, the nodes will now what the best action is for any realization. The resulting dynamic program thus suffers from the curse of dimensionality. For our simulations, we use a near-optimal online policy that can be computed in polynomial time. In this approach, the actions depend only on the current state and not on the particular epoch n. In order to calculate the actions, we can choose N to be large enough that the dynamic program turns into an infinite horizon problem and iterate:

$$ J(x) = \max_{\phi} \left(R(\phi(x)) + \mathbb{E}_{x' \mid x} \left[ J(x') \right] \right), $$

((25))

until we have a stationary action ϕ(x) for all states x. Note that we dropped all epoch dependencies in Equation 25. This approach yields a suboptimal solution, but it is more tractable and less expensive in terms of computation power and data storage.

5 Throughput maximization for special cases

We have so far solved the throughput maximization problem for the multi-way relay channel model in offline and online settings. This channel model has some interesting special cases that also provide us with insights into power allocation in communication networks with energy harvesting transmitters. We list below these special cases and state how Equation 3 should be modified to solve the throughput maximization problem for each special case. Note that while the infinite buffer versions of most of these problems (with the exception of the multi-pair two-way relay channel) are available in the literature, the finite buffer constraint is new.

5.1 Multi-pair relay channel

The multi-pair relay channel consists of L pairs of source nodes and a relay where each source node is only interested in the message of the other user in its pair. We again consider the case with no direct link between the users, so the relay node is necessary for communication. Then, the multi-pair relay channel is a special case of the multi-way relay channel with K=2. The optimal policy that maximizes throughput in an energy harvesting multi-pair relay channel can be found by solving the convex problem in Equation 3 with K=2.

5.2 Two-way relay channel

The two-way relay channel is a special case of the multi-way relay channel with L=1 cluster and K=2 users per cluster. Thus, the optimal policy that maximizes throughput can be found by solving Equation 3. This channel model is also considered in [28] with stochastic data arrivals and buffers at the relay where it is shown that the throughput maximization problem is convex and can be solved numerically.

References [26] and [27] study the energy harvesting two-way relay channel with no buffers at the relay and infinite backlog at the sources. In this special case, data causality and buffer size constraints at the sources (Equations 4 and 5) and at the relay (Equation 3h) are not present. Therefore, the amount of data that is received at the relay in a given epoch has to be the same as the amount of data forwarded by the relay in that epoch. The optimal policy can be found by solving Equation 3 with:

• L=1 and K=2,

• $B_{1,i}^{(\rm max)} = \infty$, i=1,2, i.e., infinite-capacity buffers at the source nodes due to the infinite backlog assumption,

• ${\bar B}_{1,i}^{(\rm max)} = 0$, i=1,2, i.e., no buffers at the relay,

• $B_{1,1}^{(1)} = B_{1,2}^{(1)} = \infty$, i.e., infinite backlog of data available at the beginning of transmission at the source nodes,

• $B_{1,1}^{(n)} = B_{1,2}^{(n)} = 0$, n∈I _N ∖{1}, i.e., no further data arrivals at the source nodes.

5.3 Two-hop channel

The two-hop channel is a special case of the multi-way channel with L=1 and K=2, but it differs from the two-way relay channel in that data flow is only in one direction, i.e., one of the two users only transmits and the other one only receives. Thus, the optimal policy that maximizes throughput can be found by solving Equation 3 with $E_{1,2}^{(n)} = 0$, n∈I _N . This way, node T _1,2 is energy deficient and cannot transmit any data to T ₀, so data flow is allowed only in one direction, from T _1,1 to T _1,2 via T ₀.

This channel model is considered in [23] and [25] but with the assumption that the transmitter T _1,1 has an infinite backlog of data; thus, stochastic data arrivals are not allowed. The optimal policy for this special case is the solution of Equation 3 with $E_{1,2}^{(n)} = 0$, n∈I _N in addition to all the modifications given above for the two-way relay channel with infinite backlog.

5.4 Two-way channel

The two-way channel consists of two users that each have a message which they wish to transmit to the other user. Although a relay node is not present, the two-way channel can still be considered as a special case of the multi-way relay channel and the optimal policy can be found with stochastic data arrivals at the two source nodes. In order to find the optimal policy, one needs to solve Equation 3 with:

• L=1 and K=2,

• $h_{1,2}^{I} = h_{1,2}^{II} = \infty$ to merge nodes T ₀ and T _1,2 together,

• ${\bar B}_{1,1}^{(\rm max)} = {\bar B}_{1,2}^{(\rm max)} = \infty$ to remove the buffer size constraints at the relay.

This way, nodes T ₀ and T _1,2 form a source node T ₀∪T _1,2 that transmits its own messages to and receives messages from node T _1,1. This channel model has not yet been studied with stochastic data arrivals and energy harvests at the two sources.

5.5 Single-user channel

The energy harvesting single-user channel consists of one transmitter and one receiver with data flow in only one direction. Similarly to the two-way channel, it can be modeled as a special case of the multi-way channel although it does not involve any relaying operation. The optimal policy that maximizes throughput in a single-user channel with stochastic data arrivals at the transmitter is the solution of Equation 3 with:

• L=1 and K=2,

• $h_{1,2}^{II} = \infty$ to merge nodes T ₀ and T _1,2 into a receiver node,

• ${\bar B}_{1,1}^{(\rm max)} = \infty$ to remove the buffer size constraints at the relay,

• $E_{1,2}^{(n)} = 0$, n∈I _N to prevent node T _1,2 from transmitting.

This way, T ₀ and T _1,2 are merged into a receiver node T ₀∪T _1,2 to reduce the model into a one-hop channel. Also, T _1,2 is not allowed to transmit which results in a single-user channel where T _1,1 is the source and T ₀∪T _1,2 is the receiver. This channel model is studied in [3] with stochastic data arrivals at T _1,1 and in [5] with a finite-capacity battery and an infinite backlog of data at T _1,1.

Remark 2.

Online policies for the special cases can be found by solving Equations 23 and 24 under the restricted settings given above for each case.

6 Numerical results

In this section, we present numerical examples of the solution of Equation 3 for various channel setups and also evaluate the performance of the offline and online solutions with respect to varying energy harvest rates and buffer capacities.

Figure 3 shows the optimal offline policy to maximize throughput in a multi-way relay channel with L=2 clusters that each contain K=3 users with unit channel gains, random epoch lengths $l_{n}, E_{0}^{(\rm max)} = 5 J, E_{j,i}^{(\rm max)} = 5 J,B_{j,i}^{(\rm max)} = 10$ kbit, and $\bar {B}_{j,i}^{(\rm max)} = 10$ kbit, for j=1,2 and i=1,2,3. The energy tunnel refers to the space of policies that satisfy the energy causality and battery constraints (Equations 3b to 3e). The upper wall of the energy tunnel represents the cumulative harvested energy at all times during transmission, and the lower wall represents how much of this cumulative energy would have to be wasted, if not utilized, due to the finite capacity of the batteries. As can be seen, the offline policy is not necessarily the shortest path between the beginning and the end of the tunnels, which was the case for some of the previously studied models, e.g., [3] and [5]. The observation that $\Delta _{0}^{(5)}$, the fraction of phase I in the last epoch, is low suggests that the relay uses its buffers to store data to transmit in the last epoch. As a special case of the multi-way relay channel, Figure 4 shows the optimal offline policy to maximize throughput in a multi-pair relay channel with L=3 pairs that each contain K=2 users with unit channel gains, random epoch lengths l _n , $E_{0}^{(\rm max)} = 5$ J, $E_{j,i}^{(\rm max)} = 5$ J, $B_{j,i}^{(\rm max)} = 10$ kbit, and $\bar {B}_{j,i}^{(\rm max)} = 10$ kbit, for j=1,2,3 and i=1,2. Similarly, we see that the optimal offline policy does not have to be the shortest path. We also observe that $\Delta _{0}^{(1)}$, the fraction of phase I in the first epoch, is high. This is because node T _3,2 harvests a very low amount of energy at the beginning of transmission, and the buffers at the relay are empty at the beginning of transmission. Thus, in order to utilize the relay power efficiently, more time has to be spared for the source nodes to transmit their data to the relay.

Figure 3

Optimal policy for a multi-way relay channel ( L =2, K =3) with stochastic data arrivals.

Figure 4

Optimal policy for a multi-pair relay channel ( L =3, K =2) with stochastic data arrivals.

Figure 5 shows the optimal offline policy that maximizes throughput in a symmetric two-way relay channel with unit channel gains, $E_{0}^{(\rm max)} = 7$ J, $E_{1,i}^{(\rm max)} = 7$ J, $B_{1,i}^{(\rm max)} = 2~\text {kbit}$, $\bar {B}_{1,i}^{(\rm max)} = 1~\text {kbit}$, i=1,2 and random epoch lengths, l _n . We observe that the optimal offline policy is not always the shortest path between the beginning and the end of the energy tunnel. This is due to the fact that the optimal policy must consider how the energy harvesting profiles at all nodes should interact for optimality. For example, node T _1,1 does not choose the shortest path for the first two epochs but rather chooses to transmit at less power in the first epoch. This is because the other two nodes T ₀ and T _1,2 harvest very low amounts of energy at s ₁ and thus cannot transmit with high power. In order for the relay to utilize its low energy in the first epoch, T _1,1 also lowers its power. But now, T _1,1 has more energy to spend in the second epoch which will otherwise be wasted since $E_{1,1}^{(3)}$ almost fills its battery. In order to utilize this extra energy at T _1,1 in the most efficient fashion, the fraction of phase I in the second epoch, $\Delta _{0}^{(2)}$ is increased to almost 1, which could possibly cause the relay to drop some of the packets if it did not have data buffers.

Figure 5

Optimal policy for a two-way relay channel ( L =1, K =2) with stochastic data arrivals.

Figure 6 shows the optimal offline policy for the same setup, except with infinite-capacity batteries and delayed data arrivals at the source nodes. Node T _1,1 starts receiving data packets at s ₂ and node T _1,2 starts receiving at s ₅. Nodes T _1,1 and T _1,2 do not spend any energy while they do not have any data. The relay chooses to forward only T _1,1’s messages in epochs 2, 3, and 4, instead of saving its energy until s ₅ which would be suboptimal since the relay has a sufficient amount of energy for later epochs, and higher rates are achievable if communication takes place only in one direction. An example of communication in only one direction throughout transmission is shown in Figure 7 where we set $E_{1,2}^{(n)} = 0$ J, n=1,2,…,N to prevent node T _1,2 from transmitting. This special case is a two-hop channel. In Figure 7, the power values at node T _1,2 are not shown since they have to be zero with no harvested energy. As can be seen, the optimal policy takes into account the energy harvesting profiles of both nodes, T ₀ and T _1,1, and selects fraction values $\Delta _{0}^{(n)}$ accordingly.

Figure 6

Optimal policy for a two-way relay channel ( L =1, K =2) with delayed data arrivals.

Figure 7

Optimal policy for a two-hop channel.

Figure 8 shows the throughput achieved by the optimal offline policy and the online policy along with lower- and upper-bounds for comparison in a multi-way relay channel setup with L=3 clusters that each contain K=3 users, unit channel gains, $E_{0}^{(\rm max)} = 5$ J, $E_{j,i}^{(\rm max)} = 5$ J, $B_{j,i}^{(\rm max)} = 10$ kbit, and $\bar {B}_{j,i}^{(\rm max)} = 10$ kbit, for j=1,2,3 and i=1,2,3. The peak energy harvesting rate is fixed at 5 J for all nodes except T _1,1 and varied from 0 to 5 J for T _1,1. The upper bound corresponds to the case where all nodes are assumed to receive all the energy at the beginning of transmission with infinite batteries. This assumption removes the constraints in Equations 3b to 3e and results in a larger feasible region for Equation 3; thus, we get an upper bound on the achievable throughput. The lower bound is the hasty policy where the nodes do not have batteries, so they cannot store energy for later epochs, and also the relay does not have data buffers. This way, the solution is computationally less expensive, but the feasible region is very limited. The online policy is found by iterating Equation 25 for each peak harvest rate for T _1,1. As can be seen in Figure 8, the throughput curves achieved by the optimal offline policy and the online policy are monotonically increasing and concave in the peak harvest rate for node T _1,1. The throughput achieved by the offline policy is significantly higher than the lower bound achieved by the hasty policy, which demonstrates how batteries and data buffers can help achieve higher performance in an energy harvesting network, and is very close to the upper bound under energy deficient conditions which are more likely to occur in an energy harvesting setup. The online policy appears to have slightly lower performance compared to the offline policy. This is due to two main factors: (a) The offline policy uses the entire energy harvest and data arrival profile for the system at all points during transmission, whereas the online policy only has causal knowledge, and (b) for numerical purposes, we consider a suboptimal online policy found by iterating Equation 25.

Figure 8

Throughput with lower and upper bounds for varying peak harvest rates for node T _1,1 .

Figure 9 shows throughput values achieved by the offline and online policies with Rayleigh fading. The average normalized channel gain ranges from −40 to 10 dB for all links. Here, only the offline policy has noncausal knowledge of the fading patterns as well as the energy and data arrivals. Consequently, we observe that the difference between the optimal throughput values grows with better links between the source nodes and the relay.

Figure 9

Throughput for varying average fading levels for all links.

Lastly, Figures 10 and 11 show how the buffer sizes affect the achievable throughput in the multi-way relay channel setup given above. For the solid curve in Figure 10, the sizes of the buffers at the relay, $\bar {B}_{j,i}^{(\rm max)}$, j=1,2,3 and i=1,2,3 are set equal, and this value is raised from 0 kbit, which corresponds to the case where the relay does not have any data buffers, to 4 kbit. As can be seen, larger buffers allow the relay to store more data for later epochs when the relay can more efficiently forward messages, and naturally, the achieved throughput increases in buffer size. However, after a certain buffer size, the achieved throughput is not affected by increasing buffer size. This is because at this critical point, the relay has sufficient storage for optimality and larger buffers are not necessary. This gives some idea about required buffer sizes in a practical energy harvesting multi-way communication system as the buffers in a practical system are bound to be limited in capacity. Note that in Figure 10, an upper bound with all sources sharing a single buffer at the relay is also presented for completeness. (For the upper bound, the average buffer size per source node is shown on the horizontal axis). The dashed curve in Figure 10 corresponds to this upper bound where the relay node employs only one data buffer to be shared among all source nodes and optimally allocates buffer sizes between the LK source nodes. In Figure 11, all nodes are given buffers with the same capacity which is varied from 0 to 2 kbit. Similarly to the previous figure, we observe that the achieved throughput saturates and is not affected by buffer sizes higher than a critical value. As a final remark, we note that communication is possible in the absence of data buffers at the relay or at all nodes as can be seen in Figures 10 and 11. However, as expected, the throughput improves by employing data buffers.

Figure 10

Throughput for varying buffer sizes at the relay.

Figure 11

Throughput for varying buffer sizes at all nodes.

7 Discussion and conclusion

In this paper, we studied the energy harvesting multi-way relay channel with finite batteries, finite buffers, and stochastic data arrivals at the source nodes. We formulated the offline throughput maximization problem and solved it with the aid of added slack variables and using an iterative algorithm. We also expressed the online throughput maximization problem as a dynamic program and solved it for online policies. We then showed that simpler channel models, some of which have been studied before, can be considered as special cases of our model and the solution to the throughput maximization problem for these cases can be found using our method. We provided and discussed numerical results for our general model and some of its special cases to understand how optimality can be achieved in an energy harvesting network with stochastic data arrivals. Lastly, we compared the performance of our offline and online solutions with an upper bound and the throughput achieved by the hasty policy and also demonstrated how the buffers at the relay help achieve higher throughput.

We note that relaying schemes other than decode-and-forward can also be considered for a multi-way relay channel with stochastic data arrivals. For example, the lattice-forwarding scheme in [42], which performs better in phase I than decode-and-forward at high power, is shown to improve performance in [27] in an energy harvesting two-way channel with no buffers at the relay. It is also shown in [29] that lattice-forwarding may improve performance in an energy harvesting multi-way channel. However, considering decode-and-forward relaying suffices to give us insights into multi-way communications with energy harvesting, and hybrid strategies for the same setup are left as future work.