Dynamic power control for energy harvesting wireless multimedia sensor networks
 MohammedAmine Koulali^{1, 2}Email author,
 Abdellatif Kobbane^{1},
 Mohammed El Koutbi^{1},
 Hamidou Tembine^{3} and
 Jalel BenOthman^{4}
https://doi.org/10.1186/168714992012158
© Koulali et al; licensee Springer. 2012
Received: 1 October 2011
Accepted: 1 May 2012
Published: 1 May 2012
Abstract
Optimization of energy usage in wireless sensor networks (WSN) has been an active research field for the last decades and various approaches have been explored. In fact, A well designed energy consumption model is the foundation for developing and evaluating a power management scheme in network of energy constrained devices such as: WSN. We are interested in developing optimal centralized power control policies for energy harvesting wireless multimedia sensor networks (WMSN) equipped with photovoltaic cells. We propose a new complete information Markov decision process model to characterize sensor's battery discharge/recharge process and inspect the structural properties of optimal transmit policies.
1 Introduction
The recent technological advances in the fields of microelectronic, wireless communication along with reduction of production costs have motivated the development of a novel generation of wireless networks. wireless sensor networks (WSN) are articulated over a set of miniaturized battery powered devices (sensors) with communication capabilities and are expected to become highly integrated into our daily activities. This class of networks is perceived as an evolution of AdHoc networks with specific energy and computation limitations. Also, the increasing availability, at low cost, of multimedia devices (cameras, microphones,...) has triggered the emergence of multimedia wireless sensor networks (WMSN) [1, 2]. With the diversity of their application domains, ranging from healthcare and intelligent patient monitoring to disaster relief and industrial process supervision through intrusion detection and border protection, WMSN hold a promising future [3]. It is noteworthy that the volume and the nature of carried multimedia content, mainly composed of images and/or video streams, impose severe requirements on sensor's residual energy and available bandwidth.
The energy scarcity represents one of the major limitations of WMSN, indeed, postdeployment replacement of the sensors batteries is generally not practical or even impossible. Therefore, a proper management strategy of the residual energy happens to be a crucial prerequisite to any large scale WMSN deployment. In order to preserve the sensors energy, an optimal choice of transmit powers and an efficient scavenging mechanism of energy from the deployment environment along with an adequate topological placement of the sensors are necessary.
Energy harvesting [4–7] in the context of WSN received increasing attention from research community. Indeed, enabling sensors to replenish their energy reserves, extends the WMSN's deployment lifetime and enlarges their application domains. Energy harvesting sources are various and encompass solar, wind and vibratory sources. In this work we will focus our attention on sensors equipped with photovoltaic cells that realize solar energy transformation into electric energy needed to recharge their batteries.
Energy consumption optimization remains a crucial issue even for energy harvesting WMSN. In fact, the harvested energy should be exploited optimally to cope with energy sources periodicity (day/night cycles) and unpredictability (wind activity/inactivity periods). Since, most of energy consumption is incurred at transceiver level, a balance should be found between conflicting objectives: maximizing the achieved throughput while reducing energy consumption and consequently extending the sensor's battery lifetime. Optimal energy management policies for energy harvesting sensors are considered in [8]. The discounted throughput is maximized over an infinite horizon, where queuing for data is also considered. In [9], the authors consider a binary power control problem: at each slot the wireless device could either transmit at a constant power or remain silent. The authors consider only the single user case and the optimal transmission policy is shown to be of a threshold type for the soft and strict delay constraint cases. The authors of [10] presented a decentralized power control with stochastic channel variation scheme. The proposed scheme considers a cost function that accounts for the QoS of each user and the interference to other users. The single user optimal policy is generalized to the multiple users scenario for the ergodic regime where the spreading factor and the number of users grow infinitely but their ratio remains constant. In [11], the authors consider a single transmitterreceiver scenario where the transmitter has a finite buffer, and solved the problem of dynamically assigning rates/powers to packets in order to minimize the longterm average transmission energy subject to an upper bound on the buffer overflow probability. The problem is formulated as a constrained Markov decision process (CMDP) and an analytical solution is given and proved to be monotone in queue length. The authors of [12] use an evolutionary game theoretic formulation to characterize the equilibrium policy for power allocation under channel uncertainty and delayed imperfect payoffs. A heterogonous learning framework that accounts for user and technology specificities is proposed. The authors of [13] address the problem of network resource allocation for energyharvesting sensor platforms with timevarying battery recharging rates. They propose a joint approach that combines QuickFix for getting the optimal sampling rate and SnapIt that adapts the sampling rate with the objective of maintaining the battery at a target level. The considered networks are characterized by a special directed acyclic network graph (DAG) structure and the choice of a given rate implies a specific transmit power. In [14], an energy harvesting body sensor network formed by sensors with a twostate energy harvester device is considered. The authors develop policies based on the energyerror probability tradeoff to maximize successful transmission probability while minimizing probability of running out of energy. The developed strategies exploit the knowledge of the current energy level and the process governing event generation and battery recharge to select the appropriate transmission mode. The problem of throughput optimal energy allocation is studied for energy harvesting systems in a time constrained slotted setting in [15]. The structural properties of the optimal power allocation policy are obtained through dynamic programming and convex optimization. The optimal use of the harvested energy for different energy profiles and storage capabilities is discussed in [16] with the outage probability considered as a performance metric. The authors developed a discrete time Markov model of the evolution of battery and transmission state and provide optimal transmission strategy that minimizes outage probability.
Our objective is to define an optimal energy management policy for the centralized dynamic power control problem for energy harvesting WMSN. Thus, at each time slot, the transmission powers of all the sensors are fixed by the base station to maximize the expected system's throughput under minimum energy consumption.
The added value of our work covers the following points:

We formalize the centralized power control problem in the context of hierarchical energy harvesting WMSN as a complete information MDP.

A stochastic model for the discharge/recharge battery process for energy harvesting WMSN is provided.

We consider a novel utility function that balances the sojourn in a each battery state along with achieved throughput for a given transmit policy.

The structural properties of centrally computed optimal transmit policies are inspected.
This article is organized as follows: In Section 2, we give a mathematical formulation for dynamic power control problem in the context of energy harvesting WMSN. Optimal transmission policies are treated in Sections 3 and 4 for the single and multiple sensors scenarios. Then we present numerical simulation results in Section 5. Finally, we conclude the article and announce our future works in Section 6.
2 System model
We consider a WMSN formed by a set of sensors $\mathcal{N}=\left\{1,\dots ,n\right\}$, under the authority of a single base station or gateway. Each sensor is equipped by its own battery and uses power to communicate with the base station either directly or through multimedia processing hubs. We discretize each sensor's battery capacity to several intervals that give a more coarsegrained description of the battery state, e.g., full, medium, discharged. The sensors are equipped with photovoltaic cells that make them capable of harvesting solar energy while undergoing the discharge process. The residual energy of the battery dictates available transmission powers and the achieved throughput is affected by the chosen transmit powers of other sensors. Time is discrete and at each time slot t, each sensor knows its own battery state, whereas the battery state of the other sensors and their actions remain unknown.
The formulated problem fits within the MDP framework with full information and infinite planning horizon, where the base station will compute and provide each sensor with its optimal transmission strategy, given the fact that it has access to all sensors' information i.e environment, battery and radio channel status. In order to be aware of each sensor battery level, we assume that time is slotted into virtual slots that encompasses several physical slots. the initial physical slots will be affected to the sensors to communicate their battery levels to the base station in a TDMA fashion. The remaining slots will serve for data exchange with interference possibility. We use three bits to code the energy level of each battery and thus, we could represent up to eight battery states.
2.1 Mathematical formulation
Where:

${\mathcal{X}}^{j}=\left\{0,1,2,\dots ,m1\right\}$ is a finite set of states of sensor's j battery. A state of the battery represents some interval of percentage of the remaining energy. The energy level of the battery increases (respectively decreases) sequentially. Initially, the sensor's battery is in its highest state (m  1), as sensors perform their affected tasks (event detection, packets forwarding,...) they consume energy and their batteries energy levels decrease sequentially. The harvested solar energy will be converted into electrical energy and will increase the sensors batteries residual energy level sequentially.

$\forall {s}_{j,t}\in {\mathcal{X}}^{j},{A}^{j}\left({w}_{t},\phantom{\rule{2.77695pt}{0ex}}{s}_{j,t}\right)=\left\{{p}_{0},\dots ,{p}_{{s}_{j,t}}\right\}$ is a finite set of available transmission powers for sensor j. This set satisfies A^{ j }(w_{ t }, s_{ j, t } 1) ⊂ A^{ j }(w_{ t }, s_{ j, t }): more powers are available at higher states. A sensor makes a decision on its transmit power ${p}_{t}^{j}\in {A}^{j}\left({w}_{t},{s}_{j,t}\right)$ based on its remaining energy s_{ j, t }at time t.

q^{ j }is the state transition probability of sensor's j battery. Given the state of environment w_{ t }, the state s_{ j, t }of the battery of j, the state c_{ j, t }of the radio channel in the vicinity of j, and the actions of the others ${p}_{t}^{j}=\left({p}_{t}^{1},{p}_{t}^{2},\dots ,{p}_{t}^{j1},{p}_{t}^{j+1},\dots ,{p}_{t}^{n}\right)$ the new states are (s_{j, t+1}, c_{j, t+1}) with the probability ${q}^{j}\left(\left({s}_{j,t+1},\phantom{\rule{2.77695pt}{0ex}}{c}_{j,t+1}\right)\left({s}_{j,t},\phantom{\rule{2.77695pt}{0ex}}{c}_{j,t}\right),{p}_{t}^{j},\phantom{\rule{2.77695pt}{0ex}}{p}_{t}^{j}\right)$.

The discount factor λ indicates for a user the decay in the gain value with the evolution of the time.
2.2 Stochastic battery model for sensors with energy harvesting capabilities
At each time slot t, depending on the remaining energy off the battery, the sensor j makes a decision on its transmit power ${p}_{t}^{j}$. Denote by the policy ${d}_{j}^{\infty}=\left({d}_{0}^{j},{d}_{1}^{j},\dots ,{d}_{m1}^{j}\right)$ the collection of decision rules of that sensor under infinite planning horizon, where $\forall {s}_{i}\in {\mathcal{X}}^{j},\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{0.3em}{0ex}}{d}_{i}^{j}\in \Delta \left({A}^{j}\left({s}_{i}\right)\right)$ and Δ(A^{ j } (s_{ i })) stands for the space of probability distribution over sensor's j action space for the battery state s_{ i }.
3 Optimal policy for a single sensor
The solution of this linear program (LP) is obtained through application of the simplex method. After solving the LP above, we recover the optimal decision rules of the associated MDP by applying the rule [19]:

A discounted MDP has always a deterministic optimal decision rule [20] that we select based on the following criterion:$\forall {p}^{l}\in {A}^{j}\left({s}_{k}\right),{d}_{k}^{j}={p}^{l}\Rightarrow x\left({s}_{k},{p}^{l}\right)>0,\phantom{\rule{1em}{0ex}}{s}_{k}\in {\mathcal{X}}^{j}.$(10)
We argue that there exists structured decision rules for the MDP Ω.
Proposition 1. There exists optimal monotone nondecreasing decision rules on ${\mathcal{X}}^{j}$for the MDP Ω.
The detailed proof of Proposition 1 is given below:
 (1)
The immediate reward r^{ j } for sensor j is nondecreasing on ${\mathcal{X}}^{j}$ for all ${p}_{t}^{\prime j}\in {A}^{j}$.
 (2)The immediate reward r^{ j } for sensor j is superadditive.$\begin{array}{c}r\left({s}_{l,t}+1,{p}_{t}^{\prime j}\right)+r\left({s}_{l,t},{p}_{t}^{j}\right)r\left({s}_{l,t}+1,{p}_{t}^{j}\right)r\left({s}_{l,t},{p}_{t}^{\prime j}\right)=\hfill \\ \left(\frac{1}{{T}^{j}\left({s}_{l,t},{p}_{t}^{\prime j}\right)}\frac{1}{{T}^{j}\left({s}_{l,t}+1,{p}_{t}^{\prime j}\right)}\right)\text{log}\left(1+\frac{{p}_{t}^{\prime j}{h}^{j}}{{N}_{0}}\right)+\left(\frac{1}{{T}^{j}\left({s}_{l,t}+1,{p}_{t}^{j}\right)}\frac{1}{{T}^{j}\left({s}_{l,t},{p}_{t}^{j}\right)}\right)\text{log}\left(1+\frac{{p}_{t}^{j}{h}^{j}}{{N}_{0}}\right)\hfill \\ \phantom{\rule{4em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}\phantom{\rule{1em}{0ex}}=g\left({p}_{t}^{\prime j}\right)g\left({p}_{t}^{j}\right)\hfill \end{array}$
 (3)$Q\left(k,\phantom{\rule{2.77695pt}{0ex}}l,{p}_{t}^{j}\right)$ is nondecreasing on ${\mathcal{X}}^{j}:\text{Let}\left({s}_{k},{s}_{l}\right)\in {\mathcal{X}}^{j}\times {\mathcal{X}}^{j}$,$\begin{array}{c}{s}_{k}>{s}_{l}+1:\Delta Q\left(k,\phantom{\rule{2.77695pt}{0ex}}l,{p}_{t}^{j}\right)=Q\left(k,\phantom{\rule{2.77695pt}{0ex}}l+1,{p}_{t}^{j}\right)Q\left(k,\phantom{\rule{2.77695pt}{0ex}}l,{p}_{t}^{j}\right)=Q\left(k,\phantom{\rule{2.77695pt}{0ex}}l+1,{p}_{t}^{j}\right)={p}_{\text{harvest}}\ge 0.\hfill \\ {s}_{k}={s}_{l}+1:\hfill \end{array}$$\begin{array}{cc}\hfill \Delta Q\left(k,l,{p}_{t}^{j}\right)& =Q\left(k,l+1,{p}_{t}^{j}\right)Q\left(k,l,{p}_{t}^{j}\right)\hfill \\ =\sum _{i=k}^{i=m1}{q}^{j}\left({s}_{i,t}{s}_{l,t}+1,{p}_{t}^{j}\right){q}^{j}\left({s}_{i,t}{s}_{l,t},{p}_{t}^{j}\right)\hfill \\ ={q}^{j}\left({s}_{l,t}+1{s}_{l,t}+1\right)+{p}_{\text{harvest}}{p}_{\text{harvest}}\ge 0.\hfill \end{array}$
 (4)
$Q\left(k,\phantom{\rule{2.77695pt}{0ex}}l,{p}_{t}^{j}\right)$ is globally superadditive on ${\mathcal{X}}^{j}\times {A}^{j}$, denote by $\vartheta =Q\left(k,l+1,{p}_{t}^{\prime j}\right)+Q\left(k,l,{p}_{t}^{j}\right)Q\left(k,l+1,{p}_{t}^{j}\right)Q\left(k,l,{p}_{t}^{\prime j}\right)$:
s_{ k } > s_{ l } + 1: ϑ = P_{harvest}  P_{harvest} = 0.
We conclude by virtue of Theorem 6.11.6 of [20] that deterministic optimal monotone nondecreasing policies over the set X^{ j } exists for the MDP (1).
4 Optimality for all the sensors batteries lifetime
Under the assumption that each sensor uses CDMA with mutual orthogonal codes to communicate with the base station or the multimedia processing hubs, sensors transmissions do not interfere. Therefore, the overall policy $\left({d}_{1}^{\infty *},\dots ,{d}_{n}^{\infty *}\right)$ realized when every sensor adopts its optimal transmit power happens to be the optimal transmit policy for the overall system noted d^{∞}*.
5 Numerical investigations
Transmit power panel
Battery state  Available transmit powers (W) 

0  {P_{0} = 0.0} 
1  {P_{0}, P_{1} = 10.0, P_{2} = 17.0, P_{3} = 18.0} 
2  {P_{0}  P_{3}, P_{4} = 23.0, P_{5} = 25.0, P_{6} = 30.0, P_{7} = 35.0} 
3  {P_{0}  P_{7}, P_{8} = 50.0, P_{9} = 58.0} 
4  {P_{0} P_{9}, P_{10} = 65.0, P_{11} = 75.0} 
Optimal transmit power
Network State  Transmit power (W) 

{0, 0, 0}  {P_{0}, P_{0}, P_{0}} 
{{0, 0, 1}, {0, 1, 1}, {2, 2, 1}, {1, 0, 1}, {1, 2, 1}, {2, 0, 1}, {0, 2, 1}}  {P_{0}, P_{0}, P_{2}} 
{0, 0, 2}  {P_{0}, P_{0}, P_{7}} 
{{0, 1, 2}, {0, 1, 0}, {2, 1, 0}, {2, 1, 1}, {2, 1, 2}}  {P_{0}, P_{2}, P_{0}} 
{{0, 2, 2}, {0, 2, 0}}  {P_{0}, P_{7}, P_{0}} 
{{1, 0, 2}, {1, 2, 2}}  {P_{2}, P_{0}, P_{7}} 
{1, 1, 1}  {P_{2}, P_{0}, P_{2}} 
{{1, 1, 2}, {1, 1, 0}}  {P_{2}, P_{2}, P_{0}} 
{{1, 2, 0}, {1, 0, 0}}  {P_{3}, P_{0}, P_{0}} 
{{2, 2, 2}, {2, 2, 0}, {2, 0, 2}, {2, 0, 0}}  {P_{7}, P_{0}, P_{0}} 
6 Concluding remarks
In this article we considered the problem of dynamic centralized power allocation for energy harvesting WMSN. We focus on solar powered sensors and provide a stochastic model for the associated battery discharge/recharge process. The dynamic power control problem was formulated as a MDP and the structural properties of optimal transmission policies established. We plan, in a near future, to generalize our approach for the decentralized case with partial channel information using stochastic game theory.
Declarations
Authors’ Affiliations
References
 Akyildiz I, Melodia T, Chowdhury K: A survey on wireless multimedia sensor networks. Comput Netw 2007, 51(4):921960. 10.1016/j.comnet.2006.10.002View ArticleGoogle Scholar
 Almalkawi I, Guerrero Zapata M, AlKaraki J, MorilloPozo J: Wireless multimedia sensor networks: current trends and future directions. Sensors 2010, 10(7):66626717. 10.3390/s100706662View ArticleGoogle Scholar
 Fowler K: The future of sensors and sensor networks survey results projecting the next 5 years. In Proc Sensors Applications Symposium, 2009 (SAS 2009). New Orleans, LA, USA; 2009:16.View ArticleGoogle Scholar
 Sudevalayam S, Kulkarni P: Energy harvesting sensor nodes: survey and implications. IEEE Commun Surv Tutor 2010, PP(99):119.Google Scholar
 Seah W, Eu Z, Tan H: Wireless sensor networks powered by ambient energy harvesting (WSNHEAP)Survey and challenges. In Proc 1st International Conference on Wireless Communication, Vehicular Technology, Information Theory and Aerospace & Electronic Systems Technology, 2009. Wireless VITAE 2009. Aalborg, Danemark; 2009:15.View ArticleGoogle Scholar
 Kansal A, Srivastava M: An environmental energy harvesting framework for sensor networks. Proceedings of the 2003 international symposium on Low power electronics and design 2003, 481486.Google Scholar
 Kansal A, Hsu J, Zahedi S, Srivastava M: Power management in energy harvesting sensor networks. ACM Trans Embed Comput Syst (TECS) 2007, 6(4):3266. 10.1145/1274858.1274870View ArticleGoogle Scholar
 Sharma V, Mukherji U, Joseph V, Gupta S: Optimal energy management policies for energy harvesting sensor nodes. IEEE Trans Wirel Commun 2010, 9(4):13261336.View ArticleGoogle Scholar
 Wang H, Mandayam N, Goodman DJ, Ligdas P: Dynamic power control under energy and delay constraints. In Proc Global Telecommunications Conference, 2001 (GLOBECOM '01). San Antonio, TX, USA; 2001:12871291.Google Scholar
 Chamberland J, Veeravalli V: Decentralized dynamic power control for cellular CDMA systems. IEEE Trans Wirel Commun 2003, 2(3):549559. 10.1109/TWC.2003.811186View ArticleGoogle Scholar
 Ata B: Dynamic power control in a wireless static channel subject to a qualityofservice constraint. Operat Res 2005, 53(5):842851. 10.1287/opre.1040.0188MathSciNetView ArticleMATHGoogle Scholar
 Tembine H, Kobbane A, El Koutbi M: Robust power allocation games under channel uncertainty and time delays. In Proc Wireless Days (WD) (2010 IFIP). Venice, Italy; 2010:15.View ArticleGoogle Scholar
 Liu R, Sinha P, Koksal C: Joint energy management and resource allocation in rechargeable sensor networks. INFOCOM, 2010 Proceedings IEEE 2010, 19.Google Scholar
 Seyedi A, Sikdar B: Energy efficient transmission strategies for body sensor networks with energy harvesting. IEEE Trans Commun 2010, 58(7):21162126.View ArticleGoogle Scholar
 Ho C, Zhang R: Optimal energy allocation for wireless communications powered by energy harvesters. IEEE International Symposium on Information Theory Proceedings (ISIT) 2010, 23682372.Google Scholar
 Medepally B, Mehta N, Murthy C: Implications of energy profile and storage on energy harvesting sensor link performance. Global Telecommunications Conference, 2009. GLOBECOM 2009. IEEE 2009, 16.View ArticleGoogle Scholar
 Fallgren M: On the complexity of maximizing the minimum Shannon capacity in wireless networks by joint channel assignment and power allocation. In Proc 18th IEEE International Workshop on Quality of Service (IWQoS 2010). Beijing, China; 2010:17.View ArticleGoogle Scholar
 Schrijver A: Theory of Linear and Integer Programming. Wiley, New York; 1986.MATHGoogle Scholar
 Maros I: Computational Techniques of the Simplex Method. Volume 61. Springer, New York; 2003.MATHGoogle Scholar
 Puterman M: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York; 1994.View ArticleMATHGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.