Subcarrier allocation based on correlated equilibrium in multi-cell OFDMA systems
© Zheng et al.; licensee Springer. 2012
Received: 14 January 2012
Accepted: 11 July 2012
Published: 25 July 2012
In uplink orthogonal frequency division multiple access (OFDMA) systems, efficient resource allocation can greatly improve system performance. In this article, taking throughput, inter-cell interference and complexity into account, we present a game-theoretical approach to perform distributed subcarrier allocation for multi-cell OFDMA systems with limited base station (BS) coordination. Firstly, we construct a multi-cell resource allocation game. Creatively, the subcarrier is viewed as a game player to choose the most satisfying user and the BS acts as a referee or a coordinator. Then, we introduce the correlated equilibrium which helps the non-cooperative players coordinate their strategies, hence, achieves better performance compared to Nash equilibrium. Particularly, we point out the condition under which the correlated equilibrium is Pareto efficient. Moreover, we propose a novel subcarrier allocation algorithm based on the no-regret procedure that guarantees convergence to correlated equilibrium, in which the BS coordinates the players’ strategies and provides only partial information exchange. Extensive simulation results are provided to demonstrate the effectiveness of the proposed algorithm.
Orthogonal frequency division multiple access (OFDMA) has emerged as one of the most promising multiple access techniques for high data rate transmission over wireless channels due to its ability to mitigate multipath fading and its efficient implementation using IFFT and FFT blocks. The most recently proposed next generation wireless communication technologies, such as wireless wide area network (WWAN) standards, 3GPP2 ultra mobile broadband (UMB), IEEE 802.20 mobile broadband wireless access (MBWA), 3GPP LTE and worldwide interoperability for microwave access (WiMAX) are all OFDMA based.
In an OFDMA system, the spectrum is orthogonally divided into time-frequency resource blocks (RBs), which increases flexibility in resource allocation, thereby allowing high spectral efficiency. Exploiting all RBs simultaneously in every cell to achieve universal frequency reuse becomes a key objective toward the deployment of 4G networks. Focusing on the universal frequency reuse scheme, inter-cell interference is a major impairment that limits the system throughput. In multi-cell environment, one of the major issues to research is how to maximize the performance by controlling the co-channel interference among the neighboring cells. Interference coordination can be fulfilled by allocating system resources with interference awareness in terms of frequency, time, transmit power and space, etc.[5, 6]. Due to limited and precious radio resources in cellular networks, interference aware resource allocation is a challenging problem and has received much attention from both research and standardization communities in recent years[5–9]. Moreover, because any change of resource allocation in a specific cell will affect the performance of the nearby cells, joint resource allocation considering both throughput and interference over a cluster of neighboring cells via BS coordination is a promising solution.
Recently, BS coordination, where neighboring BSs connected through high-speed wireline links only share channel state information (CSI) and can jointly compute their transmit power and user scheduling, has been proposed as a major technique to mitigate co-channel interference, since it shifts the signal processing burden to the BSs. Many works have been done on coordinated resource allocation in cellular wireless networks, including both centralized and distributed procedures. Centralized algorithms (e.g.,[12–14]) require global information to decide the user assignment and transmit power in each cell. The problem is often formulated as an optimization task subject to bit rate, power level, or other types of constraints[5, 8]. However, since most of the optimization problems in cellular networks can be proved to be NP-hard[15, 16] (as a mixed integer-nonlinear problem), standard optimization techniques do not apply directly and even centralized algorithms cannot guarantee that the globally optimal solution is found. In addition, even if computational issues were to be resolved, the optimal solution still requires a central controller updated with instantaneous intercell channel gains which would create serious signaling overhead issues, thus hinder it from practice. Consequently, distributed algorithms (e.g.,[11, 18–20]) are more attractive as they do not require a central controller and may demand less information exchange and computational complexity.
Game theory, which is naturally the dominant paradigm for analyzing the decentralized framework, is recently adopted by many researchers to seek for a satisfactory solution to the problem of resource allocation and/or interference coordination[4, 21–23]. Kwon and Le design the utility function that represents the weighted sum of the data rates and the power consumption in a cell. The problem of maximizing the utility under the maximum power constraint is modeled as a noncooperative resource allocation game, in which the BS is viewed as the game player. Liang et al. focus on the adaptive allocation of subcarrier, bit, and power among BSs of a downlink multi-cell OFDMA systems. The utility function takes both data rate and power consumption into account. However, the authors have not formulated the problem from the perspective of interference mitigation. In, a noncooperative game in which each user selfishly tries to minimize its own transmitted power subject to a transmission rate constraint is proposed. Nevertheless, the proposed game is not guaranteed to converge to a Nash equilibrium, and therefore a virtual referee is introduced to monitor the resource allocation and force it to a stable and efficient equilibrium point. Al-Zahrani et al. consider a transmit power adaptation method using a noncooperative game theory approach to reduce the inter-cell interference in the OFDM networks. The throughput is enhanced by finding the optimum transmit power for each co-channel user using game theory-based scheme. However, no subcarrier allocation is discussed.
The existing works based on game theory mainly concentrate on power control, while subcarrier allocation is more or less simplified. Moreover, the power allocation is easier to be solved by continuous game method, while the discrete game applying to subcarrier allocation is much harder that few works consider. Thus in this work we make a game-theoretic study on the distributed subcarrier allocation algorithm in the uplink multi-cell OFDMA systems. Note that the pure non-cooperative game may result in non-convergence or some undesirable Nash equilibria with low system and individual performance. To enhance the performance, we introduce an important generalization of the Nash Equilibrium, known as the correlated equilibrium, which is more preferable than Nash equilibrium since it directly considers the ability of agents to coordinate their actions. This coordination can lead to better performance than if each agent was required to act in isolation.
The main contributions of this article are summarized as follows:
In this article, we formulate the subcarrier allocation problem in a novel point of view that each subcarrier performs as a game player to choose the most satisfying user, which guarantees the fairness from the perspective of the subcarriers. Therefore, it is different from the traditional subcarrier allocation, in which the subcarriers are allocated passively.
An efficient distributed learning algorithm is developed to perform subcarrier allocation for the multi-cell scenario, which achieves a good performance, jointly considering the throughput, interference and fairness. The proposed algorithm exhibits low complexity and converges to the set of correlated equilibria with probability one.
In general, the outcomes of individual optimization might not always be as good as those of system optimization. To solve this problem, the BS is introduced as a referee or coordinator, which is in charge of monitoring and improving the outcome of non-cooperative competition among the distributed players. Thus, strictly speaking, the approach is limited cooperation among BSs by adopting distributed algorithms, which is recognized as a good tradeoff between the performance gain nd the relevant cost, considering that they demand less information exchange and computational complexity.
The rest of this article is organized as follows. In Section “System model and problem formulation”, we present the system model and a novel utility function considering both throughput and interference. In Section “Correlated equilibrium for joint strategy selection”, we study the correlated equilibrium. Then, we construct a distributed subcarrier allocation algorithm based on no-regret procedure in Section “Distributed learning algorithm for joint strategy selection” and prove that the algorithm converges to a set of correlated equilibria. Simulation results are shown in Section “Simulation results and analysis” and finally conclusions are drawn in Section “Conclusion”.
System model and problem formulation
In each single-cell OFDMA network, K is always much larger than the number of users and no subcarrier can simultaneously support transmission for more than one user. Hence, all the users can simultaneously transmit data to the BS on one or more subcarriers without interfering others.
Each user is served by only one BS which locates in his cell. Thus, , for l ≠ l ′ .
The bandwidth of each subchannel is less than the coherence bandwidth of the channel so that each subcarrier experiences flat fading.
The subcarriers are perfectly orthogonal such that no intersymbol interference between adjacent symbols occurs.
Perfect synchronization is assumed so that there will not be intersubcarrier interference.
The BS periodically estimates the uplink channel gains on all subcarriers for all the users through pilot signals. And all the CSI needed can be accurately tracked by the BS.
The network is geographically static in the sense that the time scale of algorithm convergence is shorter than the channel’s coherence time. Thus, the channel gains on subcarriers remain unchanged during one implementation of the algorithm .
In this section, we model the subcarrier allocation as a multi-player discrete, finite strategy game, in which the subcarriers are considered as players. Therefore, there is a shift in perspective from a user’s view of allocation to a subcarrier’s view, and the subcarriers can choose the most satisfying users for themselves. It means that the fairness from the perspective of the subcarriers can be ensured.
Each BS is assumed to have access to all available subcarriers, i.e., the frequency reuse factor is 1. Consequently, each BS has K available subcarriers. Note that different cells have different user sets, which means that the same subcarrier in different cells has different strategy sets, thereby the same subcarriers in different cells should be treated as different players. Thus, we denote the subcarrier k in cell l by k l for distinguishing. And the strategy of player k l is denoted by, while the joint strategy by opponents of player k l is denoted by. Hence, is the joint strategy of all players, also known as a strategy profile. or denotes the utility function of player k l .
where and denote the users chosen by subcarrier k l and k l ′ , respectively. Obviously, user i and user j are in different cells. Subcarrier k l and k l ′ are the same frequency band which are allocated by different BSs. l and l ′ denote the cell index.
is the index set of the players (we use player, subcarrier interchangeably). In fact, the same subcarrier in different cells acts as the different players making their own decisions alone. Therefore, the number of the total players is KL.
is the strategy space of player k l . Obviously, . Therefore, the space for the joint strategy profile is defined by .
is the individual utility mapping the joint strategy space to the set of the real number.
Although the subcarriers can decide which user to choose at their will, they have no capability of performing strategy selection. Therefore, the subcarriers are virtual game players actually. In essence, the game is managed by BSs who act as the coordinators or referees, and BSs allocate the subcarriers according to the equilibrium point of the game.
Correlated equilibrium for joint strategy selection
In order to analyze the outcome of the proposed game, we focus on an important generalization of the Nash equilibrium, known as the correlated equilibrium that a strategy profile is chosen randomly according to a certain distribution given to the players by some “coordinator” or “referee”. Each player is given-privately-instructions for his own play only and the joint distribution is known to all of them. It is to the players’ best interests to conform to this recommended strategy, and the distribution is called the correlated equilibrium.
The inequality means that when the recommendation to player k l is to choose action, then choosing the any other action instead cannot obtain a higher expected utility.
For the multi-cell subcarrier allocation game G, a correlated equilibrium always exists.
The result from shows that every finite game has a correlated equilibrium. Hence, Theorem 1 is justified, and enables the application of the proposed game. □
The set of correlated equilibria is nonempty, closed and convex in G. In fact, every Nash equilibrium is a correlated equilibrium and Nash equilibrium corresponds to the special case where the action of the different players is independent, i.e.,. Moreover, the set of correlated equilibrium distributions of G is a convex polytope and the Nash equilibia all lie on the boundary of the polytope.
Optimal correlated equilibrium
where is the expectation over p. The constraints guarantee that the solution is within the correlated equilibrium set.
In the proposed game G, the correlated equilibrium which maximizes the expected sum of utilities of the subcarriers, P∗, is Pareto efficient.
If the resulting correlated equilibrium P∗ is not Pareto efficient, there must exist a different probability distribution P ′ such that,, and for some k l , thus, which contradicts the fact that P∗ is the optimal solution. The proof is completed. □
Distributed learning algorithm for joint strategy selection
In this section, we present a distributed learning algorithm which always leads to the set of correlated equilibria. From the result, each player can independently determine its own cooperative strategy. Concretely, the proposed algorithm is based on the no-regret procedure of . In this procedure, players may depart from their current play with probabilities that are proportional to measures of regret for not having used other strategies in the past.
Initialization: At the initial time n = 1, each player initializes its strategy arbitrarily
Iterative update process: For n = 1,2,3,…
For all, each player calculates the utility of the current strategy and the utility for choosing the different strategy.
Regret value update
where represents the average regret value at time n for not having played, every time that was played in the past, the different strategy.
Transition probability update
where μ is a normalization factor which is chosen to ensure the probabilities are non-negative.
At the period n + 1, k l updates its decision strategy according to the transition probability distribution.
In the proposed algorithm, each player does not need to be concerned about the individual strategies and utilities of other players, global network structure, etc. Each one just needs to know the effect of other players on its individual utility function. In addition, each player views its current actual strategy as a reference point, and makes a decision for next period according to propensities to depart from it. However, the change should bring the improvement in individual utility, relative to the current choice.
The implementation of the proposed algorithm needs the history of play given. And the BSs take the responsibility naturally and expediently, thus the cooperative strategy is obtained. As Hart and Mas-Colell observe in that, “there is a natural coordination device: the common history, observed by all players.”
where denotes the dimensional unit vector with the one in the position of Sn + 1.
If every player follows the proposed algorithm, the empirical distributions of play z n converge almost surely as to the set of correlated equilibria of our game. That z n converges to the set of correlated equilibria has been proved in many works, such as[24, 28, 34]. Here, we only provide a brief sketch of these proofs:
Huang and Krishnamuthy  prove convergence indirectly by proving an inequality which is originated from the Blackwell’s sufficient condition for approachability
In , the proof is based on a recursive formula for the distance of the vector of regrets to the negative orthant. In particular, by adopting multi-period recursion where a large “block” of periods is combined together instead of one-period recursion, the conditions of Blackwell’s approachability theorem are proved
In , the proof relies on a stochastic averaging theory. Due to the set theoretic nature of the correlated equilibira, the convergence analysis is carried out through a differential inclusion, which is the set theoretic extension of a differential equation.
Computational complexity analysis
At each iteration, each player k l needs to keep a record of the utility of choosing the current strategy and the utilities for changing to the other strategies. In addition, the proposed algorithm requires one table lookup, not more than n + KL additions and KL + 1 multiplication to update the regret value, and one comparison to choose the next strategy. And similar to the analysis in[35, 36], the complexity of our algorithm only depends on the number of player’s strategies, that is,.
The information exchange among the BSs only includes the history play of the players in other cells. Thus, the signaling overhead is very low.
Simulation results and analysis
We initialize the game with a random user assignment for each player. The players will take action to search for improvement in utility value by looking for the best response strategy after observing the opponent’s action.
In this work, we have presented a distributed subcarrier allocation approach with limited BS coordination for multi-cell OFDMA systems. The goal is to maximize the performance by controlling the co-channel interference at the same time. Concretely, we model a joint strategy selection game in a novel point of view that each subcarrier performs as a game player to choose the most satisfying user, which guarantees the fairness from the perspective of the subcarriers and focus on the implementation of the set of correlated equilibria to analyze the outcome of the proposed game. Moreover, since any change of resource allocation in a specific cell will affect the performance of the nearby cells and the outcomes of individual optimization might not always be as good as those of system optimization, joint resource allocation via BS coordination is considered. Then, we develop a novel distributed subcarrier allocation algorithm based on no-regret procedure to learn the correlated equilibrium, which demands less information exchange and computational complexity. The simulation results show that the proposed algorithm achieves good performance, such as quick convergence, large interference mitigation, evident capacity improvement, and good fairness. Further study could be focused on both the power and subcarrier allocation simultaneously to achieve a higher overall throughput of the system.
This study was supported by the National Natural Science Foundation of Jiangsu Province (No. BK2010101), the National Natural Science Foundation of China (No. 61001107) and the Major National Science & Technology Specific Project (No. 2010ZX03006-002-04).
- Wang MM, Ji T: Dynamic resource allocation for interference management in orthogonal frequency division multiple access cellular communications. IET Commun 2010, 4(11):675-682.MathSciNetView ArticleGoogle Scholar
- Cheng S-M, Lien S-Y, Chu F-S, Chen K-C: On exploiting cognitive radio to mitigate interference in macro/femto heterogeneous networks. IEEE Wirel. Commun 2011, 18(3):40-66.View ArticleGoogle Scholar
- Yang K, Prasad N, Wang X: An auction approach to resource allocation in uplink OFDMA systems. IEEE Trans. Signal Process 2009, 57(11):4482-4496.MathSciNetView ArticleGoogle Scholar
- Kwon H, Lee BG: Distributed resource allocation through noncooperative game approach in multi-cell OFDMA systems. Proceedings of the IEEE International Conference on Communications (ICC), (Istanbul, Turkey, 2006), pp. 4345–4350Google Scholar
- Liang L, Feng G: A game-theoretic framework for interference coordination in OFDMA relay networks. IEEE Trans. Veh. Technol 2012, 61(1):321-332.View ArticleGoogle Scholar
- Boudreau G, Panicker J, Guo N, Chang R, Wang N, Vrzic S: Interference coordination and cancellation for 4G networks. IEEE Commun. Mag 2009, 47(4):74-81.View ArticleGoogle Scholar
- Fodor G, Koutsimanis C, Rácz A, Reider N: Intercell interference coordination in OFDMA networks and in the 3GPP long term evolution system. J. Commun 2009, 4(7):445-453.View ArticleGoogle Scholar
- Rácz A, Reider N, Fodor G: On the impact of inter-cell interference in LTE. Proceedings of the IEEE Global Telecommunication Conference (GLOBECOM), (New Orleans, USA, 2008), pp. 1–6Google Scholar
- Mao X, Maaref A, Teo KH: Adaptive soft frequency reuse for inter-cell interference coordination in SC-FDMA based 3GPP LTE uplinks. Proceedings of the IEEE Global Telecommunication Conference (GLOBECOM), (New Orleans, USA, 2008), pp. 1–6Google Scholar
- Ng DWK, Schober R: Resource allocation and scheduling in multi-cell OFDMA systems with decode-and-forward relaying. IEEE Trans. Wirel. Commun 2011, 10(7):2246-2258.View ArticleGoogle Scholar
- Zhang H, Venturino L, Prasad N, Li P, Rangarajan S, Wang X: Weighted sum-rate maximization in multi-cell networks via coordinated scheduling and discrete power control. IEEE J. Sel. Areas Commun 2011, 29(6):1214-1224.View ArticleGoogle Scholar
- Ng CY, Sung CW: Low complexity subcarrier and power allocation for utility maximization in uplink OFDMA systems. IEEE Trans. Wirel. Commun 2008, 7(5):1667-1675.View ArticleGoogle Scholar
- Venturino L, Prasad N, Wang X: An improved iterative waterfilling algorithm for multi-cell interference mitigation in downlink OFDMA networks. Proceedings of Asilomar Conference on Signal, Systems and Computers (ACSSC), (Pacific Grove, USA, 2007), pp. 1718–1722Google Scholar
- Venturino L, Prasad N, Wang X: A successive convex approximation algorithm for weighted sum-rate maximization in downlink OFDMA networks. Proceedings of 2008 IEEE Conference on, Information Sciences and Systems, (Princeton, USA, 2008), pp. 379–384Google Scholar
- Lee SB, Pefkianakis I, Meyerson A, Xu S, Lu S: Proportional fair frequency-domain packet scheduling for 3GPP LTE uplink. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), (Rio de Janeiro, Brazil, 2009), pp. 2611–2615Google Scholar
- Hua Y, Zhang Q, Niu Z: Resource allocation in multi-cell OFDMA-based relay networks. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), (San Diego, USA, 2010), pp. 1–9Google Scholar
- Gesbert D, Kountouris M: Rate scaling laws in multicell networks under distributed power control and user scheduling. IEEE Trans. Inf. Theory 2011, 57(1):234-244.MathSciNetView ArticleGoogle Scholar
- Lv G, Zhu S, Hui H: A distributed power allocation algorithm with inter-cell interference coordination for multi-cell OFDMA systems. Proceedings of the IEEE Global Telecommunication Conference (GLOBECOM), (Honolulu, USA, 2009), pp. 1–6Google Scholar
- Mhatre VP, Papagiannaki K, Baccelli F: Interference mitigation through power control in high density 802.11 WLANs. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), (Anchorage, Alaska, USA, 2007), pp. 535–543Google Scholar
- Stolyar AL, Viswanathan H: Self-organizing dynamic fractional frequency reuse in OFDMA systems. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), (Phoenix, USA, 2008), pp. 691–699Google Scholar
- Liang Z, Chew Y, Ko CC: On the modeling of a noncooperative multicell OFDMA resource allocation game with integer bit-loading. Proceedings of the IEEE Global Telecommunication Conference (GLOBECOM), (Honolulu, USA, 2009), pp. 1–6Google Scholar
- Han Z, Ji Z, Liu KJR: Non-cooperative resource competition game by virtual referee in multi-cell OFDMA networks. IEEE J. Sel. Areas Commun 2007, 25(6):1079-1090.View ArticleGoogle Scholar
- Al-Zahrani AY, Yu FR: A game theory approach for inter-cell interference management in OFDM networks. Proceedings of the IEEE International Conference on Communications (ICC), (Kyoto, Japan, 2011), pp. 1–5Google Scholar
- Huang JW, Krishnamuthy V: Cognitive base stations in LTE/3GPP femtocells: a correlated equilibrium game-theoretic approach. IEEE Trans. Commun 2011, 59(12):3485-3493.View ArticleGoogle Scholar
- Huang H, Lau VKN: Delay-optimal user scheduling and inter-cell interference management in cellular network via distributive stochastic learning. IEEE Trans. Wirel. Commun 2010, 9(12):3790-3797.View ArticleGoogle Scholar
- Tan CK, Sim ML, Chuah TC: Blotto game-based low-complexity fair multiuser subcarrier allocation for uplink OFDMA networks. EURASIP J. Wirel. Commun. Netw 2011, 2011(1):1-14. doi:10.1186/1687-1499-2011-53 10.1186/1687-1499-2011-1View ArticleGoogle Scholar
- La QD, Chew YH, Soong BH: An interference minimization game theoretic subcarrier allocation algorithm for OFDMA-based distributed systems. Proceedings of the IEEE Global Telecommunication Conference (GLOBECOM), (Honolulu, USA, 2009), pp. 1–6Google Scholar
- Hart S, Mas-Colell A: A simple adaptive procedure leading to correlated equilibrium. Econometrica 2000, 68(5):1127-1150. 10.1111/1468-0262.00153MathSciNetView ArticleMATHGoogle Scholar
- Han Z, Pandana C, Liu K: Distributive opportunistic spectrum access for cognitive radio using correlated equilibrium and no-regret learning. Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), (Hongkong, China, 2007), pp. 11–15Google Scholar
- Hart S, Schmeidler D: Existence of correlated equilibria. Math. Operat. Res 1989, 14(1):18-25. 10.1287/moor.14.1.18MathSciNetView ArticleMATHGoogle Scholar
- Nau R, Canovas SG, Hansen P: On the geometry of Nash equilibria and correlated equilibria. Int. J. Game Theory 2003, 32: 443-453. doi:10.1007/s001820300162MathSciNetMATHGoogle Scholar
- Altman E, Bonneau N, Debbah M: Correlated equilibrium in access control for wireless communications. Lect. Notes Comput. Sci 2006, 39(7):173-183.View ArticleGoogle Scholar
- Han Z, Liu KJR: Resource Allocation for Wireless Networks: Basics, Techniques, and Applications. (Cambridge University Press, UK, 2008)View ArticleGoogle Scholar
- Benainm M, Hofbauer J, Sorin S: Stochastic approximations and differential inclusions, part ii: applications. Mathe. Operat. Res 2006, 31(4):673-695. 10.1287/moor.1060.0213View ArticleMATHGoogle Scholar
- Wang B, Han Z, Liu KJR: Peer-to-peer file sharing game using correlated equilibrium. Proceedings of the IEEE Conference on Information Sciences and Systems (CISS), (Baltimore, USA, 2009), pp. 729–734Google Scholar
- Zhou L, Wang X, Tu W, Muntean GM, Geller B: Distributed scheduling scheme for video streaming over multi-channel multi-radio multi-hop wireless networks. IEEE J. Sel. Areas Commun 2010, 28(3):409-419.View ArticleGoogle Scholar
- Han Z, Ji Z, Liu KJR: Fair multiuser channel allocation for OFDMA networks using Nash bargaining solutions and coalitions. IEEE Trans. Commun 2005, 53(8):1366-1376. 10.1109/TCOMM.2005.852826View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License(http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.