A hierarchical game-based power allocation algorithm in 5G heterogeneous integrated networks

In this paper, we proposed a novel power allocation algorithm based on hierarchical game in 5G heterogeneous integrated networks. The radio network controller is the first layer, the base stations are the second layer, and the base station users are the third layer. The stackelberg game model is used between the first and second layers, in which the radio network controller is the leader and the base stations are the followers. The radio network controller sets the interference prices for the base stations and the users in the third layer can update their transmission power by non-cooperative game based on the setting interference price. According to the proposed scheme, the users power allocation can reach the Nash equilibrium through by non-cooperative game and obtain the optimal transmission power. Moreover, it is proved that the equilibrium solution is composed of the optimal price and the optimal transmission power, and the convergence of this algorithm is also verified. The simulation results show that the proposed algorithm can decrease the interference and improve the system capacity.

simulation results show that the proposed algorithm can decrease the interference and improve the system capacity, respectively.
stations are the followers. RNC analyzes the network situation and is responsible for the resource utilization and allocation of the whole network. RNC will allocate wireless network resources to each base station. Users only need to consider their own transmission rate. In order to improve the transmission rate, they need to improve their own transmission power. A non-cooperative game model among users is constructed to maximize the transmission rate. Then, the users can achieve the optimal performance and the whole system will achieve the stable state. Base stations need to have information interaction with RNC, while users do not need to interact with other users. Therefore, we adopt the semi-distributed power allocation which combines centralized and distributed power allocation.
The remainder of this paper is organized as follows. Section 3 outlines some related work. The system model is established in Sect. 4. Section 5 introduces the power allocation algorithm based on hierarchical game. Simulation results are presented in Sect. 6. At last, Sect. 7 concludes this paper.

Related work
With the continuous development of communication technology, the number of mobile users and the growth of data traffic, more new theories and methods are needed to solve various problems in the communication process. The idea of game theory has been widely used in wireless communication, such as spectrum allocation, resource allocation, routing design and other issues; these can be studied by using the relevant theoretical knowledge of game theory. There are many game theory models for the power allocation such as coalitional game, non-cooperative game, and stackelberg game, etc.
Sun et al. [4] use coalition formation game to allocate joint resource in device-todevice uplink underlaying cellular networks. In the two-tier cognitive femtocell network, LeAnh et al. [5] treat the sub-channel and power allocation optimization problem as NPhard and propose an autonomous framework. Then, they formulate the optimization problem as a coalitional game in partition form. Wang et al. [6] cluster users by coalition game, and then the power allocation among users in the cluster is based on Stackelberg game. Zhang et al. [7] use the cooperative Nash bargaining game theory to solve the joint uplink sub-channel and power allocation problem. To decrease the computation complexity, Liu et al. [8] use super-modular game for sub-channel matching and power allocation problems. In two-tier femtocell networks, Tsiropoulou et al. [9] treat the joint resource allocation problem as a two-variable optimization problem and utilize supermodular game to solve the above problem. Zhong et al. [10] consider combining power control and resource allocation. They propose a pricing-based game scheme and a link scheduling scheme to guarantee the quality-of-service and achieve the low-latency requirement. Zhang et al. [11] consider the delay constraints to guarantee the quality-ofservice and bring in the concept of effective capacity and use super-model game to solve the power allocation problem.
In the process of non-cooperative game, there is no consensus among the players. The players take their own interests as the starting point to take the corresponding strategic plan; the focus is to pursue the maximization of personal interests. Gonzalo et al. [12] use non-cooperative game to optimize downlink performance and solve the load problem. Leshem and Zehavi [13] use non-cooperative game to allocate channel resources, and finally achieve Nash equilibrium. Hamidi et al. [14] propose a distributed power allocation mechanism using a non-cooperative game model. Rahman et al. [15] solve the energy optimization by using non-cooperative game for an arbitrary Gaussian channel. In [16][17][18][19][20][21][22][23], non-cooperative game can effectively allocate system resources in hierarchical networks.
According to the rank order of the game, the players can be divided into leader and follower in stackelberg game. Leaders take action first, and then followers take their actions, which can be regarded as a two-level game process. The top level of the game means that leaders choose their own strategic behaviors according to the information of the followers. The bottom level of the game means that the followers observe the leaders' strategies and then choose their own strategies. When the game process reaches to equilibrium state, neither the leader nor the followers can increase the profit by changing their own strategies. Qi et al. [24] propose a two-stage pricing-based power allocation scheme based on a stackelberg game model and provide the optimal power allocation strategy. Yuan et al. [25] propose a CSI-based distributed channel-power allocation scheme. They use stackelberg game to solve the power control problem. Li et al. [26] use stackelberg game to allocate the power resource between the base station and multiple users. Su et al. [27] propose a stackelberg-game theoretic resource allocation scheme to allocate resource to mobile social users though brokers. Ruby et al. [28] use two separate stackelberg games to solve the power allocation problem and they need a centralized entity to connect these two games. Ahmad et al. [29] use bi-level stackelberg game to study the joint price and power allocation in spectrum sharing macro-femtocell networks. In [30][31][32][33][34][35][36], stackelberg game is widely used to solve the problem about power allocation in different communication environments including device-to-device communications, multiuser cooperative communication networks, heterogeneous NOMA networks and dynamic spectrum access networks.
In this paper, the game theories including non-cooperative game and stackelberg game are used into 5G heterogeneous fusion network to enhance system capacity and reduce interference. Considering the relationship between radio network controller and base stations, a three-layer heterogeneous network structure model composed of radio network controller, base station and users is established. Stackelberg game model is used between the first and second layer, and non-cooperative game is used for users' power allocation.

System model
In this section, a three-layer heterogeneous fusion network model is composed of radio network controller, base stations and base station users, respectively. RNC is the first layer, base stations are the second layer and the users are the third layer.
The system model is as follows in Fig. 1.
In the above three-layer network structure model, since each base station of the second layer is controlled and managed by the RNC in the first layer, the stackelberg game model is used between the first layer and the second layer. In the stackelberg game, RNC is the leader and base stations in the second layer play the roles of followers. The strategy behavior of the base station is going to adjust according to the strategy of the RNC. The positions and identities of the players in the game which are the users in the third layer are equal and each of them needs to decide their next strategic behavior according to the behavior of other users. They will use the non-cooperative game to acquire their transmission power according to the power allocated by the base stations in the second layer.
Orthogonal frequency division multiplexing access (OFDMA) technology is used in this communication network. Base station users share a common frequency band for data transmission. Assuming that the allocation of sub-channels has been completed in this system, multiple users belonging to the same base station cannot occupy a channel resource at the same time, so there is only one active user belong to the base station in a specific time slot. In this time slot, this active user transmits data information to its base station.
Base stations will choose their own optimal transmission power according to RNC's price. Users reach the Nash equilibrium by non-cooperative game and obtain the optimal transmission power. Then, the RNC obtains its optimal prices based on the optimal transmission power. At this time, the whole system reaches the stackelberg equilibrium.
The SINR of the user i is shown in Eq. (1).
The total number of base stations is N . p i and p j denotes transmission power of base station i and j , respectively. p 0 denotes transmission power of RNC. h ii denotes the power gain between base station i and its user, h i0 denotes the interference channel gain between RNC and base station i , and h ij denotes the interference channel gain between other base stations and base station i 's user. n 0 denotes the noise power.

Algorithm design
In the above three-layer heterogeneous fusion network model, RNC acts as the game leader, base stations acts as followers in stackelberg game and makes corresponding strategies according to the actions of RNC. Users get their optimal strategies through non-cooperative game method, that is, the optimal transmission power.
(1) In order to reduce the interference to users, RNC sets prices for each base station. If the price is too low and base station's power is too high, it will cause more interference. For its own utility and benefit, RNC will increase its price. Base station's power will be reduced accordingly. RNC's price and base station's power interact with each other, forming a stackelberg game. Base stations will choose their own optimal strategies according to RNC's price. The Nash equilibrium is achieved by non-cooperative game between users, and thus the optimal transmission power is obtained. Then, the RNC's optimal price is obtained according to the optimal transmission power. At this time, the whole system reaches a stable state. The set of the optimal price and the optimal transmission power is the equilibrium solution of the game.
The utility function of base station i is shown in Eq. (2).
w i denotes the channel transmission bandwidth. RNC's price for base station is denoted by i . At the same time, RNC should set reasonable price to reduce the interference to users. The utility function of RNC is shown in Eq. (3).
In summary, RNC's optimization problems can be expressed in Eq. (4).

R L
i and R U i are the lower and upper bounds of the link transmission rate, respectively, to ensure the quality of service while the base station i and its user transmitting data.
Base station's optimization problems are expressed in Eq. (5).
Equations (4) and (5) constitute a stackelberg game process. When RNC has mastered the optimal strategy of the base station, it will set reasonable prices to the base stations.
The base stations will adjust their own transmission power by observing the strategy of RNC and base stations will take corresponding strategies to adjust their own transmission power to maximize their benefits. Then, for a given price, users reach Nash equilibrium by the non-cooperative game and they will get the optimal transmission power. The optimal power can be obtained by iteration and then the prices of base stations set by RNC can be obtained and the whole system reaches Stackelberg equilibrium.
While solving the above optimization problems, the backward method is used to find the equilibrium solution when the users' non-cooperative game reaches Nash equilibrium, that is, to find the optimal transmission power, and prove the existence and of the equilibrium solution. The next step is to change the optimal transmission power into matrix form and simplify it into the utility function of RNC, and then we can get the optimal price. According to Eq. (2), we can get the optimal transmission power which is shown in Eq. (6).
Because the power is greater than zero, the power set is not empty.
Each base station in the system will be allocated power, so the p set is a non-empty set. And the utility function U i of base station i is a continuous function of p i and Eq. (7) clarifies that U i is a convex function. Thus, Eq. (2) is a continuous convex function. Equation (2) satisfies the two existence conditions for Nash Equilibrium solutions. Equation (7) is the optimal transmission power. Equation (7) can be expressed in Eq. (8). And H is shown in Eq. (9).
By substituting the optimal transmission power obtained into the optimization problem of RNC, U RNC can be expressed in Eq. (10).
Because the base stations are very densely distributed in 5G heterogeneous integrated network, the interference fading from any base station to other base station's users is the same. So the interference fading from one base station to its users is much smaller than that from the base station to other users, we can consider h ij as a constant number. Then, H can be expressed in Eq. (11).
And let α i = h/(h ii − h) . Then, H 's inverse matrix is shown in Eq. (12). And A is shown in Eq. (13).
Because of Eq. (14), A is a zero matrix.
Equation (10)  Then, we can get the optimal RNC's price which is shown in Eq. (18).

(11)
Equations (6) and (18) are the equilibrium solution of stackelberg game. In order to verify the convergence of the proposed algorithm, we set an iteration function, the power of the iteration function can gradually converge to the equilibrium point, and the system will reach a stable state. The iteration function is shown in Eq. (19).
Because the power is greater than zero, Eq. (19) satisfies positive definiteness. The optimal transmission power can be expressed in Eq. (20).
Equation (20) (19) satisfies positive definiteness, monotonicity and measurability. It is a standard function and it is convergent. Then, we propose a power allocation algorithm based on the above iteration functions. The pseudocode of the algorithm is given in Algorithm, which can display the proposed power allocation algorithm entirely and accurately.
In this algorithm, RNC is the leader of the game, and base stations are the followers. It is divided into the following steps.
(1) According to the base station's initial transmission power, we can calculate each base station's initial price, and then RNC broadcasts the prices to base stations; (2) After receiving the price, the base station adjusts its transmitting power according to the iteration formula. (3) After the iteration, each base station gets its optimal transmission power and RNC gets the optimal price for each base station.
The base station receives RNC's price and iterates its power to the optimal transmission power. Then, RNC updates its own price according to the convergent power and gets the optimal price. The equilibrium solution of the game is composed of both the optimal price and the optimal transmission power of base station.

Simulation analysis
According to the power allocation algorithm proposed in the previous chapter, 15 base stations are set up. And the link gain is defined as follows.
The transmission loss between RNC and base station is K i0 . The transmission loss between base station i and its user is K ii . The transmission loss between base station j and base station i 's user is K ij . The distance between RNC and base station is d i0 . The distance between base station i and its user is d ii . The distance between base station j and base station i 's user is d ij .
In order to verify the performance of this algorithm, the simulation is carried out using MATLAB 2016b. The simulation conditions are shown in Table 1. We set the RNC's power as 45 dBm and the base stations' power as 20 dBm before iteration. The (23) base station position is determined by setting different distances from the RNC and the distance value is set randomly. Figure 2 shows the change of base station transmission power following the iterations. It shows the transmission power of each base station reaches equilibrium after two iterations, and the iteration gradually converges. In order to maximize the throughput, the base station will continuously improve its own transmission power to achieve the increase in throughput. Since the base station improves the transmission power, it will produce interference. At the same time, due to the interference, the base station will continue to increase the transmission power, which will lead to more interference. After setting the RNC price, the base station not only needs to consider its own throughput, but also needs to consider the cost of interference. It can be seen from Fig. 2 that with the increasing of RNC price, the base station will pay more for interference, resulting in the decrease in its utility function, and finally the transmission power of the base station will continue to decrease. Figure 3 shows RNC's price for each base station when the iteration reaches a stable state. It shows that with the increasing RNC price, the base station will pay more because of the interference. And it results in the reduction in its utility function, and ultimately the transmission power of the base station will decrease.  In order to validate the effectiveness of the proposed power allocation method, other two power allocation methods are compared as references. One is the method with the fixed price of RNC, and the other is with the unupdated price. The distance between RNC and each base station is set randomly. The 15 base stations are arranged according to the distance between the base station and the RNC from near to the far and the x-axis is labeled 1-15 in Figs. 4 and 5. RNC sets a unified price for all base stations under the fixed-price and unupdated-price schemes and this cause some base stations' transmitting power with small interference will be greatly reduced. In order to avoid affecting the transmission performance of the whole system, RNC sets different prices for each base stations according to the interference generated by different distances in the pricing update scheme. After receiving RNC's price, the base station  adjusts its own power. RNC updates the prices for different base stations according to the optimal transmission power obtained by the base station adjustment. RNC obtains the optimal price and the optimal transmitting power for each base station through iteration. Figure 4 compares the RNC's prices of each base station under three different methods. The pricing update method adopted in this paper will set a reasonable price compared with other two methods.
Regardless of the fixed pricing, Fig. 4 shows that the interference price is related to the distance between RNC and base stations. When the base station is close to RNC, the interference will be larger, while RNC will set a higher price for the base station. The base station will pay more for adjusting its transmission power. When the base station is far away from RNC, with less interference, RNC will reduce its price. Figure 5 shows the utility function of base station under three different methods. The pricing update method sets lower prices than the other two methods. Then, the cost for interference is reduced, and the utility function of base station is increased, so the system performance can be improved.
For the purpose of maximizing system's throughput, base station will increase its throughput by continuously increasing its transmission power. Then, because the base station increases its transmission power, this will generate more interference. At the same time, due to the existence of interference, the base station will continue to increase its transmission power, which will lead to more interference. After the interference pricing of RNC is introduced, the base station not only needs to consider its throughput, but also the cost of interference needs to be considered. Combined with Figs. 4 and 5, base stations with lower price, its cost of the interference is less than others, its utility function is higher and it will obtain more power than other base stations, which is in line with the actual situation.
From the above simulation results, it can be seen that the goal of the game theory is to maximize the utility function, and each user can maximize their own utility function through the increase in transmission power. However, due to the interference caused by the increase in transmission power, the strategy of maximizing the utility function chosen by a single user will have an external effect on other users, which needs to be solved by setting the appropriate prices. Figure 6 shows that the algorithm has reached a stable state in the second iteration. The comparison of the system capacity in different scenarios is given when the number of base stations N is 5, 15 and 25, respectively. With the increase in the number of base stations, the total capacity of the system based on the above power allocation algorithm will increase. It shows that the system can accommodate as many users as possible with limited resources.

Results and discussion
This paper proposed a novel power allocation algorithm based on hierarchical game in 5G heterogeneous integrated networks. A three-layer power game model is proposed among the RNC, base stations and users, and the hierarchical game is used to solve the power allocation problem. Firstly, the stackelberg game is formulated between the first and second layers, in which the radio network controller sets the interference prices for the base stations. Then, the users in the third layer can update their transmission power by non-cooperative game based on the setting interference price. Finally, the RNC can update the price according to the obtained power and calculate the optimal price for base station, and the users power allocation can reach the Nash equilibrium based on non-cooperative game and obtain the optimal transmission power. The convergence of this algorithm is also analyzed and the simulation results show that the proposed algorithm can decrease the interference and improve the system capacity. Fig. 6 Comparison of system capacity. We set the number of base stations to 5, 15 and 25, respectively. With the increase in the number of base stations, the total capacity of the system based on the proposed power allocation algorithm will increase