# Adaptive mechanism design and game theoretic analysis of auction-driven dynamic spectrum access in cognitive radio networks

- Wei Zhong
^{1}Email author, - Youyun Xu
^{1}, - Jiaheng Wang
^{2}, - Dapeng Li
^{3}and - Huaglory Tianfield
^{4}

**2014**:44

https://doi.org/10.1186/1687-1499-2014-44

© Zhong et al.; licensee Springer. 2014

**Received: **8 March 2013

**Accepted: **19 January 2014

**Published: **24 March 2014

## Abstract

This paper studies the auction-driven dynamic spectrum access in cognitive radio networks with heterogeneous secondary users, who have different risk attitudes. First, a game theoretic framework is established for auction-driven dynamic spectrum access in cognitive radio networks. The utility functions and bidding strategies of heterogeneous secondary users are defined, and the parameterized auction mechanisms of primary user are also introduced. Then, we formulate the auction-driven dynamic spectrum access problem as a finite discrete game with a mixed- or pure-strategy Nash equilibrium solution. We study the existence and uniqueness properties of the pure-strategy Nash equilibrium in the defined game. Next, we propose a distributed learning automata algorithm (DLA) to attain the Nash equilibrium of the defined game with limited feedback. The adaptive mechanism design is realized in the updating procedure of our DLA algorithm. We further prove that our DLA algorithm converges to a Nash equilibrium of the defined game. Finally, simulation results show that our DLA algorithm is efficient and outperforms the dynamic spectrum access schemes with fixed auction mechanism.

## 1 Introduction

Cognitive radio aims to improve the utilization of radio electromagnetic spectrum, which is scarce but often underutilized [1]. To realize efficient spectrum utilization, various spectrum management techniques have been proposed for cognitive radio networks, and the auction-based spectrum management has recently attracted wide attention [2, 3].

Existing work mainly focused on using game theory to analyze the behaviors of the secondary users under a certain auction mechanism. Furthermore, the secondary users are usually assumed to be symmetric and homogeneous. Symmetry and homogeneity of secondary users are, however, generally not realistic in practical cognitive radio networks. In practice, secondary users usually have different risk attitudes in the auction, e.g., some secondary users are with high channel gain or urgent message and thus may tend to be *risk-seeking*, some secondary users are with non-urgent messages or low channel gain and thus may tend to be *risk-averse*, and the others may be regarded as *risk-neutral* buyers between *risk-seeking* and *risk-averse* buyers [4, 5].

Byde, 2003 [4] and Maskin and Riley, 1984 [6] have studied the auction with asymmetric bidders and have shown that different auction mechanisms would result in different outcomes. In other words, a fixed auction mechanism may not be optimal for the auctioneer in all circumstances [7, 8]. In order to obtain higher revenue, auction mechanism should be adaptively designed by the auctioneer according to the types of the secondary users. Therefore, for the cognitive radio networks with heterogeneous (multi-type) secondary users, it is of profound importance to study the auction-driven dynamic spectrum access with adaptive auction mechanism design.

Mechanism design [7] is applied to dynamic spectrum access in cognitive radio networks [9]. A generic parameterized auction mechanism design is presented in [9], but only two specific mechanisms are considered to suppress cheating and collusive behavior of selfish users, which contains a class of mechanisms for heterogeneous buyers. In the adaptive auction mechanism designs in [4, 10, 11], the risk attitudes of the buyers are not taken into account. Whereas in [4] evolutionary game theory is applied to study the auction mechanism design, in our proposed scheme, distributed learning automata are applied to auction mechanism design in cognitive radio networks and the dynamic spectrum access is realized jointly for secondary users. It is shown in [12] that replicator dynamics based on evolutionary game theory and the learning automata [13] are quite similar and are actually equivalent in some circumstances. However, learning automata are computationally simple and efficient and thus are more appropriate in designing practical distributed algorithm with limited information. Teng et al., 2011 [14] presents a dynamic spectrum sharing through a double auction mechanism, and [15] designs a truthful online spectrum auction for efficient spectrum allocation. However, neither auction model has considered heterogeneous secondary users.

Shen et al., 2011 [16] studies dynamic spectrum auction in wireless networks based on coexistent matrix, and [17] studies network pricing of private wireless access points (AP) through a location-dependent multi-AP reverse auction. Wang et al., 2012 [18] designs an efficient scheme of spectrum auction game for near-optimal spectrum allocation. In [19], an auction-based game theoretic approach is used for relay power allocation in wireless networks. However, in all these auction models, neither heterogeneous buyers with different risk attitudes nor adaptive auction mechanism design has been considered.

In this paper, we study cognitive radio networks with multiple secondary users that have different risk attitudes (i.e., heterogeneous buyers). The secondary users compete to access an idle channel under an auction simultaneously. Moreover, the primary user, i.e., the auctioneer, is able to apply multiple sealed-bid auction mechanisms that are adaptively designed. In practice, users would not have exactly full information of the network. Therefore, the aim of this paper is to address the following questions: *how the auctioneer should design the auction mechanism adaptively to maximize the utility and how the secondary users should choose their best bidding strategies with limited or local information.*

For this purpose, we formulate the auction-driven spectrum access problem as a discrete game and define appropriate utility functions for both the secondary users and the primary user. We then study the existence and uniqueness properties of the Nash equilibrium solution of the defined game. Given that the primary user and the secondary users would not have full information on cognitive radio networks, centralized algorithm is impractical. Therefore, we propose a *practical distributed learning automata* algorithm to attain the Nash equilibrium *with only limited feedback information*.

The remainder of the paper is organized as follows. In Section 2, the system model of cognitive radio network is presented. In Section 3, a game theoretic framework is established for auction-driven dynamic spectrum access in cognitive radio networks. Specifically, bidding strategies and utility functions are defined for the secondary users, and a generic parameterized auction mechanism and the utility function are defined for the primary user. Then, the auction-driven dynamic spectrum access in cognitive radio networks is formulated as a finite discrete game. In Section 4, a distributed learning automata (DLA) algorithm is proposed for evolving the defined mixed-strategy game to attain the Nash equilibrium solution to the auction-driven dynamic spectrum access. The adaptive mechanism design is realized in the updating procedure of our DLA algorithm, and the convergence of our DLA algorithm is proved. In Section 5, simulation results are presented for auction-driven dynamic spectrum access in cognitive radio networks. Finally, conclusions are drawn in Section 6.

## 2 System model of cognitive radio networks

*n*different secondary users simultaneously sensing the idle spectrum (i.e., channel) of the primary user. From the resource market point of view, the channel are the goods to be auctioned, the primary user is the

*seller*or

*auctioneer*, and the secondary users are the

*buyers*or

*bidders*who are competing to win the goods. For the sake of fairness and maximum revenue, the primary user shall adaptively design an auction mechanism, under which the secondary users compete for the channel. We assume that the secondary users choose their own bidding strategies after evaluating the channel conditions and other necessary information fed back from the primary user. The primary user has multiple, say

*M*, auction mechanisms to apply. The secondary users employ the same set of bidding strategies.

We assume that in each fixed duration, the primary user would organize the auction for the secondary users to compete under and announces the available channels at the beginning of every auction. Suppose that each channel allows only one secondary user to access at an instance. Whenever a secondary user wins the bid, a certain portion, proportional to its bid, would be registered. After the data transmission time, the secondary user who has successfully accessed the channel shall pay the price at the end of the frame.

The channel gain between secondary user *i* and the primary user is *h*_{
i
}. $\forall i\in \mathcal{N}$, *h*_{
i
} is assumed to be invariant over duration *T*, where $\mathcal{N}=\{1,2,\cdots \phantom{\rule{0.3em}{0ex}},n\}$ is the index set of the secondary users. We should point it out that different channel models will not affect the theoretic analysis and results of this paper.

**Notations**

Symbol | Definition |
---|---|

| Total transmission time of the frame |

| Channel coefficient of secondary user |

The set of secondary users | |

| The bidding strategy of secondary user |

| The |

| The set of bidding strategy of secondary user |

| The cardinality of |

| The utility reward to secondary user |

| The monetary efficiency function of secondary user |

| The channel capacity of secondary user |

| The transmit power of secondary user |

| The risk preference parameter of secondary user |

${\widehat{b}}_{i}$ | The bidding strategy of winner |

| The sensitive parameter of secondary user |

| The utility function of secondary user |

| The ordered auction price parameter of secondary user |

| The index of the winning secondary users |

bid | The ordered ( |

| The set of parameterized auction mechanisms |

| The cardinality of |

${\omega}_{2}^{m}$ | The |

| The utility function of base station |

| The probability of the |

| The probability distribution over |

| The probability of |

| The probability distribution over |

| The expected utility of secondary user |

$\stackrel{\u0304}{r}$ | The expected utility of the primary user |

| The reaction of the primary user |

${\gamma}_{{i}^{\ast}}^{s}\left(t\right)$ | The reaction of |

| The normalized parameter of the primary user |

| The normalized parameter of the secondary users |

| The stepsize of the primary user |

| The stepsize of the secondary users |

| The distance from the primary user to secondary user |

## 3 Strategy game formulation of auction-driven dynamic spectrum access

### 3.1 Bidding strategies and utility functions of secondary users

Let *b*_{
i
} denote the bidding strategy of secondary user *i* and *b*_{
i
}∈*B*_{
i
}. ${B}_{i}=\{{b}_{i1},\cdots \phantom{\rule{0.3em}{0ex}},{b}_{\mathit{\text{ik}}},\cdots \phantom{\rule{0.3em}{0ex}},{b}_{{\mathit{\text{iK}}}_{i}}\}$ is an ordered discrete finite set of bidding strategies of secondary user *i*, where *K*_{
i
} is the cardinality of *B*_{
i
}, *K*_{
i
}≥2, *b*_{
i
k
} is the *k* th bidding strategy in *B*_{
i
}, and ${b}_{i1}<\cdots <{b}_{{\mathit{\text{iK}}}_{i}}$. For each secondary user *i*, *b*_{i 1} represents the null bid and without loss of generality, is simply normalized to be zero; *b*_{i 2} denotes the lowest non-zero admissible bid, i.e., the reserve price, and it is assumed that *b*_{i 2}≥1. For the sake of simplicity, assume that $\forall i,j\in \mathcal{N}$, *b*_{i 2}=*b*_{j 2}. Moreover, the auction should be replayed if all secondary users choose the null bid.

*i*who wins the channel at bid

*b*

_{ i }is a discrete function, denoted as

*u*

_{ i }(

*v*

_{ i }(

*C*

_{ i },

*b*

_{ i })), where

*v*

_{ i }is defined as an increasing function of the channel capacity (

*C*

_{ i }) of secondary user

*i*, but a decreasing function of

*b*

_{ i }.

*u*

_{ i }(

*v*

_{ i }) reflects the value of the goods (i.e., channel) to secondary user

*i*.

*C*

_{ i }is defined as

where *ρ*_{
i
} is the transmit power of secondary user *i*, *W* is the bandwidth of the channel, *σ*^{2} is the mean channel noise power, and SNR denotes signal-to-noise ratio.

*risk-seeking*bidders [4]. However, secondary users with non-real-time services, or low SNR, or non-urgent messages, may not be so aggressive and thus are

*risk-averse*and in other cases, secondary users are

*risk-neural*bidders [4]. The characterization of the heterogeneous secondary users and the risk attitude is described in Table 2.

**Characterization of heterogeneous secondary users**

β
| Risk attitudes | Types of secondary users |
---|---|---|

| Risk-seeking | High channel gain, real-time services, |

urgent messages | ||

| Risk-averse | Low channel gain, non-real-time services, |

non-urgent messages | ||

| Risk-neutral | Modest channel gain, non-urgent messages |

*E*(

*u*

_{ i }(

*v*

_{ i })) of secondary user

*i*is larger (or smaller) than the utility

*u*

_{ i }(

*E*(

*v*

_{ i })), then secondary user

*i*prefers not to or prefers to take the risk presented by the uncertainty, and thus, secondary user

*i*is risk-averse (or risk-seeking). According to microeconomics such as textbook [7], utility functions of the three types of risk attitudes should follow the curves as illustrated conceptually in Figure 2.

*u*

_{ i },

*i*=1,2,⋯,

*n*. In our work,

*u*

_{ i }is defined as

where *β*_{
i
} is the *risk attitude parameter* of secondary user *i*, ${\widehat{b}}_{i}$ is the price that the winning secondary user *i* should pay to the primary user, and *θ*_{
i
}>0 is the parameter controlling the sensitivity of the channel quality for secondary user *i* and keeping the term of *C*_{
i
} consistent with the unit of *b*_{
i
}.

As shown in Figure 2, when *β*_{
i
}>0, it can be seen that the expected utility *E*(*u*_{
i
}(*v*_{
i
})) of secondary user *i* is smaller than its real utility *u*_{
i
}(*E*(*v*_{
i
})), which means that secondary user *i* tends to take the risk presented by the uncertainty; so, it is risk-seeking. When *β*_{
i
}<0, the expected utility *E*(*u*_{
i
}(*v*_{
i
})) of secondary user *i* is smaller than its real utility *u*_{
i
}(*E*(*v*_{
i
})), which means that secondary user *i* tends not to take the risk presented by the uncertainty; so, it is risk-averse. When *β*_{
i
}=0, the expected utility *E*(*u*_{
i
}(*v*_{
i
})) of secondary user *i* is equal to its real utility *u*_{
i
}(*E*(*v*_{
i
})), which means that it is risk-neutral.

In practice, the value of *β*_{
i
} depends on the type of secondary user *i*. In our work, as shown in Table 2, *β*_{
i
}>0 corresponds to a risk-seeking secondary user *i* who has urgent message or real-time service and is rich, *β*_{
i
}<0 to a risk-averse secondary user *i* who has neither urgent message nor real-time service and is poor of money, and *β*_{
i
}=0 to a risk-neutral secondary user *i* who has no urgent message and is of middle class.

Moreover, we assume that if all secondary users choose the same bid, no secondary users are allowed to access the channel and the auction should be replayed. In such a situation, the utility of each secondary user and the revenue of the primary user are equal to zero.

where *u*_{
i
} is given in (2).

Note that this is a generic model of the heterogeneous secondary users. All secondary users being of the same type can be regarded as a special case of our proposed model.

### 3.2 Auction mechanism design and utility function of primary user

There are numerous kinds of auction mechanisms, such as progressive auctions, Dutch or British auctions, sealed-bid auctions, and discriminatory auctions. In our work, we assume that the auction mechanisms are parameterized into a finite discrete set.

Let’s take sealed-bid auction as an example. In a seal-bid auction, a goods is put up for sale, and each potential buyer submits a bid to the auctioneer; the auctioneer chooses a winner and receives the payment from the winning bidder. In most variants of sealed-bid auction, the goods is awarded to the buyer who submits the highest bid, and only the winner pays. There are two standard sealed-bid auctions. In first-price auction, the winner’s payment is its bid; in second-price auction, the winner’s payment is the second highest bid. In this paper, instead of relying on a particular auction, we parameterize a class of auction mechanisms including both standard sealed-bid auctions and non-standard ones.

In [4], a concept of vector-price auction is designed as follows.

**Definition** **1**.

*ω*-price auction is a sealed-bid auction in which the highest bidder, denoted as

*i*

^{∗}, wins the goods and pays a calculated price as follows

where *n* is the minimum between $\widehat{n}$ and the number of bidders, and bid_{1}, bid_{2},… are the bids ordered from the highest to the lowest.

By using a special two-dimensional sub-space of *ω*-price auctions, namely *ω*=(1−*ω*_{2},*ω*_{2}) with 0≤*ω*_{2}≤1, we can already characterize a class of auction mechanisms. In particular, *ω*_{2}=0 corresponds to a standard first-price auction, and *ω*_{2}=1 corresponds to a standard second-price auction; all other values of *ω*_{2} correspond to non-standard auctions. In this paper, we assume that the value of *ω*_{2} is chosen from a finite set *Ω*. Let *M* be the cardinality of *Ω*. Then, there are *M* mechanisms available for the primary user to apply. Thus, the auctioneer can adjust parameters in response to the results observed from the previous auctions.

The primary user plays the role of an auctioneer, which involves organizing the auction process and designing an auction mechanism out of all different kinds of auction mechanisms that is able to achieve the maximum expected utility. Denote $\Omega =\left\{{\omega}_{2}^{1},\cdots \phantom{\rule{0.3em}{0ex}},{\omega}_{2}^{m},\cdots \phantom{\rule{0.3em}{0ex}},{\omega}_{2}^{M}\right\}$, where ${\omega}_{2}^{m}$ is the *m* th element in *Ω*. Then, whenever ${\omega}_{2}^{m}$ is given, the auction mechanism is actually specified. In other words, $\forall m\in \{1,\cdots \phantom{\rule{0.3em}{0ex}},M\},{\omega}_{2}^{m}$ corresponds to $\left(1-{\omega}_{2}^{m},{\omega}_{2}^{m}\right)$-price sealed-bid auction. Then, choosing an auction mechanism becomes simply choosing ${\omega}_{2}^{m}$ from *Ω*.

*r*denote the utility function (i.e., the revenue) of the primary user, then

where *i*^{∗} denotes the secondary user who wins the auction. For example, if the primary user designs (0.8,0.2)-price sealed-bid auction mechanism, the bid of the winner is bid_{1}, and the second highest bid is bid_{2}; the utility function of the primary user is then given as 0.8∗bid_{1}+0.2∗bid_{2}.

### 3.3 Game theoretic formulation of auction-driven dynamic spectrum access

#### 3.3.1 Game theoretic setting

*players*; the auction mechanism and the bidding strategies constitute the

*pure strategies*of the players. The utilities of the secondary users and the utility of the primary user constitute the

*payoffs*of the players. Then, the interaction between the primary user and the secondary users follows the framework of a finite discrete game, which can be defined as follows:

#### 3.3.2 Analysis of pure-strategy Nash Equilibrium

It is known from Nash [7] that every finite game in a strategic form has a Nash equilibrium in mixed strategy. Thus, our defined game must have at least one Nash equilibrium in a mixed strategy. Generally speaking, it is difficult to ascertain how many Nash equilibriums has and whether the Nash equilibriums are in mixed strategy or pure strategy. However, in some cases, we can analytically characterize the existence and uniqueness of the Nash equilibrium of .

##### Theorem 1

If there is only one secondary user in the network $(\text{i.e.,}\mathcal{N}=\{1\left\}\right),\mathcal{G}$ possesses a pure-strategy Nash equilibrium.

##### Proof

*b*

_{12}) will give the secondary user a larger payoff than any other bidding strategies. Therefore, the reserve price is the dominant strategy of the secondary user. Meanwhile, no matter what bidding strategy the secondary user is choosing, all auction mechanisms will give the primary user the same revenue. Consequently, we have

According to the definition of pure-strategy Nash equilibrium [7], we can draw the conclusion that a weak pure-strategy Nash equilibrium exists in .

In this case, we have the following theorem.

##### Theorem 2

If ${B}_{i}={B}_{j},\forall i,j\in \mathcal{N}$, then$\phantom{\rule{0.1em}{0ex}}\u011c$ is an ordinarily symmetric game^{a}. In addition, if ${K}_{i}={K}_{j}=2,\forall i,j\in \mathcal{N}$, then$\phantom{\rule{0.1em}{0ex}}\u011c$ possesses a pure-strategy Nash equilibrium.

*Proof*.

If ${B}_{i}={B}_{j},\forall i,j\in \mathcal{N}$, then with the utility function defined in (4) according to the definition of the ordinarily symmetric games [20, 21], for any permutation *π*, *U*_{
i
}(*b*_{1},⋯,*b*_{
i
},⋯,*b*_{
n
}) and *U*_{π(i)}(*b*_{π(1)},⋯,*b*_{π(i)},⋯,*b*_{π(n)}) must have the same ordinal rank of the payoffs. Furthermore, from [21], it says that every two-strategy symmetric game has a pure-strategy (not necessarily symmetric, though) Nash equilibrium. Hence, the theorem is proved.

Since the pure-strategy Nash equilibrium of does not always exist in general, we should study the defined game in a mixed strategy. In a mixed-strategy form of the defined game , the expectation utility function is used as the metric of the satisfaction of each secondary user or the primary user. In the following section, we will investigate this issue by using learning automata.

## 4 Distributed learning automata algorithm for evolving the mixed-strategy game

### 4.1 Mixed-strategy game setting

In practice, the primary user usually has no prior information of the secondary users and even has no knowledge of how many secondary users are in a cognitive radio network; therefore, it may incur a large amount of overheads for the primary user to obtain the perfect information about all secondary users. Here, ‘perfect’ means complete classes of the information, e.g., risk attitude, valuations, and strategy spaces, and also means no information should have error. Likewise, the secondary users in practice are unable to have the perfect information, and they even have no knowledge of the auction mechanisms the primary user may apply. Thus, it is impractical to employ a centralized decision making process to attain the equilibrium of the game.

Based on the concept of learning automata, we propose a distributed stochastic learning algorithm to adaptively update the bidding strategies of the secondary users and the auction mechanism of the primary user in a non-cooperative manner. Stochastic learning technique has been used for price dynamics [22], discrete power control [23], multi-mode precoding strategy selection [24], and so on. Our proposed distributed algorithm involves only limited amount of feedback and is computationally simple and efficient.

*p*

_{ i k }denotes the probability of secondary user

*i*choosing bid

*b*

_{ i k }, and ${\sum}_{k=1}^{{K}_{i}}{p}_{\mathit{\text{ik}}}=1$. Let ${\mathbf{p}}_{i}=\left({p}_{i1},\cdots \phantom{\rule{0.3em}{0ex}},{p}_{\mathit{\text{ik}}},\cdots \phantom{\rule{0.3em}{0ex}},{p}_{{\mathit{\text{iK}}}_{i}}\right)$ be the probability distribution over

*B*

_{ i }. Furthermore, let

**q**=(

*q*

_{1},⋯,

*q*

_{ m }, ⋯,

*q*

_{ M }) be the probability distribution over

*Ω*, where

*q*

_{ m }denotes the probability of the primary user designing the

*m*th auction mechanism (i.e., ${\omega}_{2}^{m}$), and ${\sum}_{m=1}^{M}{q}_{m}=1$.

**p**

_{ i }and

**q**are also referred to the

*mixed strategies*of the secondary users and the primary user, respectively. Then, the expected utility

*g*

^{ i }for secondary user

*i*is given as follows:

### 4.2 Distributed learning automata algorithm for mixed-strategy game

From the learning automata point of view, the secondary users and the primary user are the automata; the bidding strategies of the secondary users and the mechanism design strategies of the primary user are the actions of the automata. **q**(*t*) is the action probability distribution of the primary user at instant *t*, and **p**_{
i
}(*t*) is the action probability distribution of the *i* th secondary user at instant *t*. Furthermore, the normalized utilities of the secondary users and the normalized revenue of the primary user are the reactions of the automata.

Then, we can propose a distributed learning automata (DLA) algorithm based on learning automata. In our proposed DLA algorithm, the primary user learns probability distribution **q** over the set of auction mechanisms (i.e., *Ω*) so that it can maximize its expected revenue based on imperfect information and the secondary users learn probability distributions ${\mathbf{p}}_{i},\forall i\in \mathcal{N}$ over the set of bidding strategies in order to maximize their own utilities.

**Distributed Learning Automata Algorithm (DLA).**

In Algorithm 1, at instant *t*, ${\gamma}_{i}^{s}\left(t\right)$ is defined as ${\gamma}_{i}^{s}\left(t\right)={\lambda}^{s}{\xdb}_{i}\left(t\right)$, where ${\xdb}_{i}\left(t\right)={U}_{i}\left(t\right)+\varphi $, *ϕ*≥0, and 0<*λ*^{
s
}<1, such that the value of ${\gamma}_{i}^{s}\left(t\right)$ lies within the interval [ 0,1). Moreover, at instant *t*, ${\gamma}_{i}^{b}\left(t\right)$ is defined as ${\gamma}_{i}^{b}\left(t\right)={\lambda}^{b}\widehat{r}\left(t\right)$, where ${\widehat{r}}_{i}\left(t\right)={r}_{i}\left(t\right)+\varphi $, *ϕ*≥0, 0<*λ*^{
b
}<1, such that the value of ${\gamma}_{i}^{b}\left(t\right)$ lies within the interval [ 0,1). Since secondary users usually have no prior knowledge of their utilities, we employ a *parameter self-adjusting scheme* which can be stated as follows. At instant *t*, if $\forall i\in \mathcal{N},{\xdb}_{i}\left(t\right)>\frac{1}{{\lambda}^{s}}$, we set ${\lambda}^{s}=\frac{1}{{\xdb}_{i}\left(t\right)+\tau}$, where *τ* is a positive scalar; otherwise, *λ*^{
s
} remains unchanged. Similarly, if $\widehat{r}\left(t\right)>\frac{1}{{\lambda}^{b}}$, we set ${\lambda}^{b}=\frac{1}{r\left(t\right)+\tau}$; otherwise, *λ*^{
b
} remains unchanged.

### 4.3 Convergence of the distributed learning automata algorithm

The updating procedure of Algorithm 1 is known as a linear reward-inaction(*L*_{R−I}) [13]. Its convergence can be set out by the following theorem.

#### Theorem 3

Algorithm 1 converges to a Nash equilibrium of
with sufficient small *δ*^{
b
} and *δ*^{
s
} for any initial points in the mixed strategies except the initial points in the pure strategies.

*Proof*.

*F*as

Then, according to theorem 3.3 in [13], the theorem is proved.

From **Theorem 1**, it is known that the convergence of the proposed algorithm is guaranteed. In addition, it is known from [13] that when a pure-strategy Nash equilibrium exists, it will be a locally asymptotically stable point of Algorithm 1.

## 5 Performance evaluation

*B*

_{1}=⋯=

*B*

_{ n }=

*B*, and

*K*

_{1}=⋯=

*K*

_{ n }=

*K*. Furthermore, it is assumed that the channel gains of the secondary users include path loss and fading effects. Mathematically, the channel gain of secondary user

*i*is

where const is a constant, *α* is the path loss component, ${h}_{i}^{\prime}\sim \mathbf{\text{CN}}(0,1)$, and *d*_{
i
} is the distance between secondary user *i* and the primary user. {*d*_{1}, *d*_{2},⋯,*d*_{
n
}} are assumed to be randomly distributed over interval $\left[\underline{d},\overline{d}\right]$. Furthermore, it is assumed that there are five secondary users in the cognitive radio network unless otherwise stated. They are randomly distributed over interval [20 m,15 0m], i.e., $\underline{d}=20$ m and $\overline{d}=150$ m. We set values as follows: $\forall i,j\in \mathcal{N}$, *B*_{
i
}=*B*_{
j
}={0,1,2,⋯,13}, *K*_{
i
}=*K*_{
j
}=*K*=14, *ρ*_{
i
}=*ρ*_{
j
}=0.01 Watts, const=0.097, *α*=4, *W*=10 Hz, *σ*^{2}=5∗10^{−14} Watts, *θ*_{1}=1.5,*θ*_{2}=0.7,*θ*_{3}=0.8,*θ*_{4}=1,*θ*_{5}=1, *β*_{1}=−0.1,*β*_{2}=0,*β*_{3}=0.1,*β*_{4}=−0.1, *β*_{5}=−0.3. The set of parameterized auction mechanism of the primary user *Ω*=(0,0.2,0.4,0.6,0.8,1), *M*=6, *δ*^{
s
}=*δ*^{
b
}=0.2, *τ*=10, and *ϕ*=300.

**q**for the primary user for a specific channel realization in Figure 4. For illustration, we only plot the evolution of the choice probabilities

**p**of bidding strategies for some selected secondary users, i.e.,

*p*

_{13},

*p*

_{26},

*p*

_{39},

*p*

_{49},

*p*

_{510}, in Figure 5 for a specific channel realization. From these two graphs, we can see that the proposed Algorithm 1 has good convergence, and the probabilities of the auction mechanisms converge faster. This is because the number of auction mechanisms is much smaller than the total number of the bidding strategies. And, we find that the proposed Algorithm 1 converges to a mixed-strategy Nash equilibrium in this simulation. Furthermore, since we assume that the duration of the auction is fixed, we should control the runtime of Algorithm 1 so that it will not be longer than the auction duration. In practice, the

*adaptive stepsize scheme*in [24] can be used to control the convergence speed of Algorithm 1 and limit the runtime of Algorithm 1.

It can be seen from Figures 6, 7, and 8 that our proposed Algorithm 1 always achieves the highest performance. Compared to RAM-RB, Algorithm 1 enables the primary user to receive higher average revenue (where the maximum improvement is about 43%). Furthermore, Algorithm 1 also enables the secondary users to achieve higher average sum utilities (where the maximum improvement is about 300%) and sum capacities (where the maximum improvement is about 32%) than the other schemes.

*β*

_{1}=1,

*β*

_{2}=0.5,

*β*

_{3}=1,

*β*

_{4}=1,

*β*

_{5}=0, and ${\theta}_{i}={\theta}_{j}=1,\forall i,j\in \mathcal{N}$. These two figures show that the proposed Algorithm 1 achieves higher average revenue than the other schemes, and the average sum rate of Algorithm 1 is the largest. Furthermore, we can see that the first-price mechanism fails to result in the highest revenue.

These results mean that the first-price auction or the second-price auction may not be optimal when the buyers have different risk attitudes. The results also indicate that our proposed algorithm can induce the secondary users to choose higher bids and achieve higher average sum utility.

Note that in Figures 9 and 10, the term *1st-price* denotes the result where the primary user uses the traditional first-price auction mechanism and the secondary users use our proposed algorithm to choose their bidding strategies; the term *2nd-price* denotes the result where the primary user uses the traditional second-price auction mechanism and the secondary users use our proposed algorithm to choose their bidding strategies. The term *ω* *-price* (e.g., 0.2-price) denotes the result where the primary user uses the non-standard *ω*-price (0.2-price) auction mechanism and the secondary users use our proposed algorithm to choose their bidding strategies.

**q**for the primary user and the secondary users in Figures 11 and 12 for

*n*=2 and in Figures 13 and 14 for

*n*=3 over a particular channel realization. Notice that Algorithm 1 converges in all cases. However, in Figures 11 and 12, Algorithm 1 converges to a pure-strategy Nash equilibrium, while in Figures 13 and 14, it converges to a mixed-strategy Nash equilibrium. The results in Figures 11, 12, 13 and 14 have verified that the mixed-strategy Nash equilibrium always exists in our defined game model, yet a pure-strategy Nash equilibrium may not.

*n*=4 secondary users in the cognitive radio network. They are randomly distributed over interval [20m,150m], i.e., $\underline{d}=20$ m and $\overline{d}=150$ m. We set the values as follows: $\forall i,j\in \mathcal{N}$,

*B*

_{ i }=

*B*

_{ j }={0,0.5,1,⋯,6.5},

*K*

_{ i }=

*K*

_{ j }=

*K*=14,

*ρ*

_{ i }=

*ρ*

_{ j }=0.05 Watts; const=0.097,

*α*=4,

*W*=1 Hz,

*σ*

^{2}=5∗10

^{−14}Watts,

*θ*

_{1}=1,

*θ*

_{2}=1,

*θ*

_{3}=1. The set of parameterized auction mechanisms of the primary user

*Ω*=(0,0.2,0.4,0.6,0.8,1),

*M*=6,

*δ*

^{ s }=

*δ*

^{ b }=0.2,

*τ*=10, and

*ϕ*=100.

In Figure 15, risk-averse means that *β*_{1}=−1,*β*_{2}=−1,*β*_{3}=−1,*β*_{4}=−1, risk-neutral means that *β*_{1}=0,*β*_{2}=0,*β*_{3}=0,*β*_{4}=0, risk-seeking means that *β*_{1}=1,*β*_{2}=1,*β*_{3}=1,*β*_{4}=1, and the term ‘arbitrary’ means that *β*_{1}=0,*β*_{2}=−1,*β*_{3}=0,*β*_{4}=1. It is found that if the secondary users are all risk-averse, then the revenue of the primary users will be the lowest. If the secondary users are all risk-neutral or risk-seeking, then the primary user will receive higher revenue; the secondary users of ‘arbitrary types’ will result in modest revenue for the primary user.

## 6 Conclusions

In this paper, we have established a game theoretic framework for auction-driven dynamical spectrum access in cognitive radio networks with heterogeneous secondary users. We have studied the Nash equilibrium of the defined game and have proved that a unique pure-strategy Nash equilibrium exists in some particular situations. Due to the dynamic environment in cognitive radio networks, neither the primary user nor the secondary users have full information of the system. Thus, centralized algorithm is impractical. To attain an equilibrium solution with only incomplete information in a distributed setting, we have proposed a distributed learning automata algorithm. We have proved that our proposed algorithm can converge to a Nash equilibrium of our defined game with proper stepsize parameters. The simulation results have shown that our proposed algorithm can enable the auctioneer to obtain much higher revenue compared to the use of a fixed auction mechanism. At the same time, the proposed algorithm can significantly improve both the average sum capacities and the average sum utilities of secondary users.

## Endnote

^{a} Ordinarily symmetric games are those that are symmetric with respect to the ordinal structure of the payoffs.

## Declarations

### Acknowledgements

Wei Zhong was supported by the Natural Science Foundation of China under Grant No. 61201218, by the Natural Science Foundation of Jiangsu Province under Grant No. BK2012056, and the Project funded by China Postdoctoral Science Foundation. Jiaheng Wang was supported by the 973 Program (2013CB336600), National Natural Science Foundation of China (61201174), Natural Science Foundation of Jiangsu (BK2012325), and the Fundamental Research Funds for the Central Universities. Dapeng Li was supported by National Science Foundation of China under grant 61201162, Basic Research Program of Jiangsu Province (NSF) (BK2012434), NSF of the Higher Education Institutions of Jiangsu Province (12KJB510020), New Teacher Fund for Doctor Station, and The Ministry of Education (20123223120001).

## Authors’ Affiliations

## References

- Mitola IJ: Cognitive radio for flexible mobile multimedia communications. Paper presented at the IEEE international workshop on mobile multimedia communications (MoMuC), San Diego, CA, USA, 15–17 Nov 1999, pp. 3–10Google Scholar
- Losifidis G, Koutsopoulos L: Challenges in auction theory driven spectrum management.
*IEEE Commun. Magz*2011, 49(8):128-135.View ArticleGoogle Scholar - Bulow J, Levin J, Milgrom P: Winning play in spectrum auctions. Tech. rep. (NBER, Cambridge, 2009). . Accessed March 2009 http://www.nber.org/papers/w14765 Tech. rep. (NBER, Cambridge, 2009). . Accessed March 2009
- Byde A: Applying evolutionary game theory to auction mechanism design. Paper presented at the ACM conference on electronic commerce, San Diego, CA, USA, 9–12 June 2003, pp. 192–198Google Scholar
- Myerson R: Optimal auction design.
*Math. Oper. Res*1981, 6: 58-73. 10.1287/moor.6.1.58MathSciNetView ArticleMATHGoogle Scholar - Maskin E, Riley J: Optimal auctions with risk averse buyers.
*Econometrica*1984, 52(6):1473-1518. 10.2307/1913516MathSciNetView ArticleMATHGoogle Scholar - Myerson R:
*Game Theory: Analysis of Conflict*. Cambridge: Harvard University Press; 1991.MATHGoogle Scholar - Levin J: An optimal auction for complements.
*Game. Econ. Behav*1997, 18: 176-192. 10.1006/game.1997.0534View ArticleMathSciNetMATHGoogle Scholar - Wang B, Wu Y, Ji Z, Liu KJR: Game theoretical mechanism design methods.
*IEEE Signal Process. Mag*2008, 25(6):74-84.View ArticleGoogle Scholar - Klos T, van Ahee G: Evolutionary dynamics for designing multiperiod auctions. Paper presented at the 7th international joint conference on autonomous agents and multiagent systems (AAMAS), Estoril, Portugal, 12–16, May 2008, pp. 1589–1592Google Scholar
- Balcan M, Blum A, Hartline JD, Mansour Y: Mechanism design via machine learning. Paper presented at the 46th annual IEEE symposium on foundations of computer science (FOCS 2005), Pittsburgh, PA, USA, 22–25 Oct, 2005, pp. 605–614View ArticleGoogle Scholar
- Tuyls K, Nowe A: Evolutionary game theory and multi-agent reinforcement learning.
*Knowl. Eng. Rev*2005, 20: 63-90. 10.1017/S026988890500041XView ArticleGoogle Scholar - Sastry P, Phansalkar VV, Thathacha M: Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information.
*IEEE Trans. Syst. Man Cybern*1994, 24: 769-777. 10.1109/21.293490View ArticleMathSciNetGoogle Scholar - Teng Y, Zhang Y, Dai C, Yang F, Song M: Dynamic spectrum sharing through double auction mechanism in cognitive radio networks. Paper presented at the 2011 IEEE wireless communications and networking conference (WCNC 2011), Cancun, Mexico, 28–31 March, 2011, pp. 90–95View ArticleGoogle Scholar
- Deek L, Zhou X, Almeroth K, Zheng H: To preempt or not: tackling bid and time-based cheating in online spectrum auctions. Paper presented at the 30th IEEE international conference on computer communications (IEEE INFOCOM 2011), Shanghai, China, 10–15 Apr 2011, pp. 2219–2227Google Scholar
- Shen B, Long C, Chen C, Guan X, Zhang Q: Dynamic spectrum auction based on coexistent matrix. Paper presented at the IEEE international conference on communications (ICC), Kyoto, Japan, 5–9 June 2011, pp. 1–5Google Scholar
- Li P, Xing K, Wang H, Fan P: Tussle between APs in a pricing game: a location-dependent multi-AP reverse auction. Paper presented at the IEEE international conference on communications (ICC), Kyoto, Japan, 5–9 June 2011, pp. 1–5Google Scholar
- Wang X, Sun G, Yin J, Wang Y, Tian X, Wang X: Near-optimal spectrum allocation for cognitive radios: a frequency-time auction perspective. Paper presented at the IEEE global communications conference (GLOBECOM), Anaheim, CA, USA, 3–7 Dec 2012, pp. 4921–4926Google Scholar
- Wu D, Cai Y, Zhou L, Rodrigues J: Relay power allocation in auction-based game approach. Paper presented at the IEEE global communications conference (GLOBECOM), Anaheim, CA, USA, 3–7 Dec 2012, pp. 1812–1817Google Scholar
- Brandt F, Fischer F, Holze M: Symmetries and efficient solvability in multi-player games. Paper presented at 17th international conference on game theory, Stony Brook, NY, USA, 10–14 July 2006Google Scholar
- Cheng SF, Reeves DM, Vorobeychik Y, Wellman MP: Notes on equilibria in symmetric games. Paper presented at the 6th international workshop On game theoretic and decision theoretic agents (GTDT), New York, USA, 19–23 July 2004, pp. 23–28Google Scholar
- Xing Y, Chandramouli R, Cordeiro C: Price dynamics in competitive agile spectrum access markets.
*IEEE J. Sel. Areas Commun*2007, 25(3):613-621.View ArticleGoogle Scholar - Xing Y, Chandramouli R: Stochastic learning solution for distributed discrete power control game in wireless data networks.
*IEEE/ACM Trans. Netw*2008, 4(16):932-944.View ArticleGoogle Scholar - Zhong W, Xu Y, Tao M, Cai Y: Game theoretic multimode precoding strategy selection for MIMO multiple access channels.
*IEEE Signal Process. Lett*2010, 17(6):563-566.View ArticleGoogle Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.