Skip to content

Advertisement

Open Access

Recent advances on artificial intelligence and learning techniques in cognitive radio networks

EURASIP Journal on Wireless Communications and Networking20152015:174

https://doi.org/10.1186/s13638-015-0381-7

Received: 1 September 2014

Accepted: 13 May 2015

Published: 19 June 2015

Abstract

Cognitive radios are expected to play a major role towards meeting the exploding traffic demand over wireless systems. A cognitive radio node senses the environment, analyzes the outdoor parameters, and then makes decisions for dynamic time-frequency-space resource allocation and management to improve the utilization of the radio spectrum. For efficient real-time process, the cognitive radio is usually combined with artificial intelligence and machine-learning techniques so that an adaptive and intelligent allocation is achieved. This paper firstly presents the cognitive radio networks, resources, objectives, constraints, and challenges. Then, it introduces artificial intelligence and machine-learning techniques and emphasizes the role of learning in cognitive radios. Then, a survey on the state-of-the-art of machine-learning techniques in cognitive radios is presented. The literature survey is organized based on different artificial intelligence techniques such as fuzzy logic, genetic algorithms, neural networks, game theory, reinforcement learning, support vector machine, case-based reasoning, entropy, Bayesian, Markov model, multi-agent systems, and artificial bee colony algorithm. This paper also discusses the cognitive radio implementation and the learning challenges foreseen in cognitive radio applications.

Keywords

Cognitive radioArtificial intelligenceAdaptive and flexible radio access techniques

1 Review

1.1 Introduction

According to Cisco Visual Networking Index, the global IP traffic will reach 168 exabytes per month by 2019 [1], and the number of devices will be three times the global population. In addition, the resources in terms of power and bandwidth are scarce. Therefore, novel solutions are needed to minimize energy consumption and optimize resource allocation. Cognitive radio (CR) was introduced by Joseph Mitola III and Gerald Q. Maguire in 1999 for a flexible spectrum access [2]. Basically, they defined cognitive radio as the integration of model-based reasoning with software radio technologies [3]. In 2005, Simon Haykin had given a review of the cognitive radio concept and had treated it as brain-empowered wireless communications [4]. Cognitive radio is a radio or system that senses the environment, analyzes its transmission parameters, and then makes decisions for dynamic time-frequency-space resource allocation and management to improve the utilization of the radio electromagnetic spectrum.

Generally, radio resource management aims at optimizing the utilization of various radio resources such that the performance of the radio system is improved. For instance, the authors in [5] proposed an optimal resource (power and bandwidth) allocation in cognitive radio networks (CRNs), specifically in the scenario of spectrum underlay, while taking into consideration the limitations of interference temperature limits. The optimization formulations provide optimal solutions for resources allocation at, sometimes, the detriment of global convergence, computation time, and complexity.

To reduce the complexity and achieve efficient real-time resource allocation, cognitive radio networks need to be equipped with learning and reasoning abilities. The cognitive engine needs to coordinate the actions of the CR by making use of machine-learning techniques. As defined by Haykin in [4], “cognitive radio is an intelligent wireless communication system that is aware of its environment and uses the methodology of understanding-by-building to learn from the environment and adapt to statistical variations in the input stimuli”. Therefore, a CR is expected to be intelligent and capable of learning from its experience by interacting with its RF environment. Accordingly, learning is an indispensable component of CR that can be provided using artificial intelligence and machine-learning techniques.

In this paper, we firstly present the cognitive radiosystem principle, its main resources, parameters, andobjectives. Then, we introduce the artificial intelligence techniques, the learning cycle, the role, and the importance of learning in cognitive radios. The paper then discusses a literature survey on the state-of-the-artachievements in cognitive radios that use learning techniques. Several surveys were conducted to study the application of learning techniques in cognitive radio tasks; however, they still lack some components of a comprehensive study on cognitive radio systems. For instance, the authors of [6] presented a brief survey on artificial intelligence techniques; however, their work’s focus was on CR application and testbed development and implementation. The authors in [7] presented a survey on different learning techniques such as reinforcement learning, game theory, neural networks, support vector machine, and Markov model. They also discussed their strengths, weaknesses, and the challenges in applying these techniques in CR tasks. In [8], the authors considered game theory, reinforcement learning, and reasoning approaches such as Bayesian networks, fuzzy logic, and case-based reasoning. In contrast to the literature, we present a comprehensive survey considering all the learning techniques that were used in cognitive networks. The survey is organized based on different artificial intelligence approaches including the following: (a) fuzzy logic, (b) genetic algorithms, (c) neural networks, (d) game theory, (e) reinforcement learning, (f) support vector machine, (g) case-based reasoning, (h) decision tree, (i) entropy, (j) Bayesian, (k) Markov model, (l) multi-agent systems, and (m) artificial bee colony algorithm.

The main contributions of this paper are as follows: (1) it provides a comprehensive study on learning approach and presents their application in CR networks, evaluation, strengths, complexity, limitations, and challenges; (2) this paper also presents different cognitive radio tasks, as well as the challenges that face cognitive radio implementations; (3) it evaluates the application of the learning techniques to cognitive radio tasks; and (4) categorizes learning approaches based on their implementations and their application in performing major cognitive radio tasks such as spectrum sensing and decision-making.

This paper is organized as follows: cognitive radio networks, resources, objectives, and challenges are presented in Section 1.2. Learning in cognitive radios is presented in Section 1.3. Artificial intelligence and its learning role are introduced in Section 1.3.1. The literature survey is presented in Section 1.3.2. Learning techniques challenges, strengths, weaknesses, and limitations are presented in Section 1.4.1. Discussion on applying learning techniques in cognitive radio networks is presented in Section 1.4.2. Finally, conclusions are drawn in Section 2.

1.2 Cognitive radio

Cognitive radio provides the radio system an intelligence to maintain a highly reliable communication with efficient utilization of the radio spectrum. In this section, we present the cognitive radio cycle, tasks, and corresponding challenges.

1.2.1 1.2.1 The cognitive cycle

As shown in Fig. 1, the wireless communications system is formed by base stations and radio networks where some are primary users (PUs) or networks that own the spectrum and others are secondary users (SUs) that may use the spectrum when it is available and not occupied by other networks. As shown in Fig. 2, the cognitive radio network follows the cognitive cycle for best resource management and network performance. It starts by sensing the environment, analyzing the outdoor parameters, and then making decisions for dynamic resource allocation and management to improve the utilization of the radio electromagnetic spectrum [9]. These could be briefly described as follows.
  • Sensing the environment: In cognitive radio networks, the primary network has the priority to use the spectrum than the secondary network. The secondary network may use the available spectrum but without causing harmful interference to the primary network. Therefore, it needs to primarily quantify and sense its surrounding environment parameters such as (1) channel characteristics between base station and users; (2) availability of spectrum and power; (3) availability of spectrum holes points in frequency, time, and space; (4) user and application requirements; (5) power consumption; and (6) local policies and other limitations [10].
    Fig. 1

    Wireless communication network formed by cognitive radio networks

    Fig. 2

    Learning process in cognitive radios

  • Analyzing the environment parameters: The sensed environment parameters will be used as inputs for resource management in all dimensions such as time, frequency, and space. The main resource allocation objectives in CR include but are also not limited to (1) minimizing the bit error rate, (2) minimizing the power consumption, (3) minimizing the interference, (4) maximizing the throughput, (5) improving the quality of service, (6) maximizing the spectrum efficiency, and (7) maximizing the user quality of experience. In practice, cognitive radio aims at satisfying multiple objectives; however, the combination of some objectives may create conflicting solutions such as minimizing the power consumption and bit error rate simultaneously. Therefore, tradeoff solutions are needed to guarantee a balance between the objective functions [11].

  • Making decisions for different decision variables: In order to achieve the objectives mentioned before, the CR network needs to make decisions concerning the following important variables: (1) power control, (2) frequency band allocation, (3) time slot allocation, (4) adaptive modulation and coding, (5) frame size, (6) symbol rate, (5) rate control, (6) antenna selection and parameters, (7) scheduling, (8) handover, (9) admission control, (10) congestion control, (11) load control, (12) routing plan, and (13) base station deployment [12]. The decision-making can be based on optimization algorithms; however, in order to reduce the complexity and achieve efficient real-time resource allocation, cognitive radio networks use machine learning and artificial intelligence.

1.2.2 1.2.2 Cognitive radio tasks and corresponding challenges

The major role of cognitive radio is to identify spectrum holes across multiple dimensions such as time, frequency, and space and accordingly adjust its transmission parameters such as modulation and coding, frequency and time slot allocation, power control, and antenna parameters. Therefore, the CR behaving as a secondary user needs to dynamically reconfigure itself to avoid any noticeable interference to the primary user by efficiently using its (1) cognitive capability, (2) reconfigurable capability, and (3) learning capability. However, these capabilities are subjected to many challenges presented as follows.

Task 1—The cognitive capability can be achieved using efficient situation awareness and spectrum-sensing techniques. It includes location and geographical awareness, RF environment, network topology, and operational knowledge. The main challenges facing spectrum-sensing techniques are the decision accuracy on spectrum availability, sensing duration, frequency and periodicity, uncertainty in background noise power especially at low SNR due to multi-path fading and shadowing, detection of spread spectrum primary signals, and sensing with limited information about the environment. To improve the spectrum-sensing performance, cooperative sensing and geo-location technology were proposed. First, cooperative detection showed collaborative communications gains; however, it is still facing many challenges such as overhead, developing efficient cooperation framework including information sharing algorithms and networks, and dynamic information exchange with minimum delay. Second, combining geo-location technology with spectrum sensing may reduce the complexity, power, and cost at the CR device. The CR will be using database look-up for location awareness as well as unused spectrum and channels. Geo-location technology also faces many challenges such as updating the databases, efficient look-up techniques and algorithms, accuracy, trust, and security of the databases [1315].

Task 2—After performing spectrum sensing and situation awareness, the CR network needs to use its reconfigurable capability to dynamically adjust operational and transmission parameters and policies to achieve the highest performance gain such as maximizing the utilization of the spectrum and throughput, reducing the energy and power consumption, and reducing the interference level while meeting users’ quality of service (QoS) requirements such as rate, bit error rate, and delay. The main reconfigurable parameters listed in Section 1.2 include for instance (1) power control, (2) frequency band allocation, (3) time slot allocation, (4) adaptive modulation and coding, (5) frame size, (6) symbol rate, (5) rate control, (6) antenna selection and parameters, (7) scheduling, (8) handover, (9) admission control, (10) congestion control, (11) load control, (12) routing plan, and (13) base station deployment. The reconfigurable capability is based on decision-making, which can be based on optimization algorithms. The main challenge here concerns the complexity and the convergence of these techniques within a limited time. In order to reduce the complexity and achieve efficient real-time resource allocation, cognitive radio networks use machine learning and artificial intelligence. This decision-making is based on models built using the CR learning capability, which is based on the environment information. However, the latter may not be complete or accurate due to limited training data. In addition, the decision-making procedure needs to be dynamic and fast [16]. Therefore, focusing future research contributions on any of these two aspects is needed as they are the bottleneck of the reconfiguration capability in CR networks.

Task 3—The learning capability is used to build and develop the learning model for decision-making. The main challenge here is to enable the devices to learn from past decisions and use this knowledge to improve their performance. Some learning techniques may require previous knowledge of the system, predefined rules, policies and architecture, and a large number of iterations which may increase the delay and reduce the efficiency of the system. Therefore, the choice of a learning technique for performing specific CR task is considered a challenge as well as the accuracy and efficiency of the techniques.

1.3 Learning in cognitive radio networks

Learning in cognitive radios has recently gained a lot of interest in the literature. In this section, artificial intelligence and machine learning are introduced as well as a survey of the state-of-the-art achievements in applying learning techniques in cognitive radio networks.

1.3.1 1.3.1 Introduction to artificial intelligence and machine learning

Artificial intelligence aims at making machines perform tasks in a manner similar to an expert. The intelligent machine will perceive its environment and take actions to maximize its own utility. The central problems in artificial intelligence include deduction, reasoning, problem solving, knowledge representation, and learning [17].

The major steps in machine learning in cognitive radios are shown in Fig. 2 and can be presented as follows: (1) sensing the radio frequency (RF) parameters such as channel quality, (2) observing the environment and analyzing its feedback such as ACK responses, (3) learning, (4) keeping the decisions and observations for updating the model and obtaining better accuracy in future decision-making, and finally (5) deciding on issues of resource management and adjusting the transmission errors accordingly [7, 18]. In [6], Zhao et al. introduced the concepts of cognitive radio from the perspectives of artificial intelligence and machine learning. Moreover, the authors presented the possible applications and fundamental ideas that drive the CR technology.

Artificial intelligence may be represented but not limited to the following learning techniques: fuzzy logic, genetic algorithms, neural networks, game theory, reinforcement learning, support vector machine, case-based reasoning, decision tree, entropy, Bayesian, Markov model, multi-agent systems, and artificial bee colony algorithm. However, the mentioned approaches are the main techniques used and applied in CR networks.

1.3.2 1.3.2 Applying artificial intelligence techniques to CRs

In this section, a survey of the state-of-the-art research considering learning in CRs is presented. They are grouped based on artificial intelligence and learning techniques for CR.

Fuzzy logic

The fuzzy set theory was proposed by Lotfi A. Zadeh in 1965 to solve and model uncertainty, ambiguity, imprecisions, and vagueness using mathematical and empirical models [19]. The variables in fuzzy logic are not limited to only two values (True or False) as it is defined in classical and crisp sets [8]. A fuzzy element has a degree of membership or compatibility with the set and its negation. Fuzzy logic provides the system with (1) approximate reasoning by taking fuzzy variables as an input and producing a decision by using sets of if-then rules, (2) decision-making capability under uncertainty by predicting consequences, (3) learning from old experience, and (4) generalization to adapt to the new situations [20, 21]. In general, the inputs for the fuzzy inference system (FIS) need to be fuzzified or categorized into levels or degrees such as low, medium, and high. FIS using if-then rules will allow determining the output of the system.

The authors in [2224] and [25] applied fuzzy logic theory in cognitive radio to solve the following objectives: bandwidth allocation, interference and power management, spectrum availability assessment methods, and resource allocation. In [22], the authors have proposed a centralized fuzzy inference system that can allocate the available bandwidth among cognitive users considering traffic intensity, type, and quality of service priority. The secondary users (SU) have to submit bandwidth requests to the master SU which uses fuzzy logic to grant the SU bandwidth access. First, the master SU assesses traffic intensity of the SU queue and of the bandwidth allocation queue to determine the allowed access latency for SUs. Second, combining the allowed access latency to the traffic type and priority, the master will be able to decide on the amount of bandwidth that may be allocated to the required SU. Depending on the combination of access latency and traffic priority, the bandwidth to be allocated is characterized as very high, high, medium, low, and very low.

In [23], Aryal et al. have presented an approach for power management while reducing interference and maintaining quality of service. Their algorithm considers the number of users, mobility, spectrum efficiency, and synchronization constraint. These inputs are categorized as low, moderate, high, and very high. Fuzzy rules are then used to determine the power adjustments as (1) remain unchanged, (2) slightly increase, (3) moderately increase, (4) highly increase, and (5) fully increase.

The authors in [24] used fuzzy logic to determine the proper method for detecting available bandwidth. Four input parameters are considered for the selection of the spectrum-sensing method: (1) required probability of detection, (2) operational signal-to-noise ratio (SNR), (3) available time for performing the detection, and (4) a priori information. The outputs are (1) energy detection, (2) correlation detection, (3) feature detection, (4) matched filtering, and (5) cooperative energy detection. The input parameters are first fuzzified from measurable values to fuzzy linguistic variables by using input membership functions such as low, medium, or high. Based on the if-then rules, the fuzzy values of the input parameters will then specify the method for spectrum availability detection.

Qin et al. in [25] proposed the use of fuzzy inference rules for resource management in a distributed heterogeneous wireless environment. The fuzzy convergence is designed in two levels. First, the local convergence calculation is based on local parameters such as interference power, bandwidth of a frequency band, and path loss index. Second, the local convergence calculations collected from all nodes will be aggregated to generate a global control for each node. The aggregation weights are identified using: the nodes control, the link state aggregation date, and the link states amount.

Genetic algorithms

Genetic algorithms (GA) are originated in the work of Friedberg (1958), who attempted to produce learning by mutating small FORTRAN programs. Therefore, by making an appropriate series of small mutations to a machine code program, one can generate a program with good performance for any particular simple task [18]. Genetic algorithms simply search in the space, with the goal of finding an element or solution that maximizes the fitness function by evolving a population of solutions, or chromosomes, towards better solutions. The chromosomes are represented as a string of binary digits. This string grows as more parameters are used by the system. The search is parallel instead of processing a single solution because each element can be seen as a separate search. A genetic algorithm-based engine can provide awareness-processing, decision-making, and learning for cognitive radios [26, 27].

The authors in [28] used genetic algorithms for enhancing the system performance by solving multi-objective problems that aim to minimize the bit error rate and power and maximize the throughput. The authors have encoded the operating variables for different numbers of subcarriers into a chromosome, including the center frequency, transmission power, and modulation type. They presented several approaches such as population adaptation, variable quantization, variable adaptation, and multi-objective genetic algorithms. Their results showed that the information from the past states of the environment from previous cognition cycles can be used to reduce the convergence time of the GA and that genetic algorithms can enhance the convergence of optimization problems.

In [29], the authors addressed spectrum optimization in CRs using elitism in genetic algorithms. They used four parameters for the chromosome structure representation: frequency, power, bit error rate, and modulation scheme. The solution has provided the most efficient performance for the user’s quality of service, subjected to different constraints on the genes in the chromosome structure. Elitism is used for the selection of the best chromosomes among the population to be transferred to the next generation before performing crossover and mutation. This prevents the loss of the most likely solutions in the available pool of solutions.

Hauris et al. used genetic algorithms in [30] for RF parameter optimization in CR. The genes used are modulation and coding schemes, antenna parameters, transmit and receive antenna gains, receiver noise figure, transmit power, data rate, coding gain, bandwidth, and frequency. The fitness measure is calculated from the following key performance parameters: link Margin, C/I, data rate, and spectral efficiency. The maximum fitness measure and its associated chromosomes are tracked and saved. The best member is then utilized as the optimal solution for setting the RF parameters.

The authors of [31] presented cognitive radio resource allocation based on a modified genetic algorithm named niche adaptive genetic algorithm (NAGA). NAGA solved the problem of fixed crossover and mutation probabilities and adaptively adjusted them to achieve optimal performance. The goal was to determine the assignment of the subcarrier and modulation schemes to the users, in order to maximize the total transfer speed of CR networks while satisfying minimum rate requirements, maximum allowed bit error rate, and total power consumption limits.

Neural networks

Neural networks were introduced by Warren McCulloch and Walter Pitts in 1943 and were inspired from the central nervous system. Similar to the biological neural network, the artificial neural network will be formed by nodes, also called neurons or processing elements, which are connected together to form a network. The artificial neural network gets information from all neighboring neurons and gives an output depending on its weight and activation functions. The adaptive weights may represent the connection strengths between neurons. To accomplish the learning process, the weights need to be adjusted until the output of the network is approximately equal to the desired output. Artificial neural networks have been used to make the cognitive radio learn from the environment and take decisions, in order to improve the quality of service of the communication system [32, 33].

In [34], Xuezhi Tan et al. addressed the problem of spectrum lack and inefficiency in the current communication networks by introducing a new solution using artificial neural networks (ANNs) to replace the current frequency allocation system. They presented theoretical analysis about two different scenarios: the single-user case scenario and the multi-user scenario with weighted allocation. They focused on the Back Propagation Theory mainly formed by the idea of the exchange of information going forward and error being transmitted backward.

The authors in [35] tried to improve the performance of spectrum sensing in CR networks based on a new ANN solution. The authors installed ANN at every secondary user to predict the sensing probabilities of these units. Their idea was to create a new cooperative spectrum-sensing system through the collaboration of the SUs equipped with ANN capabilities and a fusion center using the theory of the belief propagation network. The results showed a global low false-alarm probability for the CR network.

Yang et al. in [36] proposed a design of the cognitive engine based on genetic algorithms and radial basis function network (RBF) in order to adjust the parameters of the system so as to effectively adapt to the environment as it changes. They utilized a decision-making table to train the RBF learning model whereas they mainly made use of the GA to adjust the operating parameters of the RBF neural network such as the data rate, MAC window, and transmitting power.

The authors in [37] presented a general review of the main spectrum-sensing methods and then proposed their own automatic modulation classification detection method. The main idea of [37] is based on the fact that the secondary user is not supposed to have a priori information about the primary user’s signal type and not supposed to address the issue of the hidden node. The authors developed the digital classifier using an ANN that allows the user to detect all forms of primary radio signals whether weak, strong, pre-known, or unknown.

Game theory

The first known discussion of game theory occurred in a letter written by James Waldegrave in 1713. Game theory is used as a decision-making technique where several players must make choices and consequently affect the interests of other players. Each player decides on his actions based on the history of actions selected by other players in previous rounds of the game. In cognitive networks, the CR networks are the players in the game. The actions are setting the RF parameters such as transmit power and channel selection. These actions will be taken by CR networks based on observations represented by environment parameters such as channel availability, channel quality, and interference. Therefore, each CR network will learn from its past actions, observe the actions of the other CR networks, and modify its actions accordingly [38, 39].

Abdulghfoor et al. in [40] introduced the concepts of modeling resource allocation in ad hoc cognitive radio networks with game theory and compared between two scenarios related to the presence or absence of cooperation using game theory. They also outlined game theory applications in other layers and future directions for using game theory in CR. The authors showed that game theory can be used to design efficient distributed algorithm in ad hoc cognitive radio networks whereas its application in the MAC layer proved to be most challenging.

The authors in [41] addressed spectrum sensing in CR networks. They proposed a single framework for cooperative spectrum sensing of CR networks as well as for self-organization of femtocells by relying on the data generated by the CR users to help in a better understanding of the environment. They proposed the creation of a large spectrum database that relies on macrocell and femtocell networks. The game theory approach provides mutual benefits to the CR users, and each femtocell is considered as a player in the game of spectrum sensing and spectrum utilization. The results showed a lower false alarm probability and a tradeoff between the gain increase and the size of the coalition as the time to generate the reports has a negative effect on the matter.

In [42], Pandit et al. approached the problem of the fixed spectrum allocation policy from an economic point of view where they proposed a simulation model to improve the bandwidth allocation between the primary and secondary users of a CR network. The authors developed an algorithm that minimizes the cost of the bandwidth utilization while maximizing the effectiveness of the SUs in the CR network. The authors utilized game theory as a utility to model the payoffs between the SUs and PU, considered as players. In their approach, the PUs’ main aim was to maximize their revenue whereas the SUs’ aim was to improve their QoS satisfaction at an acceptable cost.

The authors in [43] addressed spectrum management in CR. They proposed spectrum trading—spectrum management without game theory (SMWG) and spectrum competition—spectrum management with game theory (SMG). Considering the SMWG first, the authors introduced a competition factor to model the spectrum competition between the different PUs. They also introduced a new QoS level function that relates to the spectrum availability and variable according to SU requirements. In both cases, they assumed that the tradeoff is between the PU’s desire to maximize revenue and the SU’s desire to obtain the desired QoS level. In the SMG model, they formulated two games by relying on a Bertrand game with the Nash equilibrium being the solution, and the other is using the Stackelberg game having the same solution.

Reinforcement learning

Reinforcement learning (RL) plays a key role in the multi-agent domain, as it allows the agents to discover the situation and take actions using trial and error to maximize the cumulative reward as illustrated in Fig. 3. The basic reinforcement learning model consists of (1) environment states, (2) actions, (3) rules for transition between states, (4) immediate reward of transition rules, and (5) agent observation rules. In RL, an agent needs to consider the immediate rewards and the consequences of its actions to maximize the long-term system performance [44, 45].
Fig. 3

The agent-environment interaction in reinforcement learning

In [46], Yau tried to incorporate RL to correctly complete the cognition cycle in centralized and static mobile networks. The RL approach was applied at the level of the SUs where they dynamically rank channels according to PU utilization and packet error rate during data transmission to increase throughput of SUs and QoS levels while reducing delays.

The authors in [47] considered routing in cognitive networks. They proposed a new RL system that jointly works on channel selection and routing for a multi-hop CR network. The RL incorporated a system of errors and rewards based on each decision, and hence, every agent tried to maximize its own rewards. After trial and error, the CR users of RL will reach an optimal state in their decision-making where they maximize their spectrum utilization performance. The authors also used the feedback obtained from the environment and modeled the problem using the Markov decision process. The key drivers of the channel selection can be represented by link cost and transmission time. This method allows the users to choose the best route available through continuous and efficient determination of the next hop.

Zhou et al. in [48] addressed the problem of high-power consumption generated due to overhead communication between different CR users. They proposed a new power control scheme relying on RL to eliminate the need for information sharing on interference and power strategies of each CR element. The CR users compete over repeated rounds to maximize their own objective while taking into consideration the interference constraints imposed by the network.

In [49], the authors used reinforcement learning on the secondary user node for spectrum sensing, PU signal detection, and transmission decisions in cognitive radio networks. The SU will learn the behavior of the PU transmissions to dynamically fill the spectrum holes.

Support vector machine

Support vector machines (SVM) are supervised learning models used for classification and regression analysis. In the learning phase, SVM uses the training data to come up with margins to separate classes as shown in Fig. 4. A new entry or object is then classified based on these margins and the compatibility or distance between the object and the class [50, 51].
Fig. 4

SVM classifier separates the training data into two classes by determining a linear line representing the greatest separation between both sides

In [52], the authors used SVM to add a learning design to the CR engine. The proposed model is based on bit error rate (BER), SNR, data rate, and modulation mode. For training data, three channel models are considered: flat fading model, deep fading model, and no fading model. Once the model is built, the data is tested taking the BER, SNR, and data rate as inputs.

Dandan et al. in [53] used SVM for spectrum sensing and real-time detection. The sample data was classified as a primary user or not by training and testing on the proposed SVM classification model. The classes are determined based on the received signal as follows: if the signal detected is formed by the signal and AWGN noise, the class will be denoted as PU detected. When the signal is only AWGN noise, the class indicates no PU. The parameters considered in this work included carrier, pulse sequence, repeatability extension, and circulation prefix processing.

The authors in [54] used SVM for medium access control (MAC) identification. The proposed SVM model enables CR users to sense and identify the MAC protocol types of the existing transmissions and to adapt their transmission parameters accordingly. The authors used three different kernel functions for SVM: linear, polynomial, and radial basis functions.

In [55], the authors proposed applying SVM to eigenvalue-based spectrum sensing for multi-antenna cognitive radios. They built their training model by observing N samples and generating their covariance matrix eigenvalues. The new data point is then classified based on the annotated training data set and SVM kernel to indicate the presence or absence of PU.

The authors in [56] used SVM to solve the problem of beam-forming design in cognitive networks. They considered a CR network with relaying capabilities where the cognitive base station shares the spectrum with the primary network and can act as a relay to assist the primary data transmission. First, they aimed to minimize the total transmit power of the cognitive base station, while maintaining QoS requirements and mutual interference level as acceptable. SVM was used to solve the optimization problem for the beam-forming weight vectors.

Case-based reasoning

Li D. Xu introduced the concept of case-based reasoning (CBR) in [57] which relies on past problems and solutions to solve current similar situations. CBR systems build an information database about past situations, problems, and their solutions and rewards as shown in Fig. 5. New problems are then solved by finding the most similar case in memory and inferring the solution to the current situation [58].
Fig. 5

Case-based reasoning concept illustration

Ken-Shin Huan et al. introduced a new space efficient and multi-objective CBR method in [59] to solve the high-storage space required by the traditional CBR methods. The authors considered all possible cases in relation to their objectives in order to develop their model as accurately as possible. Their method relied on the divide-and-conquer technique using unity functions.

Reddy in [60] designed an efficient spectrum allocation using CBR and collaborative filtering approaches. They used the case-base reasoning to identify a channel preferred by the secondary user. They then used automatic collaborative priority filtering to assign a channel to the highest prioritized user.

The authors in [61] used CBR for proper link management, network traffic balance, and system efficiency. Each case contains the problem, a solution, and its corresponding result to provide the CR with better information utilization on the input, previous decisions, and their consequences. They aimed to reduce access time by finding similarities between cases and bucketing them.

In [62], the authors used CBR quantum genetic algorithm to adjust and optimize CR parameters. Environment change factor was used to measure similarity between the current problem and the cases in the database and initialize quantum bits to avoid the blindness of initial population search and speed up the optimization of quantum genetic algorithm.

Decision tree

Decision tree learning uses a decision tree to create a model that predicts the value of a target class based on several input variables. A decision tree has a similar structure as that of a flow chart, where each node is an attribute, and the topmost node is the root node as shown in Fig. 6. Each branch represents the outcome of a test, and each leaf node holds a class label [63].
Fig. 6

Decision tree chart where each node is an attribute and each leaf node holds a class label

In [64], the authors used decision tree learning to find the optimal wideband spectrum-sensing order. The root node is the start point, and the leaf node is the channel selected at every stage. At every node, one strategy is selected based on certain rules to produce the corresponding child node and construct a branch. At the end leaves of the tree, the sensing order can be tracked backward to the root.

The authors in [65] used decision tree for cognitive routing so that the nodes can learn their environment and adapt their parameters and decisions accordingly. A sender will then use the decision tree to select the most appropriate and reliable next hop neighbor.

In [66], the authors used decision tree for video routing in CR mesh networks. They aimed to improve the peak signal-to-noise ratio (PSNR) of the received video by considering channel status, nodes supporting video frame quality of service, effects of spectrum stability, and bandwidth availability.

The authors in [67] addressed cooperative spectrum sensing using distributed detection theory. CRs cooperate to sense the spectrum and classify overlapping air interfaces. The authors proposed the decomposition of a M-ary subtest into a set of binary tests, represented in a decision tree, to make the classification process simpler.

Entropy

In 1948, Claude E. Shannon introduced entropy in his paper “A Mathematical Theory of Communication” [68]. Entropy is a measure of the uncertainty in a random variable. It is also defined as a logarithmic measure of the rate of transfer of information in a particular message.

Zhao in [69] proposed (1) an entropy detector based on spectrum power density that provides better detection with lower computational complexity and (2) a two-stage entropy detection scheme to improve the performance of the entropy detector based on spectrum power density at low SNR.

In [7072], entropy was used for spectrum sensing. The authors in [70] addressed spectrum sensing in a maritime environment where ships are far from land. The communication ship-to-ship/ship-to-shore and ship-to-ship ad hoc network in deep sea have been realized with the support of satellite communication links.

The authors in [71] aimed to increase the spectrum-sensing performance and reliability in a cooperative wideband-sensing environment. They applied entropy estimation in each subchannel with multiple suspicious-CR-user elimination. Soft decision fusion methods: weighted gain combining and equal gain combining were used to improve the reliability of cooperative sensing. In addition, the generalized extreme studentized deviate (GESD) test was used to detect outliers and eliminate suspicious cognitive users. The authors extended their work in [72]; they performed hardware-in-the-loop simulation of the developed algorithm in a field programmable gate array (FPGA). The cooperative wideband spectrum sensing was based on entropy estimation and exclusion of suspicious CR users using GESD test and sigma limit test.

Bayesian approach

Bayesian networks are graphic probabilistic models that rely on the interaction between the different nodes to achieve learning for and from every node involved in the process. The Bayesian networks (BNs) have a role in the decision-making process if combined with utilities in order to form influence diagrams [73, 74].

Yuqing Huang et al. in [75] proposed a CR learning interference and decision-making engine based on Bayesian networks. The authors made use of the junction tree algorithm to model interference using probabilistic models obtained from the BNs. The authors developed their CR model to adapt their radio parameters to ensure QoS of the users.

In [76], the authors addressed multi-channel sensing and access in distributed networks, with and without constraint on the number of channels that SUs are able to sense and access. They proposed a cooperative approach for estimating the channel state and used Bayesian learning to solve multi-channel sensing problem.

The authors in [77] presented a Bayesian approach for spectrum sensing to maximize the spectrum utilization in CRNs. They aimed to detect known-order multi-phase shift keying (MPSK)-modulated primary signals over AWGN channels based on Bayesian decision rule.

Zhou et al. in [78] proposed a cooperative spectrum-sensing scheme based on Bayesian reputation model in CRNs where malicious secondary users may occur. They suggested the use of SUs’ reputation degrees to reflect their service quality. These reputation degrees are updated based on Bayesian reputation model to distinguish the trustworthiness of the reports from SUs and track the behaviors of malicious SUs.

In [79], the authors exploited sparsity in cooperative spectrum sensing. Sparsity occurs as follows: (a) in frequency domain when primary users occupy a small part of the system bandwidth and (b) in space domain when the number of users is small and their locations occupy a small fraction of the area. In their proposed model, the authors used the theory of Bayesian hierarchical prior modeling in the framework of sparse Bayesian learning.

Markov model

The Markov model is used to model random process changing from one state to another over time. The random process is memoryless where future states depend only on the present state [80, 81]. In Markov models, the states are visible to the observer; however, in the hidden Markov model (HMM), some states are hidden or not explicitly visible [82].

The authors in [83] used HMM for blind source separation to identify spectrum holes. Using the hidden Markov model, a CR engine will predict the primary user’s next sensing frame.

In [84], Pham et al. addressed spectrum handoff in cognitive radio networks based on the hidden Markov model. Spectrum handoff occurs when a SU needs to switch to a new idle channel due to a continuous data transmission when a PU needs the current channel. Therefore, the secondary user needs to study the behavior of the primary user and predict its future behaviors to perform spectrum handoff and ensure a continuous transmission.

Li et al. in [85] used the Markov model for modeling and analyzing the competitive spectrum access among cognitive radio networks. The cognitive network is formed by multiple dissimilar SUs and channels. They proposed decomposing the complex Markov model into a bank of separated Markov chains for each user. They focused on evaluating the SU throughput under uncertain spectrum access strategies.

The authors in [86] used the Markov decision process for dynamic spectrum access in cognitive networks. They used the HMM to model a wireless channel and predict the channel state. They decided on spectrum sensing, channel selection, modulation and coding schemes, transmitting power, and link layer frame size to minimize energy consumption.

Multi-agent systems

Jacques Ferber introduced multi-agent systems (MAS) as a smart entity aware of their surroundings, capable of skillfully acting and communicating independently. MAS contain the environment, objects, agents, and the different relations between these entities. MAS have their applications found mainly in problem solving and in the creation of a virtual world [87, 88].

In [89], Emna Trigui et al. introduced a novel approach to address the spectrum handoff within the CR domain. Their approach allows CR terminals to always switch to the spectrum band that offers the best conditions by using multi-agent system negotiation. They considered the mobile CR terminal and primary users as agents when they negotiate on prices and bandwidth trying to maximize their own profits.

The authors in [90] addressed the issue of real-time CR resource management by relying on multi-agent systems. They considered the scenario of a user stepping into a zone with bad QoS. The authors used several learning algorithms such as K-NN and decision trees in order to classify data.

In [91], the author addressed the issue of resource management and proposed a negotiation model to reduce the overhead present by relying on a novel spectrum access scheme that eliminates negotiation. The author relied on game and multi-agent Q-learning in order to create his model.

Mir et al. in [92] used a multi-agent system for dynamic spectrum sharing in cognitive radio networks. They proposed a cognitive radio network where agents are deployed over each primary and secondary user device. Accordingly, when the SU needs spectrum, its agent will cooperate and communicate with PU agents for spectrum sharing.

Artificial bee colony

The artificial bee colony (ABC) concept was introduced by Dervis Karaboga in 2005, motivated by the intelligent behavior of honey bees. In [93], ABC is defined as a heuristic approach that has the advantages of memory, multi-characters, local search, and a solution improvement mechanism. In the ABC model, the colony consists of three groups of bees: employed bees, onlookers, and scouts. The objective is to determine the locations of the best sources of food. The employed bees will look for food sources; if the nectar amount of a new source is higher than that of the one in their memory, they will memorize the new position and forget the previous one. The position of a food source represents a possible solution to the optimization problem, and the nectar amount of the source corresponds to the quality or fitness of the solution [94, 95].

In [96], Sultan et al. applied the artificial bee colony optimization to the problem of relay selection and transmit power allocation in CR networks. The authors aimed at maximizing the SNR at the secondary destination while keeping the level of interference low. In the ABC model, the fitness function can be represented by SNR, and the interference threshold level is the main constraint. The authors defined the role of employed bees to search for solutions by comparing the neighboring food sources with the one they memorized and updating their memory with the best solution that improves the fitness function and satisfies the constraints.

The authors in [97] used ABC for multiple relay selection in a cooperative cognitive relay network. Their aim is to find the optimal SNR, considered as the fitness value, and the best relay to cooperate, considered as the best food source.

The authors in [98] and [99] used ABC for spectrum allocation in cognitive radio networks. In [98], Ghasemi et al. aimed to optimize spectrum utilization while maintaining fairness. The position of a food source represented a possible solution of the optimization problem, and the amount of a food source corresponded to the quality of the solution.

In [99], the authors addressed channel allocation and fairness among devices. They aimed to maximize the utility function that considered the total reward, the interference constraint, and the conflict-free channel assignment.

Pradhan in [100] studied the performance of ABC in CR parameter adaptation. The control parameters considered were the transmit power and the type of modulation (BPSK, QPSK, and QAM). The author compared the performance of ABC with the performance of genetic algorithm and particle swarm optimization in terms of fitness convergence characteristic, optimal fitness, and computation time.

1.4 Discussion: challenges and evaluations

Firstly, this section presents the learning techniques evaluated against each other within a CR context. In addition, the challenges facing these learning techniques, strengths, and limitations are presented.

1.4.1 1.4.1 Learning techniques evaluation: strengths, limitations, and challenges

Learning techniques may have many advantages; however, their implementation may face many challenges. The strengths and limitations as well as implementation challenges of the previously mentioned learning techniques are presented in this subsection and summarized in Table 1.
Table 1

Learning techniques applications in CR, strengths and limitations

Learning technique

Spectrum sensing (SS)

Decision-making

Strengths

Limitations and challenges

   

Adaptation ability to minor changes

Require training data labels

Neural networks

×

×

Construction using few examples,

Poor generalization

   

thus reducing the complexity

Overfitting

Support

  

Generalization ability

Requires training data labels

vector

×

×

Robustness against noise and outliers

and previous knowledge of the system

machine

   

Complex with large problems

   

Multi-objective optimization

Require prior knowledge of the system

Genetic algorithms

 

×

Dynamically configure the CR

Suitable fitness function

   

based on environment changes

High complexity with large problems

Game theory

Related to the capabilities of the

×

Reduces the complexity of adaptation

Requires prior knowledge of the system

 

spectrum-sensing technique used

 

Solutions for multi-agent systems

and labeled training data

Reinforcement

×

×

Learning autonomously using feedback

Needs learning phase of the policies

learning

  

Self-adaptation progressively in real time

 

Fuzzy logic

Related to the capabilities of the

×

Simplicity, decisions are

Needs rules derivation

 

spectrum-sensing technique used

 

directly inferred from rules

Accuracy is based on these rules

Entropy approach

×

×

Statistical model

Requires prior knowledge of the system

 

Related to the capabilities of the

 

Simplicity

Requires prior knowledge of the system

Decision tree

spectrum-sensing technique used

×

Decision using tree branches

May suffer overfitting

    

Requires labeled training data

Artificial

 

×

Parallel search for solutions

Requires prior knowledge of the system

bee colony

   

Requires a fitness function

Bayesian

×

×

Probabilistic models

Requires prior knowledge of the system

    

May face computational complexity

Markov model

×

×

Statistical models

Requires prior knowledge of the system

   

Scalable

May face computational complexity

Case-

  

Find acceptable solution based

Complex search in large databases

based

 

×

on the existing case found

Requires predefined and relevant cases

reasoning

  

in the case database

Mistakes propagation

  • Neural networks Inspired from the biological nervous system, the artificial neural network is used to accomplish the learning process, identify new patterns, perform classifications, and improve the decision-making process. ANNs are based on empirical risk minimization. They provide an adaptation ability to minor changes of the environment and confidence information about the decision made. ANNs have the ability to describe a multitude of functions and are conceptually scalable. They are not sequential or deterministic; however, they process in parallel which permits solutions to problems where multiple constraints must be satisfied simultaneously. In addition, the neural network will have the ability to work on forward propagation mode as an analytical tool on other data once the network is trained to a satisfactory level. The output of a forward propagation run is the predicted model for the data which can then be used for further analysis and interpretation. Neural networks can be constructed by only a few samples, thus reducing the complexity of the solution. ANNs can be combined with CBR and GA in the training phase.

    ANNs are categorized under supervised learning which requires prior knowledge of the environment and training data. Neural networks may face slow training depending on the network size. In addition, it is possible to overtrain a neural network, which means that the network has been trained exactly to respond to only one type of input. The network will be then memorizing all input situations instead of learning. ANN will then face overfitting when the developed model is not general. ANNs may also face difficulty with infinite recursion and structured representations. Neural networks provide multiple solutions associated with local minima and for this reason may not be robust over different samples.

    Since artificial neural networks require training data labels, the outcome accuracy and algorithm performance are highly related to the data used for training the model. Therefore, the main challenge is to collect the training data and make sure to use clean and task-relevant data in the training phase. The second challenge is to deal with the data and the choice of initial parameters and attributes that can be nonlinear, complex, and numerous which may lead to slow training performance.

  • Support vector machines are supervised learning models used for classification, regression analysis, and outlier detection. They provide high performance in many real-world problems due to their generalization ability and robustness against noise and outliers. The basic idea of SVMs is detecting the soft boundary of a given set of samples so as to classify new points as belonging to that set or not. A good separation is achieved by the hyperplane that has the largest distance to the nearest training data points of any class, since in general the larger the margin the lower the generalization error of the classifier. SVMs map the input features into a high-dimensional feature space in which they become separable. This mapping from the input vector space to the feature space is a nonlinear mapping achieved by using kernel functions. SVM can accommodate high dimensional spaces and can still be effective in cases where the number of dimensions is greater than the number of samples. SVMs produce very efficient classifiers with high prediction accuracy and less overfitting even if training examples contain errors. SVMs deliver a unique solution, since the optimality problem is convex, which is considered an advantage compared to Neural Networks, which provide multiple solutions associated with local minima. SVMs are versatile, different Kernel functions can be specified for the decision function. Common kernels are provided, but it is also possible to specify custom kernels.

    SVM may provide high performance in small problems, however, its complexity increases in large problems. If the number of features is much greater than the number of samples, the method is likely to give poor performance. Support vector machines are powerful tools, but their computation and storage requirements increase rapidly with the number of training vectors. The core of an SVM is a quadratic programming problem, separating support vectors from the rest of the training data. They do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation. Therefore, SVMs are considered computationally expensive, they run slow with long training time.

    By introducing the kernel, SVMs gain flexibility in the choice of the functional margins form, however, the kernel function can be any of the following: linear, polynomial, sigmoid, Gaussian, radial basis function or custom kernels as a python function or by pre-computing the Gram matrix. Choosing the appropriate and convenient Kernel function may be challenging since it would affect the algorithm performance, efficiency and accuracy. In addition, the size of the kernel cache has a strong impact on run times for larger problems. Support vector machine algorithms require labeled training data and previous knowledge of the system which may be also challenging due to the diversity of the features affecting the decisions. SVMs are not scale invariant, so it is highly recommended to scale the data for efficient system performance.

  • Genetics algorithms are a randomized heuristic search strategy where the population is composed of candidate solutions obtained and emerged via mutation and crossover. GA solve multi-objective optimization problem and dynamically configure the CR in response to the changing wireless environment. They aim to optimize nonlinear systems with a large number of variables by providing multiple solutions having fitness scores measuring how close a candidate is to being a solution. Genetic algorithms search in parallel from a population. Therefore, it has the ability to avoid being trapped in a local optimal solution like traditional methods, which search from a single point. GAs are then faster and consume lower memory than searching a very large space. In addition, GA are simple and easy since the values of the fitness objective function are used for optimization purposes.

    Optimization based on fitness function cannot assure constant optimization response times which limits the genetic algorithm’s use in real-time applications. In addition, GA may not converge to a global optimum specially when the populations have a lot of subjects and performance metrics. The algorithm may get stuck on local maxima and may not provide optimal and complete solutions. The algorithm performance highly depends on the fitness function which may generate bad chromosome blocks in spite of the fact that only good chromosome blocks cross over. GAs are considered slow since evaluation of the fitness is computationally intensive. The main challenge is that GAs require prior knowledge-based learning and derived fitness functions; thus, new rules are formulated on the basis of training examples and/or patterns observed in previous iterations of the search. Setting up the problem and selecting the parameters to express the fitness function are critical to bias the next generation towards better genes. Determining chromosomal representation of parameters, domain, and range are also challenging since they are dependent on the studied problem.

  • Game theory reduces the complexity of adaptation algorithms in large cognitive networks. It provides solutions for decentralized multi-agent systems, similar to the players in a game, under partial observability assumptions. Games can be either cooperative or competitive. In cooperative games, all players are concerned about all the overall benefits, and they are not very worried about their own personal benefit. However, in competitive games, every user is mainly concerned about his personal payoff, and therefore, all its decisions are made competitively. Game theory primary applications are in linear programming, statistical decision-making, and operations research.

    One of the most basic limitations of game theory is that each player must know the cost functions of the other players. Game theory can use Bayesian analysis to reason about the cost based on observations of actions chosen by the player. In addition, in the case of nonzero-sum games, multiple Nash equilibria may exist. Game theory is also limited by the number of players in the game, since the analysis of the gaming strategies turns out to be increasingly complicated.

    Therefore, game theory requires prior knowledge of the system as well as labeled training data. The players require knowledge of different parameters such as SINR, power, and price from base stations, which is impractical in many situations.

  • Reinforcement learning can be used by agents to learn autonomously using the feedback of the actions. RL is used with multi-agent systems to solve classification and decision-making tasks. Reinforcement learning uses rewards to learn a successful agent function. The system can achieve self-adaptation progressively in real time. RL agents learn to interact with an environment and have the goal to optimize the cumulative reward.

    The main limitation of RL is that the actions of the agents and reward function should be defined based on the system and task requirements. A feedback is needed to make decisions.

    Using RL, the system engine needs to perform a learning phase to acquire the optimal and suboptimal policy. In dynamic environments, it would be challenging, sometimes even impossible, to provide the agents with the correct actions associated with the current situation.

  • Fuzzy logic is flexible, provides intuitive knowledge-base design with easy computation, and simple implementation and interpretation. Fuzzy logic is used as an application to systems that are difficult to model with unclear quality boundaries. Instead of using complicated mathematical formulations, fuzzy logic uses human-understandable fuzzy sets and inference rules to obtain the solution that satisfies the desired system objectives. Therefore, the main advantage of fuzzy logic is its low complexity and its suitability for real-time applications such as cognitive radio applications. Using fuzzy logic, solutions can be obtained given imprecise, noisy, and incomplete input information.

    There may be some disadvantages to fuzzy logic; for example, stability, accuracy, and optimality of the system are not guaranteed. In addition, increasing the dimensionality may result in inefficient and memory intensive settings for most functions. Fuzzy logic highly depends on the fuzzy inference rules which may be tuned manually, and settings can be made by trial and errors.

    The derivation of rules is challenging. These rules need to be relevant and rely on relevant features. Moreover, they should be updated when needed. Therefore, the accuracy of the fuzzy logic system depends on the completeness and accuracy of the rules. The system may return inappropriate decisions if the rules are not convenient and accurate.

  • Entropy is a measure of the uncertainty in a random variable which quantifies the expected value of the information contained in a message. The information entropy can therefore be seen as a numerical measure which describes how uninformative a particular probability distribution is. Independently of any Bayesian considerations, the problem considered as a constrained optimization problem has the entropy function as the objective function. Entropy can be combined with reinforcement learning and the Markov model.

    Although entropy is often used as a characterization of the information content of a data source, this information content is not absolute: it depends crucially on the probabilistic model. In addition, it requires prior knowledge of the system (mainly prior knowledge of the model-describing parameters to be learned).

  • Decision tree learning uses a decision tree to create a model that predicts the value of a target variable based on several input variables. The decision tree is simple to understand and to interpret by visualization. The decision can be directly made based on the tree branches. The cost of using the tree for prediction and decision-making is logarithmic in the number of data points used to train the tree. Decision tree learning can handle different types of data such as numerical and categorical data and multi-output problems. Each path of the decision tree contains the elements of the paths and the assigned risk factor which is the estimated likelihood of occurrence of the terminal event in the path. Decision trees depict future decision points and possible chance events. They add to the confidence and accuracy of the decisions.

    A decision tree needs labeled training data and does not support missing values. The decision tree may face overfitting which will make the system model loose its learning aspect. The model is highly related to the input training data which may make the system unstable because small variations in the data might result in a completely different tree being generated. In addition, the problem of learning an optimal decision tree may be nondeterministic-polynomial (NP) time NP-complete which may require heuristic algorithms to obtain solutions. Such algorithms may not guarantee global optimal solutions for decision trees.

    The main challenge is that the decision tree requires labeled training data and prior knowledge of the system. The selection of training data is critical. On one hand, decision trees may suffer overfitting when all cases are taken into consideration for building the tree, on the other hand, decision trees may be biased if some classes dominate in the training data set. Therefore, a balanced labeled data set is required prior to fitting with the decision tree without any missing values.

  • Artificial bee colony algorithm finds the optimal position of a food source representing a possible solution to the optimization problem. ABC searches in parallel over several constructive computational threads based on local problem data and a dynamic memory structure containing information on the quality of the previously obtained results. The ABC algorithm provides uni-modal and multi-modal numerical optimization.

    However, the artificial bee colony is considered as a NP-hard problem with high dimensionality. ABC is very effective in solving small- to medium-sized generalized assignment problems. Its efficiency depends on the size of the solution space, number of variables and constraints used in the problem modeling, and the structure of the solution space such as convexity. In addition, ABC needs prior knowledge of the system.

  • Bayesian approach is based on probabilistic learning. It provides exact inferences which do not rely on large sample approximations with simple interpretations. Bayesian inference estimates a full probability model and allows prior knowledge and results to be used in the current model. Bayesian inference has a statistical decision to facilitate decision-making, it includes uncertainty in the probability model, yielding more realistic predictions. The Bayesian approach does not face overfitting since it uses observed data only.

    The Bayesian approach is only useful when prior knowledge is reliable and distribution for all parameters is known. It may also face computational complexity since it involves high-dimensional integrals.

  • Markov model chain generates sequences of observation symbols by making transitions from state to state in time or space with fixed probabilities. The Markov chain contains a finite number of states and corresponding transition probabilities. The current state of the system depends on previous events and their successive structure to determine the process. For instance, a first-order Markov chain takes account of only a single step in the process, but it can be extended so that the probabilities associated with each transition depend on multiple events earlier than the immediately preceding one. Markov models are relatively easy to derive and infer from successive data. The Markov model is scalable and can model complex statistical models. It can be used for classification and prediction based on experiences. It does not require deep insight into the mechanisms of dynamic change. The basic transition matrix summarizes the essential parameters of dynamic change.

    However, the Markov model is computationally complex. It is expensive, both in terms of memory and time. The Markov model requires good training sequence. In some cases, the data available will be insufficient to estimate reliable transition probabilities, especially for rare transitions. In addition, the Markov model is a successional model where the performance efficiency depends on predictions of system behavior over time.

    The construction of Markov models requires accurate data information to determine the transfer probabilities and state conditions.

  • Case-based reasoning is capable of solving problems based on predefined cases. CBR is used to determine an acceptable solution based on the existing case found in the case database. CBR allows learning without knowledge on how rules and cases are created. The system will learn from previous and new situations and update its database accordingly which makes development and maintenance easier. CBR can be used when a domain model is difficult to generate, complex and has multiple-variable situations.

    CBR faces many limitations. It can only be used when records of initial and previously successful solutions exist. It can take a large storage space for all the cases and may face a large processing time to find similar cases in the database. CBR relies on previous cases which might include irrelevant patterns. Therefore, CBR needs appropriate training before deployment since incorrectly solved cases may lead to mistake propagation. In addition, populating and searching a large database can be time consuming and difficult which highlight the need of fast and efficient case-selection algorithms. Removing irrelevant patterns may reduce the case retrieval time and lead to higher efficiency.

  • Multi-agent systems are suitable for multi-player decision problems. They provide negotiation, learning, and cooperative-based approaches between agents which can be in our case cognitive radio users. MAS are quick, reliable, and flexible. They can accommodate multiple user networks; however, large networks may cause high cost and complexity. MAS can be combined with RL and game theory that also provide solutions using interactions between users for enhancing the performance of the system.

1.4.2 1.4.2 Applying learning in cognitive radio

The various cognitive radio network tasks need different learning techniques to learn and adapt to any change in the environment. In addition, a CR may not have any prior knowledge of the operating RF environment such as noise, interference, and traffic. Therefore, the most suitable learning technique depends on (1) the available information, (2) network characteristics, and (3) the CR task and problem to address.

1.4.2.1 1.4.2.0 Supervised vs. unsupervised learning

The availability of information affects the choice of learning techniques. In this scope, we differentiate between supervised and unsupervised learning techniques.

Supervised learning is used when training data is labeled and the CR has prior information about the environment. For instance, if the CR knows some signal characteristics, supervised training algorithms may help in achieving better signal detection and system performance. Supervised learning techniques such as decision tree, neural networks, support vector machine, and CBR may differ in their strengths, limitations, challenges, and applications in cognitive radio networks.

The ANN algorithm is based on empirical risk minimization. It provides real-time solutions which allow fast responses to changing radio environment. For instance, ANN can be used at the secondary user end in cognitive radios to act dynamically whenever there is PU activity detected over the channel to avoid collision. ANNs can be constructed by only a few examples, thus reducing the complexity of the solution, and can perform classification and assist in solution adaptation process. ANNs can be used in spectrum sensing and adapting radio parameters in CR. SVM algorithms are based on structural risk minimization. They provide superior performance compared to ANNs, especially for small training examples, since they avoid the problem of overfitting. The SVM classifier is embedded in a CR terminal of the secondary network and can be used for spectrum sensing and decision-making. CBR generally provides good or reasonable solutions; however, optimal solutions cannot be found. A CR can use a CBR to determine acceptable actions for the current environment based on the existing cases in the database. In general, the database may not include all possible situations a CR may encounter; therefore, a CR needs to learn new cases, generate new actions, and update the case database. Decision tree is simple to understand and to interpret since trees can be visualized. Unlike other methods, decision trees can process categorical and numerical data even without data normalization.

In contrast with the supervised approaches, unsupervised algorithms do not require labeled training data such as game theory, reinforcement learning, genetic algorithms, Markov model, ABC, entropy, and Bayesian approaches. Unsupervised learning is used when some RF components are unknown to the CR network which will have the opportunity to perform autonomously without having any prior knowledge.

Game theory, RL, and Markov model can be used in case of multi-agent interaction for decision-making and sensing and learning the environment. However, other techniques are used as a single-agent learning process. Single-agent and multi-agent systems and network characteristics are discussed further in the next subsection.

Game theory presents a suitable platform for modeling rational behavior among cognitive radio users. Game theory can be applied to power control, resource allocation, spectrum management, and routing algorithms. The decisions made by the agents define an outcome for the game in a way to maximize utility based on payoff matrix; the best prediction for the outcome of a game is the equilibrium of the game. Unlike in the game theoretical setting, in reinforcement learning, agents are not assumed to have full access to the payoff matrix. The agents are taken to be players in a normal form game, which is played repeatedly, in order to improve their strategy over time. Reinforcement learning allows agents to learn from their past states in order to better perform their following actions and moves. For instance, using RL, each SU senses the spectrum, perceives its current transmission parameters, and takes necessary actions when a PU appears. Penalty values are enforced when the SUs interfere with PUs, which improve the SU learning and the overall spectrum usage. Markov model can be used in case the problem considers network states and situations. This model can be used to identify the sequences of observations. For instance, CR will be able to observe, recognize, and classify received stimuli. Note that RL and Markov models can be used for single-agent and multi-agent systems.

When states or candidate solutions have a large number of successors, searching spaces with traditional search methods would be difficult. Genetic algorithms are then used to improve the model-training efficiency. Fuzzy logic approaches do not model the interaction between SUs and PUs for spectrum access. This modeling can be efficiently performed using Markov chains. Fuzzy logic can be used in decision-making to select the best suited SU for spectrum access, multi-hop routing, and detecting unauthorized users. Fuzzy logic approaches use a straightforward inference method based on predefined rules. Therefore, if the considered problem relies on rules with high uncertainty, fuzzy logic is recommended. However, when specific cases are considered, case-based reasoning is advised. Unlike Bayesian networks, fuzzy logic handles feedback loops. The Bayesian approach provides a statistical decision to facilitate decision-making. It includes uncertainty in the probability model, yielding more realistic predictions. Independently of any Bayesian considerations, the constrained optimization problem may have entropy function as the objective function. Entropy is a measure of the uncertainty in a random variable which quantifies the expected value of the information contained in a message. It can be combined with reinforcement learning and Markov model.

1.4.2.2 1.4.2.0 Single-agent vs. multi-agent

The learning technique depends also on the system characteristics such as single-agent, multi-agent, centralized, and decentralized systems [7].

In single-agent systems, an agent makes decisions without interactions with other agents in the network. Based on the network characteristics and available information, the user will learn the system, make actions, and reconfigure parameters accordingly. All the abovementioned techniques can be used in single-agent systems except game theory where multiple users must interact to achieve performance gains.

In multi-agent systems, the users may interact, negotiate, and cooperate to provide more effective communication between network entities. Using MAS in cognitive radio networks will allow the users to manage their own spectrum dynamically and in a decentralized manner. The agents will perceive their environment and react accordingly. For instance, using MAS, the interaction between agents can be cooperative where all agents are concerned about system benefits and performance without considering their own benefits. For instance, cooperative game theory is considered to avoid interference by reducing the transmission power of SUs. However, in competitive games, the agents make decisions competitively to maximize their individual gains. The association of MAS and the CR can provide more effective communication between network entities and can lead to a better exploitation of unused spectrum and optimal resource management while reducing the risk of interference.

1.4.2.3 1.4.2.0 Matching learning and CR tasks

In general, the CR tasks can be divided into two main categories: (1) cognitive capability for spectrum sensing and (2) reconfigurable capability for decision-making and resource management.

Performing spectrum sensing with supervised training data can be performed by using classification techniques such as neural networks and SVM. Some tasks related to spectrum sensing such as choosing a spectrum-sensing technique and spectrum-sensing order can be done using fuzzy logic, game theory, and decision tree, respectively [24, 41, 64]. In addition, Markov and entropy approaches can be used for spectrum sensing [101].

For decision-making, several learning techniques can be used such as decision tree, ABC, fuzzy logic, genetic algorithms, case-based reasoning, game theory, reinforcement learning, entropy, and Markov approaches. The use of these techniques may differ based on the system characteristics. For instance, game theory is used in decentralized multi-agent systems while reinforcement learning can be used in centralized and single-agent systems. Neural networks and SVM can be also used for some applications in decision-making, for instance, in [37], neural networks are used to adjust the parameters of the system so as to effectively adapt to the environment as it changes, and in [52], the SVM technique builds a model for determining the modulation scheme using the following inputs: bit error rate, SNR, and data rate. We remind the reader that the learning techniques and their applications in CR are summarized in Table 1.

Therefore, determining the most suitable learning technique for each task in general may not be feasible. However, the choice of learning technique will depend on the main problem objective and type, available information, and network characteristics.

2 Conclusions

This paper presented a survey on the applications of machine-learning techniques to cognitive radio networks. It introduced recent advances on cognitive radio networks and artificial intelligence and emphasized the role of learning in cognitive radios. The literature review of the state-of-the-art achievements in applying machine-learning techniques to CR is presented and categorized under the following major artificial intelligence techniques. This paper presented CR tasks and challenges with the necessary evaluation and challenges of the learning technique’s application in cognitive radio networks. Finally, the paper provided different points of view and forms different angles on the application of learning in CR networks.

Declarations

Authors’ Affiliations

(1)
Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon

References

  1. Cisco, Cisco visual networking index: forecast and methodology, 2014-2019. Cisco and/or its affiliates White Paper, 1–14 (2012).Google Scholar
  2. J Mitola, GQ Maguire, Cognitive radio: making software radios more personal. IEEE Personal Commun. 6(4), 13–18 (1999).View ArticleGoogle Scholar
  3. J Mitola III, in Dissertation for the degree of Doctor of Technology in Teleinformatics. Cognitive radio: an integrated agent architecture for software defined radio (Royal Institute of Technology (KTH)Stockholm, Sweden, 2000).Google Scholar
  4. S Haykin, Cognitive radio: brain-empowered wireless communications. IEEE J. Selected Areas Commun. 23(2), 201–220 (2005).View ArticleGoogle Scholar
  5. L Wang, W Xu, Z He, J Lin, in Proceedings of the Second International Conference on Power Electronics and Intelligent Transportation System. Algorithms for optimal resource allocation in heterogeneous cognitive radio networks (Shenzhen, China, 2009).Google Scholar
  6. Y Zhao, L Morales-Tirado, in Proceedings of the International Conference on Computing, Networking and Communications. Cognitive radio technology: principles and practice (Maui, Hawaii, 2012).Google Scholar
  7. M Bkassiny, Y Li, SK Jayaweera, A survey on machine-learning techniques in cognitive radios. IEEE Commun. Surveys Tutor. 15(3), 1136–1159 (2013).View ArticleGoogle Scholar
  8. L Gavrilovska, V Atanasovski, I Macaluso, L DaSilva, Learning and reasoning in cognitive radio networks. IEEE Commun. Surveys Tutor. 15(4), 1761–1777 (2013).View ArticleGoogle Scholar
  9. E Biglieri, AJ Goldsmith, LJ Greenstein, NB Mandayam, HV Poor, Principles of Cognitive Radio (Cambridge University Press, New York, USA, 2012).Google Scholar
  10. F Khan, K Nakagawa, Comparative study of spectrum sensing techniques in cognitive radio networks, (Sousse, Tunisia, 2013).Google Scholar
  11. AM Wyglinski, M Nekovee, T Hou, Cognitive Radio Communications and Networks: Principles and Practice (Elsevier, San Diego, California, USA, 2009).Google Scholar
  12. RC Qiu, Z Hu, H Li, MC Wicks, Cognitive Radio Communications and Networking Principles and Practice (John Wiley and Sons, United Kingdom, 2012).Google Scholar
  13. R Umar, AUH Sheikh, in Proceedings of the International Conference on Multimedia Computing and Systems. Cognitive radio oriented wireless networks: challenges and solutions (Tangier, Morocco, 2012).Google Scholar
  14. A Ghasemi, ES Sousa, Spectrum sensing in cognitive radio networks: requirements, challenges and design trade-offs. IEEE Commun. Mag. 46(4), 32–39 (2008).View ArticleGoogle Scholar
  15. B Wang, KJR Liu, Advances in cognitive radio networks: a survey. IEEE J. Selected Topics Signal Process. 5(1), 5–23 (2011).View ArticleGoogle Scholar
  16. VT Nguyen, F Villain, Y Le Guillou, in Proceedings of the 3rd International Conference on Awareness Science and Technology. Cognitive radio systems: overview and challenges (Dalian, China, 2011).Google Scholar
  17. WA Woods, Important issues in knowledge representation. Proceedings of the IEEE. 74(10), 1322–1334 (1986).View ArticleGoogle Scholar
  18. SJ Russell, P Norvig, Artificial Intelligence A Modern Approach (Pearson Education Limited, London, United Kingdom, 2014).Google Scholar
  19. LA Zadeh, Fuzzy sets. Information and Control. 8, 338–353 (1965).MATHMathSciNetView ArticleGoogle Scholar
  20. LA Zadeh, Communication fuzzy algorithms. Inform. Control. 12, 94–102 (1968).MATHMathSciNetView ArticleGoogle Scholar
  21. EP Dadios, Fuzzy Logic - Algorithms, Techniques and Implementations (InTech, Rijeka, Croatia, 2012). doi:doi:10.5772/2663.
  22. P Kaur, M Uddin, A Khosla, in Proceedings of the Eighth International Conference on ICT and Knowledge Engineering. Fuzzy based adaptive bandwidth allocation scheme in cognitive radio networks (Bangkok, Thailand, 2010).Google Scholar
  23. SR Aryal, H Dhungana, K Paudyal, in Proceedings of the Third Asian Himalayas International Conference on Internet. Novel approach for interference management in cognitive radio (Kathmandu, Nepal, 2012).Google Scholar
  24. M Matinmikko, J Ser Del, T Rauma, M Mustonen, Fuzzy-logic based framework for spectrum availability assessment in cognitive radio systems. IEEE J. Selected Areas Commun. 31(11), 1136–1159 (2013).View ArticleGoogle Scholar
  25. H Qin, L Zhu, D Li, in Proceedings of the Eighth International Conference on Wireless Communications, Networking and Mobile Computing. Artificial mapping for dynamic resource management of cognitive radio networks (Shanghai, China, 2012).Google Scholar
  26. K Lee, M El-Sharkawi, Modern Heuristic Optimization Techniques: Theory and Applications to Power Systems (John Wiley and Sons, Hoboken, New Jersey, USA, 2008).Google Scholar
  27. M Kantardzic, Data Mining: Concepts, Models, Methods, and Algorithms (John Wiley and Sons, Hoboken, New Jersey, USA, 2003).Google Scholar
  28. S Chen, TR Newman, JB Evans, AM Wyglinski, Genetic algorithm-based optimization for cognitive radio networks. IEEE Sarnoff Symposium. (Princeton, New Jersey, USA, 12-14 April 2010).Google Scholar
  29. M Riaz Moghal, MA Khan, HA Bhatti, in Proceedings of the 6th International Conference on Emerging Technologies. Spectrum optimization in cognitive radios using elitism in genetic algorithms (Islamabad, Pakistan, 2010).Google Scholar
  30. JF Hauris, D He, G Michel, C Ozbay, in Proceedings of the IEEE Military Communications Conference. Cognitive radio and RF communications design optimization using genetic algorithms (Orlando, Florida, USA, 2007).Google Scholar
  31. C Zeng, Y Zu, in Proceedings of the International Conference on Communication Technology and Application. Cognitive radio resource allocation based on niche adaptive genetic algorithm (Beijing, China, 2011).Google Scholar
  32. R Rojas, Neural Networks A Systematic Introduction (Springer, Berlin Heigelberg, Germany, 1996).Google Scholar
  33. SO Haykin, Neural Networks and Learning Machines (Pearson Prentice Hall, New Jersey, USA, 2008).Google Scholar
  34. X Tan, H Huang, L Ma, in Proceedings of the IEEE TENCON Spring Conference. Frequency allocation with artificial neural networks in cognitive radio system (Sydney, Australia, 2013).Google Scholar
  35. T Zhang, M Wu, C Liu, in Proceedings of the 8th International Conference on Wireless Communications, Networking and Mobile Computing. Cooperative spectrum sensing based on artificial neural network for cognitive radio systems (Shanghai, China, 2012).Google Scholar
  36. Y Yang, H Jiang, C Liu, Z Lan, in Proceedings of the Spring Congress on Engineering and Technology. Research on cognitive radio engine based on genetic algorithm and radial basis function neural network (Xian, China, 2012).Google Scholar
  37. JJ Popoola, R Van Olst, in Proceedings of the AFRICON Conference. Application of neural network for sensing primary radio signals in a cognitive radio environment (Livingstone, Zambia, 2011).Google Scholar
  38. D Bellhouse, The problem of Waldegrave. Electronic J. History Probability Stat. 3(2), 1–12 (2007).MathSciNetGoogle Scholar
  39. Z Han, D Niyato, W Saad, T Basar, A Hjørungnes, Game Theory in Wireless and Communication Networks: Theory, Models, and Applications (Cambridge University Press, New York, USA, 2012).Google Scholar
  40. OB Abdulghfoor, M Ismail, R Nordin, in Proceedings of the IEEE International Conference on Space Science and Communication. Application of game theory to underlay ad-hoc cognitive radio networks: an overview (Melaka, Malaysia, 2013).Google Scholar
  41. Y Li, H Zhang, T Asami, in Proceedings of the Wireless Communications and Networking Conference. On the cooperation between cognitive radio users and femtocell networks for cooperative spectrum sensing and self-organization (Shanghai, China, 2013).Google Scholar
  42. S Pandit, G Singh, in Proceedings of the 3rd International Advance Computing Conference. Spectrum sharing in cognitive radio using game theory (Ghaziabad, India, 2013).Google Scholar
  43. S Alrabaee, A Agarwal, N Goel, M Zaman, M Khasawneh, in Proceedings of Seventh International Conference on Broadband, Wireless Computing, Communication and Applications. Comparison of spectrum management without game theory (SMWG) and with game theory (SMG) for network performance in cognitive radio network (Victoria, British Columbia, Canada, 2012).Google Scholar
  44. L Busoniu, R Babuska, B De Schutter, A comprehensive survey of multiagent reinforcement learning. IEEE Trans. Systems, Man, and Cybernetics, Part C: Applications and Reviews. 38(2), 156–172 (2008).View ArticleGoogle Scholar
  45. M van Otterlo, M Wiering, Reinforcement Learning state-of-the-art (Springer, Berlin Heigelberg, Germany, 2012).Google Scholar
  46. K-LA Yau, in Proceedings of the International Conference on Wireless Communications and Applications. Reinforcement learning approach for centralized cognitive radio systems (Kuala Lumpur, Malaysia, 2012).Google Scholar
  47. SS Barve, P Kulkarni, in Proceedings of the International Conference on Computational Intelligence and Computing Research. Dynamic channel selection and routing through reinforcement learning in cognitive radio networks (Coimbatore, India, 2012).Google Scholar
  48. P Zhou, Y Chang, JA Copeland, Reinforcement learning for repeated power control game in cognitive radio networks. IEEE J. Selected Areas Commun. 30(1), 54–69 (2012).View ArticleGoogle Scholar
  49. S Arunthavanathan, S Kandeepan, RJ Evans, in Proceedings of the Global Communications Conference. Reinforcement learning based secondary user transmissions in cognitive radio networks (Atlanta, Gorgia, USA, 2013).Google Scholar
  50. J Han, M Kamber, Data Mining Concepts and Techniques (Morgan Kaufmann Publishers, Waltham, Massachusetts, USA, 2005).Google Scholar
  51. N Cristianini, J Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-based Learning Methods (Cambridge University Press, United Kingdom, 2000).Google Scholar
  52. Y Huang, H Jiang, H Hu, Y Yao, in Proceedings of the International Conference on Computational Intelligence and Software Engineering. Design of learning engine based on support vector machine in cognitive radio (Wuhan, China, 2009).Google Scholar
  53. Z Dandan, Z Xuping, in Proceedings of the 7th International Conference on Wireless Communications, Networking and Mobile Computing. SVM-based spectrum sensing in cognitive radio (Wuhan, China, 2011).Google Scholar
  54. S Hu, Y-D Yao, Z Yang, MAC protocol identification using support vector machines for cognitive radio networks. IEEE Wireless Commun. 21(1), 52–60 (2014).View ArticleGoogle Scholar
  55. OP Awe, Z Zhu, S Lambotharan, in Proceedings of the Conference on Technologies and Applications of Artificial Intelligence. Eigenvalue and support vector machine techniques for spectrum sensing in cognitive radio networks (Taipei, Taiwan, 2013).Google Scholar
  56. Min Lin, Jian Ouyang, Wei-Ping Zhu, in Proceedings of the IEEE Global Communications Conference. BF design in cognitive relay networks via support vector machines (Atlanta, Georgia, USA, 2013).Google Scholar
  57. LD Xu, Case based reasoning. IEEE Potentials. 13(5), 10–13 (1994).View ArticleGoogle Scholar
  58. J Kolodner, Case-Based Reasoning, (Morgan Kaufmann, California, USA, 1993).Google Scholar
  59. K-S Huang, C-H Lin, P-A Hsiung, in Proceedings of the International Conference on Frontier Computing, Theory, Technologies and Applications. A space-efficient and multi-objective case-based reasoning in cognitive radio (Taichung, Taiwan, 2010).Google Scholar
  60. YB Reddy, in Proceedings of the Fourth International Conference on Sensor Technologies and Applications. Efficient spectrum allocation using case-based reasoning and collaborative filtering approaches (Venice, Italy, 2010).Google Scholar
  61. D Ali, JMJ Park, A Amanna, in Proceedings of the 8th International Conference on Cognitive Radio Oriented Wireless Networks. A feature partitioning approach to casebased reasoning in cognitive radios (Washington, DC, USA, 2013).Google Scholar
  62. Z-J Zhao, H-C Lai, in Proceedings of the 14th International Conference on Communication Technology. A cognitive engine based on case-based reasoning quantum genetic algorithm (Chengdu, China, 2012).Google Scholar
  63. SR Safavian, D Landgrebe, A survey of decision tree classifier methodology. IEEE Trans. Systems, Man Cybernet. 21(3), 660–674 (1991).MathSciNetView ArticleGoogle Scholar
  64. H Han, J-L Wang, Q-H Wu, Y-Z Huang, in Proceedings of the International Conference on Wireless Communications and Signal Processing. Optimal wideband spectrum sensing order based on decision-making tree in cognitive radio (Suzhou, China, 2010).Google Scholar
  65. S Soltani, MW Mutka, A decision tree cognitive routing scheme for cognitive radio mesh networks. Wireless Communications and Mobile Computing, Wiley Online Library (2013). doi:doi:10.1002/wcm.2418.
  66. S Soltani, MW Mutka, in Proceedings of the 14th International Symposium and Workshops on World of Wireless, Mobile and Multimedia Networks. Decision tree modeling for video routing in cognitive radio mesh networks (Madrid, Spain, 2013).Google Scholar
  67. M Gandetto, C Regazzoni, Spectrum sensing: a distributed approach for cognitive terminals. IEEE J. Selected Areas Commun. 25(3), 546–557 (2007).View ArticleGoogle Scholar
  68. CE Shannon, A mathematical theory of communication. Bell System Technical J. 27(3), 379–423 (1948).MATHMathSciNetView ArticleGoogle Scholar
  69. N Zhao, in Proceedings of the 14th International Conference on Communication Technology. A novel two-stage entropy-based spectrum sensing scheme in cognitive radio (Chengdu, China, 2012).Google Scholar
  70. W Ejaz, MK Atiq, HS Kim, GA Shah, in Proceedings of the 8th International Conference on Cognitive Radio Oriented Wireless Networks. Optimal entropy-based spectrum sensing for cognitive radio networks under severe path loss conditions (Washington, DC, USA, 2013).Google Scholar
  71. S Srinu, SL Sabat, SK Udgata, in Proceedings of the 6th International Conference on Signal Processing and Communication Systems. Reliable cooperative wideband spectrum sensing based on entropy estimation (Gold Coast, Australia, 2012).Google Scholar
  72. S Srinu, SL Sabat, Cooperative wideband spectrum sensing in suspicious cognitive radio network. IET Wireless Sensor Syst. 3(2), 153–161 (2013).View ArticleGoogle Scholar
  73. C Yonghui, in Proceedings of the International Conference on E-Health Networking, Digital Ecosystems and Technologies. Study of the Bayesian networks (Shenzhen, China, 2010).Google Scholar
  74. WM Bolstad, Introduction to Bayesian Statistics (John Wiley and Sons, Hoboken, New Jersey, USA, 2007).Google Scholar
  75. Y Huang, J Wang, H Jiang, in Proceedings of the Second International Conference on Networks Security Wireless Communications and Trusted Computing. Modeling of learning inference and decision-making engine in cognitive radio (Wuhan, China, 2010).Google Scholar
  76. C Jiang, Y Chen, KJR Liu, Multi-channel sensing and access game: Bayesian social learning with negative network externality. IEEE Trans. Wireless Commun. 13(4), 2176–2188 (2014).MathSciNetView ArticleGoogle Scholar
  77. S Zheng, P-Y Kam, LiangY-Ch Fellow, Y Zeng, Spectrum sensing for digital primary signals in cognitive radio: a Bayesian approach for maximizing spectrum utilization. IEEE Trans. Wireless Commun. 12(4), 1774–1782 (2013).View ArticleGoogle Scholar
  78. M Zhou, J Shen, H Chen, L Xie, in Proceedings of the Wireless Communications and Networking Conference. A cooperative spectrum sensing scheme based on the Bayesian reputation model in cognitive radio networks (Shanghai, China, 2013).Google Scholar
  79. F Li, Z Xu, Sparse Bayesian hierarchical prior modeling based cooperative spectrum sensing in wideband cognitive radio networks. IEEE Signal Process. Lett. 21(5), 586–590 (2014).View ArticleGoogle Scholar
  80. W-K Ching, X Huang, MK Ng, TK Siu, Markov Chains: Models, Algorithms and Applications (Springer, New York, USA, 2013).Google Scholar
  81. JR Norris, Markov Chains (Cambridge University Press, New York, USA, 1998).Google Scholar
  82. AM Fraser, Hidden Markov Models and Dynamical Systems (Society for Industrial and Applied Mathematics, Philadelphia, USA, 2008).Google Scholar
  83. A Mukherjee, S Maiti, A Datta, in Proceedings of the Fourth International Conference on Advanced Computing and Communication Technologies. Spectrum sensing for cognitive radio using blind source separation and hidden Markov model (Rohtak, India, 2014).Google Scholar
  84. C Pham, NH Tran, CT Do, S Moon Il, CS Hong, in Proceedings of the International Conference on Information Networking. Spectrum handoff model based on hidden Markov model in cognitive radio networks (Phuket, Thailand, 2014).Google Scholar
  85. X Li, C Xiong, in Proceedings of the International Conference on Computing, Networking and Communications. Markov model bank for heterogenous cognitive radio networks with multiple dissimilar users and channels (Honolulu, Hawaii, 2014).Google Scholar
  86. W Yifei, T Yinglei, W Li, S Mei, W Xiaojun, QoS Provisioning energy saving dynamic access policy for overlay cognitive radio networks with hidden Markov channels. IEEE Commun. 10(12), 92–101 (2013).Google Scholar
  87. J Ferber, Multi-Agent System: An Introduction to Distributed Artificial Intelligence (Addison Wesley Longman, Harlow, United Kingdom, 1999).Google Scholar
  88. Wooldridge M, An Introduction to MultiAgent Systems (John Wiley and Sons Ltd., United Kingdom, 2009).Google Scholar
  89. E Trigui, M Esseghir, L Merghem-Boulahia, in Proceedings of the 5th International Conference on New Technologies, Mobility and Security. Multi-agent systems negotiation approach for handoff in mobile cognitive radio networks (Istanbul, Turkey, 2012).Google Scholar
  90. A Amraoui, FZ Benidriss, B Benmammar, F Krief, FT Bendimerad, in Proceedings of the 5th International Conference on New Technologies, Mobility and Security. Toward cognitive radio resource management based on multi-agent systems for improvement of real-time application performance (Istanbul, Turkey, 2012).Google Scholar
  91. Li H, in Proceedings of the 5th International Conference on Networking Technologies for Software Defined Radio. Multi-agent Q-learning for competitive spectrum access in cognitive radio systems (Boston, Massachusetts, USA, 2010).Google Scholar
  92. U Mir, L Merghem-Boulahia, M Esseghir, D Gaiti, in Proceedings of IEEE Consumer Communications and Networking Conference. Dynamic spectrum sharing for cognitive radio networks using multiagent system (Las Vegas, Nevada, USA, 2011).Google Scholar
  93. M Rout, B Majhi, UM Mohapatra, R Mahapatra, in Proceedings of the World Congress on Information and Communication Technologies. An artificial bee colony algorithm based efficient prediction model for stock market indices (Trivandrum, India, 2012).Google Scholar
  94. D Karaboga, C Ozturk, A novel clustering approach: artificial bee colony (ABC) algorithm. Appl. Soft Comput. 11(1), 652–657 (2011).View ArticleGoogle Scholar
  95. D Karaboga, B Basturk, On the performance of artificial bee colony (ABC) algorithm. Appl. Soft Comput. 8(1), 687–697 (2008).View ArticleGoogle Scholar
  96. K Sultan, IM Qureshi, M Zubair, SNR maximization through relay selection in cognitive radio networks. Res. J. Appl. Sci. Eng. Technol. 5(8), 2616–2620 (2013).Google Scholar
  97. J Wang, X Li, K Ma, in Proceedings of the Third International Conference on Instrumentation, Measurement, Computer, Communication and Control. Multiple relay selection scheme based on artificial bee colony algorithm (Shenyang, China, 2013).Google Scholar
  98. A Ghasemi, MA Masnadi-shirazi, M Biguesh, AF Jahromi, F Ghasemi, in Proceedings of the Sixth International Symposium on Telecommunications. Spectrum allocation based on artificial bee colony in cognitive radio networks (Tehran, Iran, 6-8 November 2012).Google Scholar
  99. X Cheng, M Jiang, in Proceedings of the 13th International Conference on Communication Technology. Cognitive radio spectrum assignment based on artificial bee colony algorithm (Jinan, China, 2011).Google Scholar
  100. PM Pradhan, in Proceedings of the International Conference on Energy, Automation, and Signal. Design of cognitive radio engine using artificial bee colony algorithm (Bhubaneswar, India, 2011).Google Scholar
  101. A He, KK Bae, TR Newman, J Gaeddert, K Kim, R Menon, L Morales-Tirado, JJ Neel, Y Zhao, JH Reed, WH Tranter, A survey of artificial intelligence for cognitive radios. IEEE Trans. Vehic. Technol. 49(4), 1578–1592 (2010).View ArticleGoogle Scholar

Copyright

© Abbas et al. 2015

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.

Advertisement