Within the basic cognitive cycle, we focus in this section on the analysis step, and more specifically on learning and decision making. We mainly find, in the literature two approaches. On the one hand, some of the articles focus on implementing smart behavior into radio devices to enable more adequate configurations, adapted to their environment, than those imposed by radio standards. As a matter of fact, standard configurations are usually over dimensioned to meet the requirements of various critical communication scenarios. This approach mainly focuses on one equipment, ignoring the rest of the network. We refer to the problem related to the first approach as dynamic configuration adaptation (DCA) problem. On the other hand due to a more pressing matter, most of CR related articles focus on spectrum management. These latter articles aim at enabling a more efficient use of the frequency resources because of its scarcity. This second problem is usually referred as dynamic spectrum access problem (DSA).
3.1. Design space and DCA problem
In this section, we discuss some of the limits related to the idealized CR concept before introducing the so called DCA problem. Several questions arise when designing a CR engine. We summarize our conceptual approach, presented in article [7], to dimension the decision making and learning abilities of a cognitive engine. Thus, we introduce the notion of design space as a conceptual object that defines a set of CR decision making problems by their constraints rather than by their degrees of freedom. We identified, in our analysis study, three dimensions of constrains: the environment's, the equipment's, and the user's related constrains.
Ideally speaking, CR concept--supported by an SDR platform--opens the way to infinite possibilities. Autonomous and aware of its surrounding environment as well as of it own behavior (and thus of its own abilities), any part of the radio chain could be probed and tested to evaluate its impact on the device's performance. This however implies that the equipment is also able, in its reasoning process, to validate its own choices. Namely, it must self-reference its cognition components [33]. Unfortunately, this class of reasoning is well known in the theory of computing to be a potential black hole for computational resources. Specifically, any turing-capable (TC) computational entity that reasons about itself can enter a Göel-turingcloop from which it cannot recover[33].
To mitigate this paradox, time limited reasoning has been suggested by Mitola. As a matter of fact, radio systems need to observe, decide, and act within a limited amount of time: The timer and related computationally indivisible control construct is equivalent to the computer-theoretic construct of a step-counting function over "finite minimalization." It has been proved that computations that are limited with reliable watchdog timers can avoid the Gödel-turing paradox to the reliability of the timer. This proof is a fundamental theorem for practical self-modifying systems[33].
Realistic CR frameworks need to take into account a large set of possible configurations, however, as mentioned hereabove through the Gödel-paradox, the decision making engine also needs to be constrained in order to avoid the system to crash. We argue in the rest of this paragraph that, in general, CR decision making problems are better defined by their constraints rather than by their degrees of freedom.
When designing such CR equipments the main challenge is to find an appropriate way to correctly dimension its cognitive abilities according to its environment as well as to its purpose (i.e., providing a certain service to the user). Several articles in the literature have already been concerned by this matter however their description of the problem usually remained fuzzy (e.g., [6, 14, 34–36]). We summarize their analysis by defining three "constraints" on which the design of a CR equipment depends: First, the constraints imposed by the surrounding environment, then the constraints related to the user's expectations and finally, the constraints inherent to the equipment. We argue that these constraints help dimensioning the CR decision making engine. Consequently, an a priori formulation of these elements helps the designer to implement the right tools in order to obtain a flexible and adequate CR.
-
The environment constraints: since a CR is a wireless device that operates in a surrounding communicating environment, it shall respect its rules: those imposed by regulation for instance (e.g., allocated frequency bands, tolerated interference, etc.) as well as its physical reality (propagation, multi-path and fading to name a few) and network conditions (channel load or surrounding users' activities for instance). Thus the behavior of CR equipments is highly coordinated by the constraints imposed by the environment. As a matter of fact, if the environment allows no degree of freedom to the equipments, this latter has no choice but to obey and thus looses all cognitive behavior. On the other side, if no constraints are imposed by the environment, the CR will still be constrained by its own operational abilities and the expectations of the user.
-
User's expectations: when using his wireless device for a particular application (voice communication, data, streaming and so on), the user is expecting a certain quality of service. Depending on the awaited quality of service, the CR can identify several criteria to optimize, such as, minimizing the bit error rate, minimizing energy consumption, maximizing spectral efficiency, etc. If the user is too greedy and imposes too many objectives, the designing problem to solve might become intractable because of the constraints imposed by the surrounding environment and the platform of the CR. However if the user is expecting nothing, then again there is no need for a flexible CR. Usually it is assumed that the user is reasonable in a sense that he accepts the best he could get with a minimum cost as long as the quality of service provided is above a certain level.d
-
Equipment's operational abilities: These limitations are perhaps the most obvious since one cannot ask the CR equipment to adapt itself more than what it can perform (sense and/or act). It is usually assumed in the CR literature that the equipment is an ideal software radio, and thus, that it has all the needed flexibility for the designed framework. On a real application the efficiency of CR equipments depends of course on the degrees of freedom (or equivalently the constraints) inherent to the wireless platform used to communicate. As examples of commonly analyzed degrees of freedom one can find: modulation, pulse shape, symbol rate, transmit power, equalization to name a few. In all cases, a CR is designed to target and support given scenarios. We do not consider that CR can be designed to answer all scenarios or concepts [18].
The interaction between all three constraints is further emphasized through the notion of design space. We denote by CR design space an abstract three dimensional space that characterizes the CR decision making engine as shown in Figure 4. It is indeed abstract since it does not have any rigorous mathematical meaning but it is only used to visually and conceptually illustrate the dependencies of the CR decision making engine to the "design dimensions": environment, parameters (usually referred to as knobs) and objectives (or criteria defined from the user's expectations).
In Figure 4, we represent two sub-spaces referred to as actual design space and virtual design space. On the one hand, the virtual design space refers to the upper bound support of the design space where all three dimensions are considered independently from each others. Its volume can be interpreted as the largest space of decision problems one could define from the three dimensions. On the other hand, the actual design space is included in the virtual design space. It results from the reduction of the design space when taking into account the correlation between the different constraints imposed by every dimension of the design space. For instance, some constraints on the environment such as, "imposed fixed waveform" might limit some objectives such as "find a waveform that maximizes the spectral efficiency".
To define a specific decision making problem, one needs to introduce a last-possibly implicit- function. This latter represents a functional relationship between all three dimensions, more specifically the correlation between the different constraints as illustrated by the design space. Thus, it models the interdependence of all three constraints. A simple representation of this interdependence can be expressed through an explicit objective function which numerical value is computed as a function of the equipment parameters, the environment's conditions as well as the values of other objective functions. Unfortunately such functions are not always available and might remain implicit. In such scenarios, optimization might prove problematic without using appropriate learning tools.
Finally, based on the here above presented analysis, all configuration adaptation problems seem to have the same roots. However, to define a specific problem among the set of possibilities in the design space, prior knowledge is important. This latter notion is further detailed in Section 4, where a classification of decision making tools as a function of prior knowledge is suggested. Nevertheless, the general DCA problem can be described as the most general decision making design space that we can state as follows [7]:
Within this framework, we assume that the environment constrains the CR by allowing only K possible configurations to use. This condition characterizes the environment and the equipment. Moreover we assume that there exist M ≥ 1 objectives that evaluate how well the equipment performs to meet the users expectations.
To conclude, we usually observe in the literature that these constrained based characterizations are implicitly made. Thus, usually the assumptions introduced to define the decision making framework are, unfortunately, hardly explained. These assumptions concern what we refer to as the "a priori model knowledge". In Section 4, we introduce and explain the notion of a priori knowledge and we present a brief state of the art on decision making for CR configuration adaptation using the DCA design space. We show that although the design space is the same, depending on the a priori model knowledge, different approaches are suggested by the community to tackle the defined decision making problems.
The following section describes an important case of DCA know as DSA that we briefly describe for the sake of consistency.
3.2. Spectrum scarcity and dynamic spectrum access
Since the early 90s, the radio community captured the potential industrial and economic opportunities that could emerge from a better frequency resource usage as noticed in 2004 in article [37]: A trend that has the potential to change the current industrial structure is the emergence of alternative spectrum management regimes, such as the introduction of so called "unlicensed bands", where new technologies can be introduced if they fulfil some very simple and relaxed "spectrum etiquette" rules to avoid excessive interference on existing systems. The most notable initiative in this area is the one of the federal communications commission (FCC, the regulator in USA) in the early 90s driving the development of short range wireless communication systems and wireless local area networks (WLANs).
Exploiting portions of the spectrum to unlicensed usage was a first step to introducing alternative frequency management schemes. Rethinking the main regulatory frameworks imposed for decades is the next step. As a matter of fact, during the last century, most of the meaningful spectrum resources were licensed to emerging wireless applications, where the static frequency allocation policy combined with a growing number of spectrum demanding services led to a spectrum scarcity. However, several measurements conducted in the United-States first, and then in numerous other countries [8, 23–27], showed a chronic underutilization of the frequency band resources, revealing substantial communication opportunities.
With the advent of SDR technology, it became, at least theoretically, possible to design agile systems capable of switching from one frequency band to another depending on given communication constraints. Thus, during the years 2002 and 2003 several task forces and researches suggested new frequency management policies and regulatory frameworks to enable efficient use of the spectrum resource [8, 38–43]. The consequences of this new framework are that the spectrum management model of today is abolished for large parts of the spectrum. Instead, "free"espectrum trading becomes the preferred mechanism and technical systems that allow for the dynamic use and re-use of spectrum becomes a necessity[37].
The DSA encompasses all suggested approaches that emerged from the early definitions of efficient and "free" spectrum access or trading. In 2007, article [44] suggested one possible and simple taxomonyf to classify the different suggested spectrum management approaches as illustrated in Figure 5. Three main approaches can be discriminated: dynamic exclusive use model, open sharing model (spectrum commons model), and hierarchical access model:
-
Dynamic exclusive use model: the spectrum basically is allocated exclusively to specific services or operators. However, the spectrum property rights framework allows opening a secondary market where the licensed users can sell and trade portion of their spectrum, whereas the dynamic spectrum allocation framework aims at providing a better allocation of the spectrum, to exclusive services, by adapting the spectrum allocation to space and time network load information.
-
Open sharing model (spectrum commons model): aims at generalizing the success encountered by WLAN technologies within the ISM band. In other words, it mainly suggests opening portions if the spectrum to unlicensed users.
-
Hierarchical access model: this framework introduced a secondary network that aims at exploiting resources left vacant by the incumbent users [usually referred to as primary users (PU)]. Secondary users (SUs) are able to communicate as long as they do not cause harmful interference to PUs. In this article, we do not subdivide this framework. As a matter of fact, their are as many subsets as the possible communication opportunities to exploit: power control, ultra-wide band communication under PUs noise level, spectrum hole detection and exploitation, directional communications to name a few [11]. In general, it is refers to as opportunistic spectrum access (OSA).
Since the seminal article of Haykin [10] in 2005, OSA research community has been, to the best of authors' knowledge the most active in the field of DSA. With several network models based on game theory [13], Markov chains or multi-armed Bandit (MAB) (and machine learning in general) [44–50], to name a few, and relying on the concept of CR, the community tackled several challenges encountered when dealing with OSA such as (non exhaustive): dynamic power allocation, optimal band selection (with or without prior knowledge on the occupancy pattern of the spectrum bands by PUs), as well as cooperation among the different SUs [12] centralized or decentralized, with or without observation errors.
In Section 5.2 an OSA scenario based on a MAB model, described in article [48], is summarized and illustrates the impact of observation errors on decision making for CR. In the following section, however, we introduce prior knowledge as a classification criteria among the main learning and decision making tools suggested in CR articles.