An overview of learning mechanisms for cognitive systems
© Bantouna et al; licensee Springer. 2012
Received: 20 May 2011
Accepted: 19 January 2012
Published: 19 January 2012
Cognitive systems were first introduced by Mitola and in the last decade they have proved to be beneficial in self-management functionalities of future generation networks. The advantages and the way that networks gain benefits from cognitive systems is analysed in this article. Moreover, since such systems are closely related to machine learning, the focus of this article is also placed on machine learning techniques applied both in the network and the user devices side. In particular, celebrating 10 years of cognitive systems, this survey-oriented article presents an extended state-of-the-art of machine learning applied to cognitive systems as coming from the recent research and an overview of three different learning capabilities of both the network and the user device.
The success of mobile networks has been driven by the services offered, i.e. voice in second generation and multimedia services in third generation (3G) networks. Similarly, a key issue for the success of future generation networks is considered to be the provision of enhanced, always available, personalised services. In addition to communication and entertainment, a wide range of other life sectors can benefit from evolving multimedia applications, including healthcare, environmental monitoring, transportation and public safety. In this respect, it is necessary to develop mechanisms that will enhance the end-user experience, in terms of quality of service (QoS), availability and reliability. At the same time, the complexity and heterogeneity of the infrastructure of mobile network operators increases as radio access technologies (RATs) continue to evolve and new ones emerge. In summary, fundamental requirements for the success of future networks are service personalisation, always-best-connectivity, ubiquitous service provision as well as efficient handling of the complexity of the underlying infrastructure. All these call for self-management and learning capabilities in future generation network systems. Self-management enables a system to identify opportunities for improving its performance and configuring/adapting its operation accordingly without the need for human intervention  Learning mechanisms are essential so as to increase the reliability of decision making. Learning mechanisms also provide the ground for enabling proactive handling of problematic situations, i.e. identifying and handling issues that could undermine the performance of the system before these actually occur.
In this respect, cognitive, reconfigurable systems [2–4], encompassing self-management and learning capabilities, have been devised as a solution to address all the key issues identified in the previous. More specifically, cognitive systems determine their behaviour, in a self-managed way. This is achieved reactively or proactively [5–7], based on goals, policies, knowledge and experience, obtained through learning. Towards this direction, this article provides an overview of two network centric applications based on two different learning techniques for identifying network capabilities in terms of available QoS expressed in bitrate.
Moreover, as the mobile phone becomes more and more an indispensable tool in daily activities, learning functionality is required on the user device as well in order to truly enhance the experience of all users, even technology agnostic ones. In this direction, the focus of this article is also placed in user centric learning capabilities as well by exploiting a learning technique for the identification of user preferences so as to connect to that network which will increase quality of experience (QoE) for the user.
In more detail, the article is structured as follows. Sections 2 presents an extended related work of functionalities built upon learning techniques and Section 3 provides two problem statements, one of them being network centric and one user centric. The two problems showcase the way that the techniques can and/or should be used. The article continues in Section 4 with the approaches that are followed in the problems stated in Section 3 by overviewing the learning-based mechanisms for acquiring learning capabilities both in network and user's equipment. Finally, the article concludes in Section 5.
2. Related study
For achieving the targets analysed above learning capabilities are required both in network and user equipments. Looking from the side of the management systems of the networks, learning capabilities can offer enhancements to the system by providing knowledge regarding the capabilities of the network and facilitating the decision-making mechanisms. The applications presented in this article referring to network capabilities of the system were expressed in terms of QoS, and more particularly in achieved bitrate. On the other hand, learning capabilities in user devices facilitate the building of knowledge regarding the user's preferences and thus improving QoE for the user.
Relevant past study includes research towards both directions. In particular, regarding networks capabilities a large variety of research has been recorded using enough different learning techniques. To begin with, the study in  describes fuzzy logic schemes for representing the knowledge for cross-layer information followed by fuzzy control theory which implements cross-layer optimization strategies. Towards the same direction, authors of  suggest fuzzy logic-based schemes which exploit past history and shared knowledge of the service quality experienced by active connections for processing cross-layer communication quality metrics so as to estimate the expected transport layer performance.
Moving to bio-inspired techniques, genetic algorithms (GAs) have also been proposed for similar reasons. More precisely, authors of  propose a GA for achieving the optimal transmission with respect to QoS goals (minimization of the bit-error rate, minimizing of power consumption, maximization of the throughput, etc.). For this purpose, the GA scores a subset of parameters and evolves them until the optimal value is reached for a given goal.
Furthermore, neural networks (NNs) have also been used for treating similar problems. Only a few examples coming from the recent literature and using NN-based techniques are [11–13]. Authors of  propose NN-based learning schemes with the aim to predict the data rate of a candidate radio configuration, which is to be evaluated by a cognitive radio system (CRS). Several NN-based schemes have also been tested in  for similar purposes. Therein, data rate is studied with respect to the quality of the link and the signal strength of the wireless transceiver, while scenarios that test the possibility of predicting the actual achieved throughput, in a short-term fashion in environments that are rapidly changing, also exist. Learning and predicting the performance is also the target of the cognitive controller built using multilayer feed-forward neural network . The controller performs this task for different channels in IEEE 802.11 wireless networks based on the experimental measurements and the environmental conditions, and eventually selects the optional channel.
Finally, Bayesian statistics and self-organizing maps (SOMs) have also been applied as techniques that can facilitate the estimation of network capacity. Among the articles that report so are [14–16], respectively. The specific approaches are selected to be further analysed in the next sections.
Looking from the user preferences side, effort was put on developing context awareness techniques [17–20], recording of user preferences [21, 22] and learning capabilities [11, 14] and exploiting these to influence the configuration selection [23–25]. Additionally, relevant work also includes the use of Bayesian networks in support of user modelling, as a method for evaluating, in a qualitative and quantitative manner, elements of the user behaviour and consequently updating the user profile. In this direction, diverse research efforts have utilised concepts of Bayesian statistics for various applications such as recommendation systems [26, 27], negotiations  and calendar scheduling . Issues that arise in achieving user-intent ascription through dynamic user model construction with Bayesian networks are addressed in .
The work presented in  focuses especially on the application of Bayesian statistics concepts for learning user preferences regarding the provision of services in mobile and wireless networks, such as voice, video streaming, web browsing, etc. In general, in the scope of mobile networks and ubiquitous computing, similar schemes have been developed. However, these focus on different aspects of user preferences and not on user preferences regarding the obtained QoS when using a certain service/application. For example, in  the targeted user preferences are modifications of the ringer volume or vibrate alarm and the acceptance or rejection of incoming calls. In , where the design for a context-aware collaborative filtering system is presented, the focus is on user preferences regarding activities in certain contextual situations. The challenges in progressing from modelling human behaviour to inferring human intent in context aware applications are addressed in , where the focus is more on ubiquitous virtual reality applications.
In summary, while a great amount of research efforts have focused on approaches for acquiring, learning and exploiting information on user preferences, the targeted user preferences, as well as the objectives, vary between the different approaches. The scheme for learning user preferences in  concentrates on preferences regarding service provisioning, in terms of QoS levels, for various services available in mobile and wireless networks. As the aim of  was to dynamically estimate user preferences and exploit these estimations in the selection of the most appropriate device configuration, so as to achieve the "always-best" connectivity concept and subsequently provide an enhanced experience to the user, it was selected to be presented in more details hereafter (see Section 4).
The main innovation of the study presented in this article lies in the fact that the article presents an approach for dynamically learning both context information and user preferences, the combination of which could be exploited in a later stage for the selection of the most appropriate network configuration. It is important here to clarify that the selection itself is out of the scope of this article.
3. Problem statement
3.1. Learning network capabilities
The aim of this problem is to estimate network capabilities. The term "network capabilities" refers to what the network is capable of, i.e. the main features of a network such as the QoS, its range, its location, its type (GSM, UMTS), etc. In thisarticle, the term refers explicitly to the QoS that the network may offer. Consequently, QoS may also refer to more than one parameter, such as the bitrate, the jitter, the delay, the bit error rate and the throughput of the network. In this case, QoS is mentioned in terms of achievable bitrate. Summarizing, the scope of this case is to estimate network capabilities in terms of QoS, expressed in bitrate, based on current network measurements and context. It is worth mentioning at this point that by network measurements, measurements that refer to parameters holding information related to the network identity, its RAT, its configuration, its Received Signal Strength Identifier (RSSI) and its traffic, in terms of packets or Bytes, are considered. Moreover, context refers to those parameters that hold information such as time, location and the environmental conditions.
3.2. Learning user preferences
This problem targets at dynamically learning user preferences regarding the perceived QoS level per service/application and potentially, the maximum acceptable price per service/application . The aim is to estimate the most likely user preferences/satisfaction for a specific service, QoS level, location and time zone.
The user profile has been modelled as a collection of parameters that can be classified in two main groups: observable and output parameters. Observable parameters include the currently running services/applications, corresponding QoS levels and associated QoS parameters, location, time zone and provided user feedback. User feedback is obtained in the following manner. The user initiates a specific service. At the initial stages it is considered that the user does not have any particular preferences. In other words, the user is initially considered to be indifferent between service provision choices. Every time the user obtains a service, a rating facility, embedded in the learning mechanism, allows the user to rate how much he/she liked the particular service provision. A Likert scale  is used for the rating, i.e. five different rating options are provided. In this way even non-technology expert users can provide the system with feedback on their preferences. The user is also given the choice to decline providing a rating. Output parameters depend on the value of observable parameters. Their value is dynamically updated over time. Output parameters represent the most appropriate configuration for the specific user in a certain context (user role/profile which encompasses certain location and time zone aspects). For the sake of simplicity the focus here is on one output parameter, namely the utility value. Other output parameters include for example the maximum acceptable price that the user is willing to pay in order to be provided a certain service at a specific QoS level. The utility value, a concept used in decision making theory and microeconomics, is used to represent user preferences for QoS levels when making use of a certain service. In other words, the utility value provides a ranking, by order of preference of service and QoS combinations. User preferences may vary depending on the contextual situation and may change over time. Therefore, the utility value is assumed to depend on a range of context-related parameters, as mentioned in the previous. More specifically, the utility value, apart from the service and QoS level, may be related to the location of the user, the time zone, and the feedback obtained from the user. The utility value for a QoS level may also implicitly correspond to a set of weights per QoS parameter (such as bit rate, delay, jitter, etc).
4.1. Network capabilities using SOM
An approach adopted in recent literature refers to learning network capabilities using the unsupervised learning technique known as SOMs. The elements that serve as inputs for discovering the QoS in this approach refers to parameters that are obtained given a configuration, e.g. RSSI, number of input Bytes, etc. On the other hand and as already stated, the parameter that expresses the QoS is the bit rate.
According to the technique applied here, SOM is used for mapping multidimensional data in a 2D-map. To do so, SOM requires a training process where the data are converted in data samples and, finally, in vectors which are mapped with respect to their resemblance. Each inserted vector too the training process updates the vector of the map that is closest to it according to Euclidean distance so as similar data samples to come closer to each others. As a result, similar vectors are mapped to the same cluster, i.e. group of vectors. Thus, the created map depicts the clustering of the data and the pattern of their relationship.
At this point, it is essential to clarify that the term "data sample" differs from the term "data" in the fact that a data sample consists of more than one data. In fact, each data sample is a combination of values, each of which refers to a different observed parameter.
Continuing on the analysis of the technique, as soon as the pattern has been recognised based on the resemblance of the vectors that were used for the training and the map has been trained and designed, a new data sample can be mapped on it with respect to the vector of the map that is closest to it when using Euclidean distance. Moreover, according to the SOM theory, data samples that belong in the same cluster are expected to be similar to each other. In our case, this means that the bit rate observed at the same time with the parameters that formulate one data sample of the cluster is expected to be the same with the respective bit rate of the other data samples of the same cluster. Thus, for inferring the network capacity all that is left to be done is to identify the cluster in which the new entry belongs. These last features of SOM technique also constitute the basis of the learning technique. Further information about the technique can be found in .
4.2. Network capabilities using Bayesian statistics
This approach is also devoted to the presentation of learning capabilities which facilitate the estimation of the network capabilities. In this approach, the mechanism is based on the correlation of candidate transmitter configurations with the QoS, in terms of bit rate, that is offered by the network given this configuration. In particular, the learning mechanism exploits the knowledge and the past experience by enforcing them with Bayesian statistics techniques suitable for reasoning about probabilistic relationships [36–38].
More specifically, since the goal is to associate different configurations of a transmitter with the bitrate, the probability to obtain a specific network capacity BR i given the configuration CFG i is calculated. This calculation and its frequent update constitute the basis of this technique. The update of these relies on approaches suggested in [36, 38–40].
Example of CPT
Pr[BR = br1| CFG = cfg i ]
Pr[BR = br2| CFG = cfg i ]
Pr[BR = br j | CFG = cfg i ]
Pr[BR = br|M|| CFG = cfg i ]
Finally, using the CPTs, it is easy to identify the most probable bit rate given a configuration. In fact, it will be the one which is associated to the highest conditional probability in the respective configuration column. Further information about the results and the mathematical background of the technique can also be found in .
4.3. User preferences using Bayesian statistics
Comparing the two approaches presented for learning network capabilities, Bayesian statistics offer also the possibility of online training. The latter abstracts the necessity of explicitly storing past observations, a feature that doesn't exist in SOM. As a result, the approach minimises the required memory capacity and thus it could also be applied from a user device perspective. If so, then the device would become for example capable of certifying that the offered network capability indeed reaches the value that is claimed by the network.
In addition, comparing the two problems that were identified and overviewed above, they are found to be interrelated. Their interrelation lies in the fact that the combination of a dynamical learning of context information and user preferences could be exploited in a later stage for the selection of the most appropriate network configuration. It is worth mentioning at this point that this mechanism is out of the scope of this article.
This study was performed in the framework of the E3 project National Participation, funded by the General Secretariat of Research and Technology (GSRT) of the Greek Ministry of Development and evolved in the context of the OneFIT (Opportunistic networks and Cognitive Management Systems for Efficient Application Provision in the Future InterneT, http://www.ict-onefit.eu) Project. Furthermore, evolved versions of this study support training activities in the context of the ACROPOLIS (Advanced coexistence technologies for Radio Optimisation in Licenced and Unlicensed Spectrum -Network of Excellence) project (http://www.ict-acropolis.eu). This article reflects only the authors' views and the Community is not liable for any use that may be made of the information contained therein.
- Kephart J, Chess D: The vision of autonomic computing. IEEE Comput 2003, 36(1):41-50. 10.1109/MC.2003.1160055View ArticleGoogle Scholar
- Thomas R, Friend D, DaSilva L, McKenzie A: Cognitive networks: adaptation and learning to achieve end-to-end performance objectives. IEEE Commun. Mag 2006, 44(12):51-57.View ArticleGoogle Scholar
- FP7/ICT project E3(End-to-End Efficiency) (ICT-2007-216248), January 2008-December 2009[https://ict-e3.eu/]
- Wireless Innovation Forum[http://www.wirelessinnovation.org/.]
- Mitola J III, Maguire GQ Jr: Cognitive radio: making software radios more personal. IEEE Personal Commun 1999, 6(4):13-18. 10.1109/98.788210View ArticleGoogle Scholar
- Haykin S: Cognitive radio: brain-empowered wireless communications. IEEE J Sel Areas Commun 2005, 23(2):201-220.View ArticleGoogle Scholar
- Mitola J: Dissertation. Royal Institute of Technology, KTH, Sweden; 2000.Google Scholar
- Baldo N, Zorzi M: Fuzzy logic for cross-layer optimization in cognitive radio networks. In Paper presented at the 1st IEEE Workshop on Cognitive Radio Networks (in conjunction with IEEE CCNC 2007). Las Vegas, Nevada, USA; 2007:1128-1133.Google Scholar
- Baldo N, Zorzi M: Cognitive network access using fuzzy decision making. IEEE Int Commun 2007, ICC 2007: 6504-6510.Google Scholar
- Newman TR, Barker BA, Wyglinski AM, Agah A, Evans JB, Minden GJ: Cognitive engine implementation for wireless multicarrier transceivers. Wirel Commun Mob Comput 2007, 7(9):1129-1142. 10.1002/wcm.486View ArticleGoogle Scholar
- Tsagkaris K, Katidiotis A, Demestichas P: Neural network-based learning schemes for cognitive radio systems. Comput Commun 2008, 31(14):3394-3404. 10.1016/j.comcom.2008.05.040View ArticleGoogle Scholar
- Katidiotis A, Tsagkaris K, Demestichas P: Performance evaluation of artificial neural network-based learning schemes for cognitive radio systems. Comput Electric Eng 2010, 36(3):518-535. 10.1016/j.compeleceng.2009.12.005View ArticleGoogle Scholar
- Baldo N, Tamma BR, Manoj BS, Rao R, Zorzi M: Abstracts of the 2009 IEEE international Conference on Communications. Dresden, Germany; 2009:1-5.View ArticleGoogle Scholar
- Demestichas P, Katidiotis A, Tsagkaris K, Adamopoulou E, Demestichas K: Enhancing channel estimation in cognitive radio systems by means of Bayesian networks. Wirel Personal Commun 2009, 49(1):87-105. 10.1007/s11277-008-9559-1View ArticleGoogle Scholar
- Bantouna A, Tsagkaris K, Demestichas P: ICAAN 2010: Artificial Neural Networks - ICANN 2010 PART II. Volume 6353. Edited by: Diamantaras K, Duch W, Iliadis LS. Springer-Verlag, Heidelberg; 2010:382. 20th International Conference on Artificial Neural Networks (ICANN 2010), Thessaloniki, Greece, September 2010. LCNS 10.1007/978-3-642-15822-3_46Google Scholar
- Bantouna A, Tsagkaris K, Demestichas P: EUREKA! 2010. Edited by: Papatheodorou TS. Department of Computer Engineering & Informatics, University of Partas; 2010:41. 1st International Conference for Undergraduate and Postgraduate Students in Computer Engineering, Informatics, Related Technologies and Applications 2010 (Eureka! 2010), Patras, Greece, October 2010. Proceedings 4thScientific Conference for undergratuated and graduate students in Informatics and Related ApplicationsGoogle Scholar
- Van Sinderen MJ, Van Halteren AT, Wegdam M, Meeuwissen HB, Henk Eertink E: Supporting context-aware mobile applications: an infrastructure approach. IEEE Commun Mag 2006, 44(9):96-104.View ArticleGoogle Scholar
- Gandetto M, Cattoni A, Regazzoni C: Abstracts of the Software Defined Radio (SDR' 05) Technical Conference. Anaheim, USA; 2005.Google Scholar
- Bellavista P, Corradi A, Montanari R, Tononelli A: Context-aware semantic discovery for next generation mobile systems. IEEE Commun Mag 2006, 44(9):62-71.View ArticleGoogle Scholar
- Kim H, Shin KG: Efficient discovery of spectrum opportunities with MAC-layer sensing in cognitive radio networks. IEEE Trans Mob Comput 2008, 7(5):533-545.MathSciNetView ArticleGoogle Scholar
- Demestichas K, Koutsorodi A, Adamopoulou E, Theologou M: Modelling user preferences and configuring services in B3G devices. Wirel Netw 2008, 14(5):699-713. 10.1007/s11276-007-0044-7View ArticleGoogle Scholar
- Homayounvala E, Aghvami AH: User preference modelling for access selection in multiple radio access environments. IEICE Trans Commun 2005, EE88-B(11):4186-4193.View ArticleGoogle Scholar
- Liu X, Shankar NS: Sensing-based opportunistic channel access. Mob Netw Appl J 2006, 11(4):577-591. 10.1007/s11036-006-7323-xView ArticleGoogle Scholar
- Perez-Romero J, Sallent O, Agusti R, Giupponi L: A novel on-demand cognitive pilot channel enabling dynamic spectrum allocation. In 2nd International Symposium on New Frontiers in Dynamic Spectrum Access Networks 2007 (DySPAN 2007). Dublin, Ireland; 2007:46-54.View ArticleGoogle Scholar
- Nolan K, Doyle L: Teamwork and collaboration in cognitive wireless networks. IEEE Wirel Commun Mag 2007, 14(4):22-27.View ArticleGoogle Scholar
- Coyle L, Cunningham P: ECBR 2004. Volume 3155. Edited by: Funk P, Calero PAG. Springer, Heidelberg; 2004:560. Advances in Case-Based Reasoning, 7th European Conference, ECCBR 2004, Madrid, Spain, August/September 2004. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)Google Scholar
- Zhang Y, Koren J: Abstracts of the SIGIR'07. Amsterdam, Netherlands; 2007:47-54.View ArticleGoogle Scholar
- Guo Y, Müller JP, Weinhardt C: CEEMAS'03 Proceedings of the 3rd Central and Eastern European Conference on Multi-Agent systems. Volume 2691. Edited by: Marík V, Müller JP, Pechoucek M. Springer, Heidelberg; 2003:303. 3rd International/Central and Eastern European Conference on Multi-Agent Systems, Prague, Czech Republic, June 2003. Lecture Notes in Artificial Intelligence (subseries of Lecture Notes in Computer Science)Google Scholar
- Oh J, Smith SF: PATAT. Volume 3616. Edited by: Burke EK, Trick MA. Springer, Heidelberg; 2004:3. 5th International Conference on Practice and Theory of Automated Timetabling (PATAT), Pittsburgh, PA, USA, August 2004. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) LNCSView ArticleGoogle Scholar
- Santos E Jr, Brown S, Lejter M, Ngai G, Banks S, Stytz M: Proceedings of the 12th International Florida Artificial Intelligence Research Society Conference. Edited by: Kumar AN, Russell I. AAI Press, Menlo Park, CA; 1999:3. Orlando, FL, 1999Google Scholar
- Stavroulaki V, Kritikou Y, Demestichas P: Acquiring and learning user information in the context of cognitive device management. IEEE International Conference on Communications (ICC) 2009, 1-5.Google Scholar
- Krause A, Smailagic A, Siewiorek DP: Context-aware mobile computing: learning context-dependent personal preferences from a wearable sensor array. IEEE Trans Mob Comput 2006, 5(2):113-127.View ArticleGoogle Scholar
- Chen A: LoCA 2005. Volume 3479. Edited by: Strang T, Linnho-Popien C. Springer, Heidelberg; 2004:244. Workshop on Location- and Context-Awareness (LoCA 2005), Oberpfaffenhofen, Germany, 2005. LNCSView ArticleGoogle Scholar
- Dey A: Modeling and Adding Intelligibility to Human Activity. IEEE Computer Society, Washington; 2008:5-8.Google Scholar
- Likert R: A technique for the measurement of attitudes. Arch Psychol 1932, 22(140):1-55.Google Scholar
- Neapolitan RE: Learning Bayesian Networks. Prentice-Hall Series in Artificial Intelligence, NJ; 2003.Google Scholar
- Pearl J: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco; 1988.Google Scholar
- Jensen F, Nielsen TD: Bayesian Networks and Decision Graphs. Springer, NY, USA; 2001.View ArticleGoogle Scholar
- Bolstad WM: Introduction to Bayesian Statistics. John Wiley & Sons, NJ; 2007.View ArticleGoogle Scholar
- Russell SJ, Norvig P: Artificial Intelligence: A Modern Approach. Prentice-Hall, NJ; 2002.Google Scholar
- Barco R, Lázaro P, Díez L, Wille V: Continuous versus discrete model in auto-diagnosis systems for wireless networks. IEEE Trans Mob Comput 2008, 7(6):673-681.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.