6.1 Simulation tools
The proposed countermeasures have been tested on a CWSN simulator [15]. This simulator has been developed as an enhanced version of the well-known Castalia simulator. The structure of Castalia has been improved by providing it with cognitive features. The CWSN simulator is responsible for scenario definition, spectrum state simulation, and communication between nodes from the physical to the application layer. It supports the cognitive features by means of the cognitive module, which has the following parts:
-
Repository. It retrieves information about local and/or remote nodes: information learned, decisions made, or current state. The kind of information stored depends on the context and the requirements of the system.
-
Access. This module lets a local repository access the repository of remote nodes. At the same time, it exports a subset of the local repository to remote nodes.
-
Policy. This module enforces the requirements for the global system depending on several factors. In this paper, security is the policy being optimized.
-
Optimizer. This block processes the repository information bearing in mind the requirements imposed by the policy module. Decisions regarding the behavior of the local node are the results of processing. They are stored in the repository and evaluated by the executor.
-
Executor. This module performs the decisions made by the optimizer.
Furthermore, the simulator also provides the VCC, a new method for sharing cognitive information among the CR modules of the nodes. CR modules can access exported information from remote repositories through this channel. It allows CR modules to be aware of their surroundings and even of the whole network. In this work, the VCC is completely ideal, and it does not take into account any delay or loss as a normal channel does.
The Castalia simulator channel model is an important aspect for these experiments. The channel model of this simulator is realistic, including the average path loss, the time variability, and random shadowing [16]. This variability in the channel conditions contributes negatively to the detection of anomalies, but it represents real conditions. Finally, the packet reception probability is implemented in the channel model in order to take into account the signal-to-noise ratio.
6.2 Simulation experiments
The attacker is implemented as a SU that changes its behavior in a precise moment acting like a PU. The attacker will try to adapt all its radio parameters according to the PU’s behavior. Some of them, such as modulation, encoding, or carrier frequency, probably will be exactly like those of the PU for two reasons. Firstly, the attackers and the PUs usually have the same hardware characteristics; therefore, the attackers can imitate the PU. Secondly, the attackers do not need to change these parameters to reach their possible goals, namely to use more spectrum, to transmit information to other destinations, or to obstruct SU transmissions.
Accordingly, it is reasonable to restrict the parameters that the attackers will change to transmitted power and occupied spectrum bandwidth. In this work, the received power and the time between packets have been used to detect anomalies, like a PUE attack on the network.
By setting the transmitted power and the transmission rate of the attacker to values similar to those used by a real PU, we can check how precise the algorithm is in detecting this kind of attack. In order to test the presented solution, when an attacker changes its behavior, the deviation in transmitted power is 1 dBm constantly and the transmission rate varies 1 packet/s in comparison to those of a normal PU. Even with this small change, the system has demonstrated to be very efficient in detecting anomalies. This assumption tries to simulate the worst case, where the PUs and the PUE attacker radio parameters are very close. In a real situation, if the attacker is a selfish PUE, it will try to transmit with the highest possible power in order to acquire the spectrum for itself. If the attacker is a malicious PUE, it will try to transmit with the highest possible transmission rate in order to affect the SU communications. Therefore, although our assumption could be a not entirely real situation, it is the worst case and the best scenario in order to test the sensibility of our algorithms. In a real case, where the differences between the attacker and the rest of the nodes will be larger, the algorithm could be adapted in order to reduce the false positive rate.
Several simulations have been executed in the simulator to extract results and to draw conclusions from the work. The scenarios have some common characteristics.
Each scenario has been run 100 times in order to add randomness. The scenario area is a 30 m×30 m square. The complete simulation time is 300 s. The number of nodes in the simulation varies from 50 to 200, including one server, three PUs, and a variable number of attackers. The learning stage covers the first 100 s in the CUSUM case and the first 60 s in the data clustering algorithm. The SUs and PUs send information to the sink, but the SUs only send the information when the channel is not being used by any PU. The location of the nodes is uniform. This improves the testing scenarios. The Castalia simulator channel model is used. This includes path loss and shadowing. The PUs’ packet transmission is 2 packets/s, and their transmission power is −1 dBm in all cases. The PUE attackers’ packet rate is 2 packets/s in the CUSUM algorithm and 4 packets/s in the clustering algorithm. The transmission power is 0 dBm. The attacks start between 100 and 200 s, depending on the scenario. The maximum node-level alarm is 5. Finally, the window time for clearing the alarms is 5 s in the CUSUM algorithm and 2 s in the clustering algorithm.
More than 30 scenarios have been run in order to test the operation of both algorithms against PUE attacks. Section 6.3 shows the results that best summarizes the performance of our approach.
6.3 Results and discussion
In Figures 1, 2, 3, where the PUE attack is simulated, the results of the CUSUM algorithm are presented. The system has shown a very good behavior in attack detection, with a detection rate of over 99% in all simulations. However, for some combinations of parameters, some normal nodes are detected as attackers. In Figure 1, we can see the results of a simulation with 50 nodes, including 1 PUE attacker, 3 PUs, 1 sink, and 45 SUs. In this situation, the decisions made individually by each node are complemented by the collaboration among them. Each line represents the same scenario where the percentage of SUs that collaborate on detection is variable. The x-axis represents the number of standard deviations that a sensing power measure can deviate from the learning average to be considered as a normal value. Finally, the y-axis represents the false positive percentage.
As we can see, the percentage of collaborative nodes is essential in PUE attack detection. For a percentage of around 20 of collaborating nodes, the results are very good, with a false positive rate of under 10% using a margin of 1 standard deviation for anomaly warnings regarding the average in the profile. If we increase this parameter to 1.3, the results are very satisfactory with false positive and false negative rates near 0%.
Figure 2 shows another scenario with worse conditions than the previous one. In this case, the nodes send lower quality information to the other nodes than in the previous scenario. This is because the node’s optimizer does not filter the anomalies, as Section 5.2 explains, and sends too many anomaly warnings through the VCC. However, if the margin of standard deviations is increased to 1.5 and the number of collaborative nodes is over 30%, the results are good enough.
However, if collaboration between nodes is eliminated and the filter in the nodes is improved, the system shows poor results. The system is not capable of discriminating between the PUE attackers and normal behavior.
Another interesting result can be seen in Figure 3. In this figure, the behavior of the system during a multiple PUE attack can be observed, where ten malicious nodes attack the system after the learning time. In this case, where 25% of the nodes are attackers, the system behavior gets worse. But, even in this case, if the number of collaborative nodes is over 20%, the results are satisfactory.
The results show that the most important parameter for improving PUE attack detection is the number of collaborative nodes. Other parameters, such as the application algorithm or the filter and the margin to mark data as anomalous, also affect the results but to a lesser extent.
The same analysis has been carried out taking into account the bandwidth occupied by the nodes instead of the received power. In this case, the results are not good enough. The reason for the poor results is the behavior of the secondary users. As we have explained before, the secondary nodes only send packets when the channel is free, so the occupied bandwidth has a greater variance than that obtained in the power detection-based scenarios. The PUE attack has been impossible to detect with good precision using the occupied bandwidth. This only means that the presented algorithm does not work with our definition of the SUs.
The data clustering algorithm provide a solution to the problem of using multiple features at the same time. Combining two features, the power received and the time between packets, the data clustering algorithm aims to detect the PUE attack with a lower false positive rate. Figures 4, 5, 6 summarize the results obtained with this approach. The simulations represent the same scenario as the one of the CUSUM algorithm. The percentage of collaborative nodes is the same, and the rate of false positives is the parameter presented. However, in the data clustering algorithm, the variable parameter is the initial cluster radius. These values range from 0.1 to 1 over the normalized value of the centroid. This parameter directly affects the false positive rate as we can see in Figure 4. The smaller the radius is, the greater the demand for grouping data becomes.
As we can see in Figure 5, the algorithm obtains satisfactory results when the initial radius is higher than 0.3. These results have been obtained simulating multiple scenarios and setups. The number of collaborating nodes is also important in the data clustering algorithm but to a lesser degree than in the CUSUM case. Here, with only 14% of the SUs collaborating, the results are acceptable.
A new scenario with ten PUE attackers is presented in Figure 6 in order to test the second algorithm in a more complex situation. Here, 20% of the network nodes are malicious. The results are a little bit worse than in the previous scenario but are really good for a radius greater than 0.3. In this situation, the false positive rate is under 2%.
Both algorithms, CUSUM and data clustering, have demonstrated the ability to detect anomalies caused by PUE attacks. In the previous figures, the optimal parameters have been presented. Following these results, the next section shows a comparison between both algorithms in terms of learning and detection time, scalability, use of resources, and scenario dependency.