Testing Applications in MANET Environments through Emulation
© Jorge Hortelano et al. 2009
Received: 22 July 2009
Accepted: 15 December 2009
Published: 9 February 2010
Mobile Ad Hoc Network solution testing is typically done using simulation. The simulated code is generally a simplified version of the real code, and thus code porting to actual operating systems lacks a strong validation. In this work we present Castadiva, a test-bed architecture that allows validating software solutions for real ad hoc network environments using low-cost, off-the-shelf devices and open source software. Castadiva is also compatible with the ns-2 simulator, allowing a combined and more complete evaluation. In this work we present our tool and the results obtained when evaluating videocalls in different scenarios, both static and dynamic.
Mobile ad hoc wireless networks (MANETs)  consist of mobile nodes interconnected by multihop communication paths. Unlike conventional wireless networks, ad hoc networks have no fixed network infrastructure or administrative support. Nodes belonging to a MANET can either be end-points of a data interchange or act as routers when the two end-points are not directly within their radio range. The topology of the network changes dynamically as mobile nodes join or depart the network, or when radio links between nodes become unusable. This means that users have the freedom to create their own networks, which can be deployed easily and cheaply. The importance of MANETs becomes evident by noticing the wide application area that these embrace. Special situations require communication networks to be available without any previous infrastructure, like emergency missions, military operations , or ad hoc meetings. However, a price for all those features is paid in terms of complex technological solutions.
MANETs require efficient routing protocols to operate correctly. A routing protocol specifies how nodes communicate with each other to disseminate routing information, allowing them to create routes between any two nodes on a network. Many of the academic papers dealing with ad hoc and mesh networks [3–6] evaluate protocols assuming varying degrees of mobility within a bounded area, usually with all nodes within a few hops of each other, and usually with nodes sending data at a constant rate. Different protocols are then evaluated based on the packet drop rate and the overhead introduced by the routing protocol. The growing research efforts focusing on this new technology require the availability of tools that allow researchers to evaluate their proposals.
Testing and evaluating any of the proposed protocols for MANETs is a mandatory request to guarantee its success in a real world application. Researchers in this field have three options for testing their MANET protocols: using simulation tools, using emulators, or using test-beds. A simulation tool is an application which behaves or operates like a given real system when provided with a set of controlled inputs. Currently several network simulators exist, including ns-2 , OPNET , Seawind , and REAL . As shown in , ns-2 is the most extended in the research community. ns-2 is a discrete event network simulator that is popular in academia for its extensibility (due to its open source model). It is typically used for the simulation of MAC, routing and transport protocols, and is heavily used in ad hoc networks research. ns-2 supports an array of popular network protocols, offering simulation results for wired and wireless networks alike. Simulation offers four important advantages. First, it enables experimentation with large networks. Second, it enables experimentation with configurations that may not be possible with existing technology. Third, it allows rapid prototyping. Finally, it makes reproducible experiments in a controlled environment possible. Simulations also have some disadvantages. First, researchers can not test their own real world implementation of a protocol in a realistic scenario. Second, simulators typically do not incorporate realistic models of node mobility and radio signal propagation .
Emulators provide an attractive middle ground between pure simulators and wireless test-beds, allowing scalable and repeatable experimentation and using real devices. In Section 2 we describe different emulators.
A test-bed is a platform for experimentation which allows for rigorous, transparent, and replicable testing of scientific theories, computational tools, and other new technologies .
In this work we present Castadiva, a test-bed architecture that simplifies carrying out realistic experiments; it relies on low-cost, off-the-shelf wireless routers combined with a Linux platform. Castadiva allows generating network topologies, exporting them to real devices and obtaining the test results. It can also generate different types of traffic between nodes and offers support for some well-known ad hoc routing protocols. It relies on a cheap architecture that includes two different networks: a wired network, called connection network, that connects the server with a group of wireless nodes, and a wireless network where the actual test-bed experiments are made. We developed a group of tools for administration purposes, with a friendly user interface design to help the user define the scenario of the network and the desired traffic connections between MANET nodes in a simple and straightforward manner. All of these tools were developed with open source software and they are freely available for the research community at http://castadiva.sourceforge.net/.
Castadiva was designed to be completely compatible with the ns-2 file format, which allows us to compare both in a simple and straightforward manner. Hence, we relied on ns-2 to validate our test-bed. We selected a range of both static and dynamic MANET scenarios and compared the results obtained using Castadiva with those obtained using ns-2. We confirmed that Castadiva is a reliable tool that provides results which are comparable, although not strictly identical, to those obtained in a realistic MANET.
After validating Castadiva we proceeded to assess the performance of a real videoconferencing tool in MANET environments. The application being tested is Ekiga , a platform offering real-time audio/video communication between users supporting both SIP  and H.323  standards. The metrics analyzed were the packet loss rate and the end-to-end delay. We also characterized the source traffic under different conditions since the data rate generated is dynamically adjusted by Ekiga.
The rest of this work is organized as follows. Related works are presented in Section 2. Section 3 describes Castadiva's architecture. Section 4 presents the implementation details. Section 5 presents the performance evaluation we have done to validate our tool, comparing this tool with the ns-2 when using TCP and UDP traffic. In Section 6 we show the results obtained with real audio/video traffic in different scenarios. Finally, in Section 7 we present our concluding remarks.
2. Related Work
The idea of automatic computer-controlled routing in a mesh network is not new; in fact the main idea dates back to Baran in the early 1964. Since then, several prototypes for generating real ad hoc network experiments were proposed and can be found in the literature.
An indoor radio grid emulator for controlled experimentation and an outdoor field trial network for end-user evaluations in real-world settings . This emulator needs an expensive noise generator since it emulates higher node distances, by reducing the signal-to-noise ratio. It also requires investing a high budget to create the grid of nodes (each computer is a possible position of the node in a simulation), as well as extra support servers for data storage. Thus, deploying the entire infrastructure requires a lot of room.
An emulator that strikes a balance between desktop simulations and outdoor field tests by allowing users to develop and test ad hoc protocols and applications in a laboratory environment . This emulator can only generate network topologies and capture packets.
Carnegie Mellon University Wireless Emulator
Supports real devices, applications, MAC and PHY layers on a network-wide scale while maintaining experimental control and repeatability . The disadvantages of this emulator are that it does not use commercial off-the-shelf devices, using a FPGA for digital emulation instead.
An emulator to test an ad hoc network of any scale and with any mobility scenario without actually moving the ad hoc nodes physically . We discarded this emulator for our test since it relies on expensive clusters to emulate the scenario.
Another emulator designed for performance studies of real protocols and applications on wireless networks . It uses an emulated link and router to generate the network topology and noncommercial off-the-shelf devices, making it difficult to scale up the network. Also, it can only emulate one traffic flow.
Is a hybrid wireless testbed environment targeting realistic, scalable, and flexible evaluation of wireless technologies and applications . These hybrid experimentation modes use physical and simulated elements (e.g., protocol layers, subnets) in different combinations. WHYNET uses a geographically distributed set of physical wireless testbeds, making results difficult to be replicated by other researchers.
Comparative of existing emulators.
Size of emulator
Routers, PDAs and computers
fits in a desktop
fits in a room
No (is almost a simulator)
fits in a desktop
Carnegie Mellon University Wireless Emulator
computers, but needs FPGAs
fits in a desktop
Only mobility in grid.
Huge, needs a warehouse
Yes but very expensive
No, only emulate two nodes
fits in a desktop
fits in a desktop
Yes (by simulation)
Overall, Castadiva outperforms the other emulators in terms of (a) variety of devices that can be use as nodes, (b) initial budged needed to deploy the emulator, and (c) ease of use provided by the GUI; this means that it makes a clear contribution to the research community.
3. Castadiva's Architecture Overview
Castadiva is a test-bed designed to deal with the development and the performance evaluation of protocols and applications for MANETs. The test-bed relies on actual IEEE 802.11 wireless interfaces for communication among nodes. Castadiva is composed by a server that runs the main application, several wireless nodes, two different networks (wired and wireless), and an application that coordinates all devices.
Castadiva's server executes the application and configures the network devices. Concerning the wireless nodes used, they can be any sort of computing device, like a laptop, a PDA, or a wireless router. In our prototype, each node is a Linksys router to minimize costs. The main requirement for a node is that it must have a Linux/Unix operating system installed and two network cards: an Ethernet card and an IEEE 802.11 card. If the node is a wireless router, the OpenWRT  kernel is an excellent solution. OpenWRT is an open source operating system available for a wide range of router manufacturers. This embedded Linux system natively offers SSH connections, along with the possibility of running shell scripts. Moreover, a programmer can develop its own application in a standard Linux distribution and export it to this operating system. In our case, we developed some applications in C for traffic generation and control purposes.
The coordination network is a wired network that connects Castadiva's core server with the wireless nodes. This network allows the main application to send configuration messages to all the nodes without creating any interference within the wireless network itself. It is based on Fast-Ethernet technology, to avoid large latencies. Basically, the coordination network requires a switch connected to the main server and to all nodes. Through this network the main application sends instructions to nodes, allowing them to reconfigure periodically according to the desired network topology and also to run lightweight traffic-generating applications available on each wireless node. For communication purposes, we rely on the SSH protocol  to send instructions through this network. Using a fast network means that all nodes participating in a test will start at about the same time, avoiding significant latency effects and maximizing result accuracy.
The wireless network is composed by Castadiva's wireless nodes, and the topology of this network is defined by the GUI of Castadiva, so that it can change at runtime. Nodes communicate in ad hoc mode using IEEE 802.11g technology.
4. Castadiva's Implementation Details
In this section we detail the requirements of Castadiva on the server and on the wireless nodes. We describe the software tools we have developed to connect all the wireless nodes with the server, and how Castadiva allows emulating connections among them. We also explain the process of designing network topologies by using the Scenario Generation tool, an interactive and user-friendly interface that allows defining the network's scenario and the desired traffic connections among nodes.
The connection between Castadiva's core element (server) and each node is made using both SSH and NFS connections. On Castadiva's server, the user interacts with the GUI application by defining the network topology and the traffic and selecting the desired routing protocol. Then, through SSH, the application sends a start instruction to each node through the coordination network (wired). Wireless nodes achieve coordination among themselves by executing the required binaries, which are stored in a server folder shared through NFS. This is an easy way to spread instructions to all nodes, and it also solves storage limitation problems on nodes. When tests start, a group of files with the results is created and stored into Castadiva's server. Each individual router accesses its own configuration file by relying on the NFS filesystem. We find that Ethernet connections are fast enough to export these files to the routers without significant delays.
The main application parses the results, thus obtaining the different testbed statistics. Finally, the application displays results to the user.
Next we present the Castadiva's implementation, which can be divided into two parts: the light-weight applications running on wireless nodes, and the main application.
4.1. Wireless Nodes' Software
Each node has a set of requirements that must be met for successful operation: a Linux-based operating system, a set of special-purpose scripts, and some specific applications and connectivity to Castadiva's server.
The operating system installed on each router is OpenWRT. OpenWRT allows executing BASH scripts natively; besides, it includes Dropbear, a simple SSH server used to receive instructions from Castadiva's server. Concerning the set of Castadiva's scripts, they are generated automatically by Castadiva's main application. Their purpose is to configure the wireless network topology.
Each node makes use of three applications: Iptables, TcpFlow, and UdpFlow. The first one is open source and exists in most Linux distributions, while the other two were developed by us.
Network topology configuration relies on the Iptables  tool. According to the selected topology, Iptables allows us to dynamically break the network links between pairs of nodes. This tool exists for all Linux distributions, including the OpenWRT embedded system.
To generate traffic we created the UdpFlow and TcpFlow tools. Both tools are designed to generate a traffic flow between two nodes, being that each tool creates a single traffic flow. To create a flow of data we must specify a source/destination pair, the starting and ending times for this flow, and the maximum amount of bytes to be sent.
Castadiva also includes routing agents for well-known routing protocols, such as AODV and OLSR, that are included with the OpenWRT and initiated according to user settings.
4.2. The Main Application
Castadiva's core element, a Java application running at the server, includes all the control functions required for testbed experimentation. It is responsible for network topology maintenance, traffic control, as well as result calculation and presentation. A user can define the characteristics of wireless nodes. Each node is deployed at a specific position in a simulated area as chosen by the user. Once the topology is defined, Castadiva must configure the wireless nodes according to that topology. The application communicates with each node through SSH connections to send the required instructions. The traffic flow between nodes and the routing protocol used are also set through this technique. When all the experiments are finished, Castadiva's core must calculate the result statistics for the experiment by gathering all the data obtained and finally show these results to the user.
Castadiva's main application was created using Java's Swing library. We consider that it is a good solution for visual design since most basic components are already created and can be easily modified by the programmer.
Castadiva is designed to be a test-bed where network scenarios and traffic between nodes are generated so as to resemble a real MANET. Therefore, it is expected to be an easy and useful tool for the study of MANETs.
To start a new experiment we only need to define the network topology in the corresponding window and then define the traffic flow and the routing protocol used. By pressing the start button tests begin, and Castadiva returns the test results automatically at the end of the simulation. We now offer more details about the services offered by Castadiva.
4.2.1. Adding Nodes to the Testbed
All the information is defined automatically when the user wishes to add a new one, though it can be changed by the user or can be read from a file. An internal identifier is required to distinguish a node from others in Castadiva's framework. Such identifier is then referenced when defining the network topology and data connections. The remaining parameters will be used by Castadiva's main application to connect nodes among themselves and with the main server. The MAC address is required for Castadiva to enforce topology changes.
All the executable files and scripts are stored in an NFS directory that is accessible by all nodes. This way Castadiva makes storage capacity independent of wireless nodes' memory.
Castadiva relies on its own tools to generate traffic between nodes. Such tools run at each node and must be compiled for all types of CPUs used. Currently, tools are compiled for MIPS and Intel processors, though the list can be easily augmented.
The SSH user and password fields are used by the main application to connect to each individual router and submit commands. Also, a Ping button was included to allow testing the connectivity between the server and the routers.
4.2.2. Ad Hoc Network Scenario Generation
Once all the nodes are defined, they can be configured to conform a scenario. Castadiva supports both manual and random topology generation, and the scenario is set through Castadiva's blackboard. The blackboard is a representation of a virtual environment where nodes are located. Nodes are differentiated through different colors and labels. If the adequate option is selected, the radio communication range is also shown through a circle of the same colour.
At the right hand we may edit node properties, such as position and signal range. Castadiva also can activate or deactivate the RTS/CTS 802.11 option of each node. The group of buttons appearing below allows starting a new test, stopping it, and rebooting nodes to reset all values.
At the left hand, Castadiva offers scenario option editing. We can define the scenario bounds, the test time, node mobility, and the routing protocol used. The Declare traffic button allows setting traffic, as shown, later on, and the stop button halts it.
A status bar provides general information to inform the user about what is being done, and the horizontal scrolls allows zooming in and out. Finally, the user may alternate between two different views: radio ranges or a graph. Every edge of the graph represents an IEEE 802.11 link connection, which is a more intuitive view.
4.2.3. Mobility in Castadiva
Castadiva offers the user two different approaches for emulating mobility: (a) any scenario from the ns-2 simulator could be directly used within Castadiva, and (b) the user can take advantage of the GUI to select their own mobility model. We already included the random way point mobility model . This behavior is similar to the one provided by the "setdest" tool embedded in the ns-2 simulator: if a user picks a value greater than zero in the speed option, each node acquires a random movement with a speed between zero and the inserted value. When a node arrives to a destination point, it waits for a selected pause time and then select a new random destination point to move to. Castadiva allows users to easily add new mobility models implementations.
For the emulation of mobility, Castadiva generates all node movements required for the test before it starts. In particular it generates, for each node, a mobility vector according to the selected user option in the GUI or imports it from an ns-2 scenario. Also, Castadiva calculates the variation of the topology continuously in time. Therefore, our emulator obtains the new topology of the network, updating the wireless links on each real device whenever necessary. The granularity used to upgrade the wireless links is of one second, but it can be changed. Since the real devices do no experience real mobility, we emulate the wireless links connectivity between nodes using the Iptables tool. This tool allows each node of Castadiva to only receive packets from nodes that are within range according to the emulated topology, and thus it blocks the reception of packets when nodes are out of range. All Iptables rules are stored into script files. These scripts will be loaded on each node through NFS when the simulation starts. Figure 7 shows the file loaded by AP1. We can see in this figure different Iptables rules interleaved with sleep rules. The sleep time allows enforcing Iptables' rules at suitable times. Hence, each rule is loaded only when the emulation requires a node to change its connectivity state towards other node. For example, Algorithm 1 shows what is the behaviour of Castadiva when a node (with MAC 00:14:BF:3C:39:EC) goes out of range at second 15 and comes back into range at second 35.
Algorithm 1: Iptables rules to emulate when a node goes out of range between seconds 15 and 35.
iptables -I INPUT -m mac –mac-source 00:14:BF:3C:39:EC -j DROP
iptables -D INPUT -m mac –mac-source 00:14:BF:3C:39:EC -j DROP
Castadiva also allows a user to see all node positions at a certain instant of time. When a simulation finishes, the user can activate the Show option and pick an instant of time. Immediately Castadiva shows the network topology at that time. This option is useful to do a later evaluation of the changes occurred in the network topology when mobility was activated.
4.2.4. Network Traffic Declaration
When an experiment finishes, Castadiva fills this table with results, including the average throughput, in the case of TCP traffic, or the percentage of packets received in the case of UDP traffic.
4.2.5. Random Test Generator
The user must specify the bounds of the scenario and the routing protocol used. The minimum and maximum number of nodes for testing must also be defined, along with the increase granularity (e.g., with a node interval between 4 and 10 nodes and a granularity of 2, Castadiva executes four tests with 4, 6, 8, and 10 nodes). Castadiva allows also to specify how many times each test will be repeated.
At the top left the current scenario generated is displayed, though it cannot be modified. Again, all the tests can be stored in either Castadiva's orns-2'sformat.
4.2.6. Compatibility with the ns-2 Simulator
Castadiva can also import/export scenarios to/from ns-2, making it compatible with the most widely used MANET simulator. This characteristic offers the possibility of comparing results and reaching more meaningful conclusions.
4.2.7. Castadiva Extensions for External Traffic Injection
This way we can use laptops to generate any flow of traffic and redirect it to specific nodes of Castadiva. For example, you can use real applications like Ekiga to launch a video-conference and study the behaviour of H.323, SIP, and video streaming protocols in MANETs.
5. Validation of Castadiva
In this section we validate the functionality and accuracy of Castadiva in different MANET scenarios. We divide our tests in two groups: first, we compare Castadiva with the ns-2 simulator using both TCP and UDP traffic in a static scenario; second, we then proceed with a similar analysis in a mobile scenario.
5.1. Evaluation of Castadiva with a Static Scenario
Since Castadiva is completely compatible with the ns-2 file format for scenarios, we were able to compare both in a simple and straightforward manner.
We set the wireless nodes' range to 250 meters. In terms of traffic, we define both UDP and TCP connections between each participating node and node 6. For TCP connections, the maximum transfer size is of 100 MB. UDP flows generate 55 packets per second, and packet size is fixed at 512 bytes.
On both tests we observe that Castadiva has a lower throughput than ns-2. We must take into account that the shared wireless media is prone to both transmission errors and contention among stations, which is due to the wireless devices being placed physically close to each other. In the case of ns-2, only contention effects are simulated, which explains the observed discrepancy.
UDP tests show that traffic from nodes AP7, AP8, and AP9 is now able to reach the destination, while node AP10 remains isolated as intended. Notice that, for the former nodes, the packet loss ratio increases slightly with the number of hops to destination. With ns-2 we do not observe this behaviour for the reasons referred above.
Results with TCP traffic show that the throughput for nodes 1 to 5 is reduced compared to the previous experiment (Figure 13), which is due to competing traffic from AP7, AP8, and AP9. For these nodes throughput decreases with increasing number of hops, as expected. With ns-2 we observe that both AP8 and AP9 suffer from starvation; one of the reasons for this behaviour is that, with Castadiva, all nodes share a same medium and so packet collisions between out-of-range nodes do not occur.
In terms of the control network (wired), we observe that the bandwidth consumption of the SSH protocol over Ethernet is far from approaching saturation, and that latency is low enough to allow adequate coordination of all nodes.
5.2. Evaluation of Castadiva with a Mobile Scenario
We now define a mobile scenario with an area of and simulation time of 500 seconds and we set the wireless nodes' range to 250 meters. In terms of traffic, we define both UDP and TCP connections between each participating node and node AP7. For TCP connections, the maximum transfer size is of 1000 MB. UDP flows generate 4 packets per second, and packet size is fixed at 512 bytes. In the first test, each node has a maximum speed of 5 m/s and no routing protocol is used.
The figure shows that the obtained results are quite similar, which validates Castadiva's implementation. Since we have not selected any routing protocol, transmissions are successful only when the destination is a 1-hop neighbour. We also observe that Castadiva has, in general, a lower throughput/delivery rate than ns-2. When Castadiva is used, the shared wireless media is prone to both transmission errors and contentions among stations. In the case of ns-2, only contention effects are simulated, which explains the observed discrepancy for UDP traffic. In the case of TCP traffic, results are more heterogeneous since it is a stateful, bandwidth-greedy protocol prone to present nonlinearities, specially in mobile ad hoc network environments. In terms of routing, we picked the OLSR routing protocol since its implementation is available for both testing environments (simulated and real), being quite similar.
Default OLSR parameter values.
On both tests we observe that Castadiva has a lower throughput than ns-2. For TCP we observe that the difference between Castadiva and ns-2 is more significant than for UDP. To discover the reason of this, we study a controlled scenario with only two nodes and measure the arrival time of each packet. With this experiment we obtain the mean delay to do a rerouting when the OLSR performs a topology update. In ns-2 it is of five seconds. However, our nodes have a mean delay of almost eight seconds. This three-second difference causes TCP agents in ns-2 to achieve a higher throughput.
Again we observe important similarities between Castadiva and ns-2's results. For both UDP and TCP traffic the behaviour of these platforms is quite similar. With Castadiva we achieve slightly less throughput than with ns-2 for the reasons explained in Section 5.1. Tests show that the average percentage of UDP packets received is increased when a routing protocol is used, since it allows nodes AP9 and AP10 to reach their destinations. When studying the behaviour of the network using TCP traffic instead we observe that the average throughput is not increased because, in both simulations, the network is saturated. When looking at the overall trend, though, we find that there is a high degree of resemblance between both.
6. Assessing the Performance of Videoconferencing in MANETS with Castadiva
We now describe the performance results obtained when transmitting real-time video traffic through Castadiva. We test with several different scenarios where the purpose is to stress the application and evaluate the impact of the number of hops in the quality of the received video stream. We are going to employ two different kinds of video streams: a standard videocall, where the video sequence is almost static, and a more dynamic videocall, where we will point the webcams towards a screen displaying a movie.
Similarly to the previous section, we will perform tests in both static and mobile scenarios.
6.1. Static Scenario Evaluation
Initially we evaluate the performance of a videocall in a static MANET for reference.
The scenario is defined in a area, and the test time is of 5000 seconds. We set the wireless nodes' range to 250 meters. To monitor the video traffic we use the tcpdump application .
To generate the video traffic we have two laptops with a webcam attached and running Ekiga, an open source VoIP, and videocall application for Linux.
OpenWRT parameters values for the OLSR protocol.
The effects we want to study are the variations in terms of throughput, delay and jitter when varying the numbers of hops in the network.
6.1.1. Evaluating the Performance with a Standard Videocall
We now present the results obtained when using Castadiva to evaluate a standard videocall between two users. Based on traffic traces at source and destination, we measured the throughput and the interpacket delay (jitter).
In the scenario with ten hops, if we compare the interpacket arrival time with the one obtained in the one hop scenario, we can appreciate an increase of more than 2 milliseconds. This is caused by the forwarding time of the additional nodes on the path. This is expected since these packets, usually generated back-to-back by the videoconferencing application, experience significant jitter when traversing an ad hoc network with a high number of hops.
6.1.2. Evaluating the Performance of a Movie Transmission
6.1.3. Evaluating the Round-Trip Time of the Different Scenarios
We also evaluated the impact of increasing the number of hops on delay by using ping sessions and measuring the impact on round-trip time while maintaining videoconference sessions active. We generate a ping session between the two laptops in the chain scenario described earlier, varying the number of hops from one to ten. We obtain the average time of the ping session used to send a packet and receive the answer. To obtain the one-way delay, we divided the results obtained by two.
6.2. Dynamic Scenario
In this section we evaluate different scenarios with a dynamic topology. We change the nodes' mobility by testing with the following speed values: 3, 6, 9, 12, 15 meters per second. The scenario is defined in a area, and the test time is of 5000 seconds. We set the wireless nodes' range to 250 meters. To generate the video traffic we have two laptops with a webcam attached running Ekiga. In this test, as for the static scenario, we also differentiate between two types of scenarios: a standard videocall and a videocall where the webcams point to a movie being displayed.
OLSR values used for the mobility scenarios.
Since, for these tests, we picked a particular OLSR configuration, the first step is to measure the total amount of time that the route between both laptops is established. For this test we emulate 20 random scenarios for each speed, and we measure with a ping if the route has established or not.
Percentage of the simulation time when exists a route between both laptops.
% time with route
The percentage of time without route is directly related to the amount of time that OLSR needs to obtain a route when the topology is changing plus the time when the network is split, and there is no possible route between the two laptops.
6.2.1. Evaluating the Performance with a Standard Videocall
As in Section 6.1, we first evaluate the behaviour of a standard videocall in a MANET. Since the number of hops is variable in each simulation due to mobility, in this case we study the variation of the throughput with different node speeds as well as the packet losses since the protocol is based on UDP.
6.2.2. Evaluating the Performance of a Movie Transmission
In this section we also measure the throughput obtained by the videoconferencing participants when streaming a movie being displayed.
If we compare this figure with the one shown when evaluating a standard videocall, we can see a similar trend. The last test shows a higher throughput, as the transmitted video generates a higher data rate. Packet losses are decreased as in the standard videocall because (1) we have a higher throughput when a route is established due the type of video captured, (2) Ekiga's bandwidth throttling mechanism decreases the packet injected when a route is lost, and (3) on average we have the same time periods without routes between the two laptops as in the previous tests. Jointly, these factors explain why the total percentage of packets lost is reduced at higher node speeds (within the speed range tested).
In this work we present Castadiva, a novel architecture to improve research in the MANETs field by allowing to make real testbed experiments in a simple and straightforward manner.
Castadiva combines the convenience and productivity of Java with the power of the Linux kernel and accompanying tools to emulate an ad hoc network environment. The system was designed to simplify the tasks of scenario generation and starting traffic flows among independent, IEEE 802.11-based, wireless nodes. One of its key advantages is that it is fully compatible with the ns-2 simulator.
The architectural design of Castadiva differentiates wireless nodes, used for the actual experiments, from the core application, which has management and control purposes. This core application provides an easy interface to define network topologies and traffic flows between nodes. Those definitions are then translated into run-time instructions sent to testbed nodes when experiments are on-going. We observed that the use of SSH protocols and Fast Ethernet connectivity allows nodes to synchronize the start of an experiment with high accuracy, being all instructions read at once; afterwards, the testbed relies on individual clocks to synchronize instructions throughout the remaining time of an experiment.
An important issue when designing Castadiva was that of ns-2 compatibility. Castadiva is compatible with the file format used by the ns-2 simulator. In the validation section, we compared the results obtained with Castadiva and the ns-2 network simulator. By using both TCP and UDP data traffic, and under a variety of static and dynamic MANETs scenarios, we show that Castadiva is able to offer confident results while using cheap wireless off-the-shelf devices.
Castadiva was also extended to allow injecting traffic from external applications. Such functionality enabled us to assess the performance of videoconferencing in an emulated ad hoc network using a real application such as Ekiga. We studied the behavior of a videocall in both static and dynamic scenarios. For the static scenario we change the number of hops between caller and receiver from one to ten. We also vary the characteristics of the video being transmitted. The results obtained in the video evaluation show that, for a large number of hops, delay times are quite high, typically provoking annoyance to the videocall users. Packet losses also become problematic when source and destination are more than six hops away. Additionally, our experiments showed that the OLSR routing protocol is prone to suffer from instability if the hop count is high, despite altering the default parameter values to more efficient ones.
Concerning the mobile scenario, the tests performed show that, on average, a MANET can be used to support a videocall with mobility, being the quality of the video decreased and having a percentage of time without service between 7% and 36% of the total time, depending on the nodes' speed. In general, we can assert that higher degrees of mobility are better to avoid nodes becoming isolated for long periods of time.
To summarize, this work shows that the advantages of usingCastadiva with respect to other MANET test-beds are that (1) it is a very low-cost test-bed since each node costs about 50$, (2) it is fully compatible with the ns-2 simulator, allowing to compare results between both in a straightforward manner, and (3) it does not occupy a lot of physical space.
Castadiva is free software developed under the GNU GPL licence and can be downloaded at http://castadiva.sourceforge.net/.
This work was partially supported by the Ministerio de Ciencia e Innovación, Spain, under Grant TIN2008-06441-C02-01.
- Conti M, Giordano S: Multihop ad hoc networking: the theory. IEEE Communications Magazine 2007, 45(4):78-86.View ArticleGoogle Scholar
- Burbank JL, Chimento PF, Haberman BK, Kasch WT: Key challenges of military tactical networking and the elusive promise of manet technology. IEEE Communications Magazine 2006, 44(11):39-45.View ArticleGoogle Scholar
- Perkins C, Belding-Royer E, Das S: Ad hoc on-demand distance vector (AODV) routing. Request for Comments 3561, Network Working Group, , July 2003 http://www.ietf.org/rfc/rfc3561.txtView ArticleGoogle Scholar
- Clausen T, Jacquet P: Optimized link state routing protocol (olsr). Request for Comments 3626, MANET Working Group, October 2003 http://www.ietf.org/rfc/rfc3626.txtView ArticleGoogle Scholar
- Johnson D, Hu PY, Maltz D: The dynamic source routing protocol (dsr) for mobile ad hoc networks for ipv4. Request for Comments 4728, MANET Working Group, , February 2007 http://www.ietf.org/rfc/rfc4728.txtView ArticleGoogle Scholar
- Chakeres ID, Perkins CE: Dynamic MANET on-demand (DYMO) routing protocol. IETF Internet Draft, November 2007Google Scholar
- USC/ISI UC Berkeley , LBL and Xerox PARC researchers : Network simulator—ns (version 2). , 1998 http://www.isi.edu/nsnam/ns/
- OPNET Technologies Inc : OPNET making networks and applications performs. http://www.opnet.com/
- Kojo M, Gurtov A, Manner J, Sarolahti P, Alanko T, Raatikainen K: Seawind: a wireless network emulator. Proceedings 11th GI/ITG Conference on Measuring, Modelling and Evaluation of Computer and Communication Systems (MMB '01), September 2001, Aachen, GermanyGoogle Scholar
- Keshav S: REAL: a network simulator. University of California, Berkeley, Calif, USA; 1988.Google Scholar
- Andel TR, Yasinsac A: On the credibility of manet simulations. IEEE Computer Society 2006, 39(7):48-54. 10.1109/MC.2006.242View ArticleGoogle Scholar
- Maltz DA, Broch J, Johnson DB: Experiences designing and building a multi-hop wireless ad hoc network testbed. IEEE Journal on Selected Areas of Communications 1999, 17(8):1439-1453. 10.1109/49.779925View ArticleGoogle Scholar
- Ekiga, free your speech, http://ekiga.org/
- Rosenberg J, Schulzrinne H, Camarillo G, Johnston A, Peterson J: Sip: session initiation protocol. Request for Comments 3261, MANET Working Group, , June 2002 http://www.ietf.org/rfc/rfc3261.txtGoogle Scholar
- ITU-T : Packet-based multimedia communications systems. , June 2006 http://www.itu.int/rec/T-REC-H.323/enGoogle Scholar
- Baran P: On distributed communications. The RAND Corporation, 1964Google Scholar
- Raychaudhuri D, Seskar I, Ott M, Ganu S, Ramachandran K: Overview of the orbit radio grid testbed for evaluation of next-generation wireless network protocols. Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC '05), March 2005, New Orleans, La, USA 3: 1664-1669.Google Scholar
- Karygiannis A, Antonakakis E: MLab: an ad hoc network test bed. In Proceedings of the 3rd IEEE Consumer Communications and Networking Conference (CCNC '06), January 2006, Las Vegas, Nev, USA. Volume 2. National Institute of Standards and Technology; 1312-1313.Google Scholar
- Judd G, Steenkiste P: Design and implementation of an RF front end for physical layer wireless network emulation. Proceedings of the 65th IEEE Vehicular Technology Conference (VTC '07), April 2007, Dublin, Ireland 974-979.Google Scholar
- Zhang Y, Li W: An integrated environment for testing mobile ad hoc networks. http://www.wins.hrl.com/projects/adhoc
- Zhou J, Ji Z, Varshney M, Su Z, Yang Y: Whynet: a hybrid testbed for large-scale, heterogeneous and adaptive wireless networks. Proceedings of the 1st ACM International Workshop on Wireless Network Testbeds, Experimental Evaluation and Characterization (WiNTECH '06), 2006, Los Angeles, Calif, USA 3: 111-112.View ArticleGoogle Scholar
- OpenWRT, wireless freedom http://openwrt.org
- Ylonen T, Lonvick C: The secure shell (ssh) authentication protocol. Request for Comments 4252, MANET Working Group, , January 2006 http://www.ietf.org/rfc/rfc4252.txtGoogle Scholar
- Hubert B: Linux advanced routing & traffic control HOWTO. 1.43 edition, , 10 2003 http://lartc.org/Google Scholar
- Bettstetter C, Resta G, Santi P: The node distribution of the random waypoint mobility model for wireless ad hoc networks. IEEE Transactions on Mobile Computing 2003, 2: 257-269. 10.1109/TMC.2003.1233531View ArticleGoogle Scholar
- Gupta P, Kumar PR: The capacity of wireless networks. IEEE Transactions on Information Theory 2000, 46(2):388-404. 10.1109/18.825799MathSciNetView ArticleMATHGoogle Scholar
- Li J, Blake C, De Couto DSJ, Lee HI, Morris R: Capacity of ad hoc wireless networks. In Proceedings of the 7th Annual International Conference on Mobile Computing and Networking (MobiCom '01), 2001, New York, NY, USA. ACM; 61-69.View ArticleGoogle Scholar
- McCanne S, Leres V, Jacobson V: The tcpdump manual page. Lawrence Berkeley Laboratory, 6 89Google Scholar
- Gomez C, Garcia D, Paradells J: Improving performance of a real ad-hoc network by tuning olsr parameters. Proceedings of the 10th IEEE Symposium on Computers and Communications (ISCC '05), June 2005, Murcia, Cartagena, SpainGoogle Scholar
- Huang YC, Bhatti SN, Parker D: Tuning olsr. Proceedings of the 17th Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC '06), September 2006, Helsinki, FinlandGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.