- Open Access
New indoor navigation system for visually impaired people using visible light communication
EURASIP Journal on Wireless Communications and Networking volume 2013, Article number: 37 (2013)
In this study, we propose an indoor navigation system that utilizes visible light communication technology, which employs LED lights and a geomagnetic correction method, aimed at supporting visually impaired people who travel indoors. To verify the effectiveness of this system, we conducted an experiment targeting visually impaired people. Although acquiring accurate positional information and detecting directions indoors is difficult, we confirmed that using this system, accurate positional information and travel direction can be obtained utilizing visible light communication technology, which employs LED lights, and correcting the values of the geomagnetic sensor integrated in a smartphone.
Pedestrian support for visually impaired people involves the use of textured paving blocks, guide dogs, GPS-based voice navigation systems, among others. On the other hand, studies aimed at visually impaired people report that there is a need for voice information inside buildings , and that, in the future, we will need adequate indoor pedestrian support systems in large commercial facilities, such as shopping centers and underground shopping malls. However, compared to public spaces and transport facilities, no progress is being made in providing commercial facilities with textured paving blocks , and although guide dogs are effective on obstacle-free safe walkways, they cannot locate a person’s destination. Moreover, the GPS’ inability to give an accurate position indoors is equally a problem. Therefore, positioning methods using radio waves emitted from a wireless LAN access point are increasingly employed. However, this method has encountered issues with fluctuating positional accuracy due to reflected signals from the wireless LAN, obstacles, or the surrounding environment . Studies into guidance systems using tactile maps, which are effective in creating mental maps, are also underway. However, it takes time to understand a tactile map by touch, and therefore, they are difficult to use while on the move. To address these kinds of issues, our study aims to create a usable system that enables visually impaired people, especially the blind, to travel indoors unaided. A variety of data is required to travel indoors, such as the accurate current position, travel direction, distance to the destination, and information about the barriers and surroundings. This study concentrates on solving problems relating to the current position, the travel direction, and the distance. As an indoor positioning method, we focussed on visible light communication technology using the ubiquitous LED lighting. As LED lighting is often installed in pathway ceilings, accurate positional information can be sent naturally from above the user’s head. Further, we used the geomagnetic sensor in the already widespread smartphone for a method to survey the travel direction. However, because of situations where geomagnetic sensors cannot detect the accurate direction due to the effect of, for example , rebars, and because it has been reported that they are not adequate to maintain the correct direction when walking long distances [5–7], we worked on simply improving the directional accuracy by obtaining and correcting the geomagnetic information beneath the LED lights in advance.
This article is structured as follows. In section 2, we present the necessary components of an indoor navigation system, namely, visible light communication technology and indoor map data, and then, we will discuss in detail the correction method for the pedestrian’s position and direction detection. In section 3, we present the design and implementation method of the indoor navigation system, and in section 4, we present the test results for this indoor navigation system. Section 5 discusses this system, and section 6 presents the conclusions and future research.
2. Our approach
In this section, we describe the positioning method and the method for creating indoor map data through visible light communication, components of our ‘indoor navigation system’ and the direction detection method through geomagnetism.
2.1 Positioning for visible light communication
As mentioned previously, methods used for indoor positioning make estimates from the position or radio wave strength of the wireless LAN access point or use wireless active tags. However, these methods have issues with fluctuating positional accuracy due to environmental factors such as obstacles. This makes them difficult to use as a support for visually impaired people in their travels. Therefore, we have focussed on visible light communication technology using LED lights. When using visible light communication, it becomes possible to identify the user’s position within a range of 1–2m. Moreover, visible light communications technology enables users to find out which floor they are located on. The indoor positioning system by visible light communication, Wireless LAN and, RFID which use is being investigated, are shown in Table 1[8–10]. Further, this is shown concretely in Figure 1.
Visible light communication is a communication method that uses light perceptible to the human eye, and being already standardized as a communication method by Japan Electronics and Information Technology Industries Association in Japan, it can be used as a visible light ID system (CP-1222). The specifications of the visible light ID system are shown in Table 2, but for this study, we somewhat improved the system. Information can be embedded directly into the visible light ID system and sent or the positional information, for example, latitude and longitude can be obtained through an external server after a unique ID has been sent and received.
2.2 Creation of indoor map data
It is difficult for visually impaired people to look directly at maps, but accurate indoor map data are necessary to calculate and provide accurate positional information, travel direction, and distance. Indoor map data can also be represented in a local coordinate system that does not use geographic references. However, indoor map data, beginning with map portal sites, are often expressed using a geographic coordinate system. When considering outdoor–indoor seamless navigation, for example, integration with a geographic coordinate system is preferable. For that reason, we decided on a geographic coordinate system for the indoor map data used in this study. Indoor map data are built with the method for mapping each floor plan managed by the Legal Affairs Bureau, as proposed by Nakajima . The LED lights from nodes and network data linking the LED paths are created on the basis of the voice guidance’s need to retrieve the route to the destination. The small round points in Figure 2 right are LED lights.
2.3 Compensated geomagnetic sensing
For example, in car navigation systems, the direction in which a user is travelling is detected from GPS and the acceleration sensors. For travelling indoors, Nordin and Ali  propose a method to infer the travel direction using node information. However, both methods have a drawback in that the direction is not known unless the user starts travelling. Therefore, we decided to show the travel direction by calculating the smartphone’s orientation, using its embedded geomagnetic sensor. We also decided to simultaneously calculate the distance from the indoor map data. The procedure by which the travel direction and distance are calculated is shown in Figure 3.
First, the geographic coordinates are obtained from the visible light ID sent from an LED light. Second, the geomagnetism beneath the LED light is obtained from the geomagnetic sensors. Geomagnetic sensors show the absolute orientation by detecting the direction of the earth’s magnetic field lines, but as mentioned previously, we know that the magnetic field can be distorted in indoor spaces with rebars, etc. Therefore, we obtain the geomagnetic data beneath the LED light in advance and make corrections using these values. Third, we obtain the next LED lights’ geographic coordinates from the route to the destination and calculate the angle of the travel direction and the distance.
3. Development of indoor navigation system
In this section, we present the design and implementation method for the indoor navigation system.
3.1 Indoor navigation system design
We designed the indoor navigation system using the component technology mentioned in section 2. As shown in Figure 4, the system is composed of LED lights, a smartphone with integrated receiver and headphones. For obtaining positional information and calculating the route retrieval, both necessary for the indoor navigation system, we use a cloud environment from Panasonic that is referred to as the positional information base. The visible light ID is sent from the LED light using visible light communication and is received by the receiver. Then, the smartphone obtains the visible light ID from the receiver using Bluetooth. The smartphone receives the positional information, etc., from the positional information base through Wi-Fi, and the positional information and the guidance content are combined in audio files using a speech synthesizer system (Table 3) and sent to the headphones.
The overall system sequence is shown in Figure 5. First, the user long-presses the smartphone screen, using the touch screen, to indicate the start of the voice navigation The smartphone requests the positional information for the whole floor surface from the positional information base and on receipt informs the user of the destinations with their corresponding numbers. Second, the user inputs his destination number using the speech recognition function, the smartphone requests the route from the positional information base and, on receipt, informs the user of the entire route. Third, the user starts to travel, and when the system receives a visible light ID from an LED lamp while the user is travelling, it calculates, in the sequence shown in Figure 3, the travel direction and the distance and informs the user by combining this in speech as shown in Table 3. If the user has strayed from the route, the user is again informed of any route changes. This is repeated until the destination is reached, upon which the user ends the guidance by pressing the smartphone screen for a long time.
3.2 Implementation of indoor navigation system
We implemented the system based on the design described in section 3.1. We used Eclipse as a highly compatible integrated development environment as it needs to run on the Android OS, as shown in Table 4, and we developed the application using the Android Development Tool and JDK6. In the verification test, we used a Panasonic P-07C smartphone. We further prepared an additional smartphone for the test, created a visible light ID data frame as a substitute transmitter according to this study’s specifications and transmitted using Bluetooth.
4. The verification test and result
In this section, we present the verification test and document the verification log analysis results and the interviews with the test subjects.
4.1 The verification test
First, we mounted LED lights transmitting visible light IDs on the ceiling. As shown in Figure 6, we set the illumination range height to be at 1400mm from the floor and 1076mm across, and we enabled receipt of the visible light ID within this range. Moreover, we measured, at a height of 1400mm from the floor beneath the LED light, the azimuth based on the true north using a magnetic compass.
Then, we measured the geomagnetism beneath the LED lights and identified the geomagnetic distortion ranging from 40° to −157° (Figure 7). We decided to provide guidance by calculating the travel direction with these values as the correction values.
Then, we asked visually impaired people to try the system, and while obtaining logs from the visible light ID receiver and the geomagnetic sensor, we also conducted interviews with the subjects. For the verification test, we gained the cooperation of the Osaka Municipal Association for the Welfare of the Visually Impaired and conducted the test with six visually impaired people. Their attributes and the test equipment, etc., are shown in Table 5.
The subjects had a smartphone with an integrated receiver around their neck and headphones on their ears. To make sure it was not difficult to hear external voices while travelling, we used open headphones with pieces of sponge attached to the headband so that the ears did not get covered (Figure 8).
The test course, as shown in Figure 9, started beyond the reception area (icon ‘S’) and a as goal, had to reach the multipurpose toilets (icon ‘G’). From the starting point, the course went straight ahead for 25m and then made a sharp left turn. Then, it then turned right after running approximately 20m straight ahead, with the multipurpose toilets 5m on the right side.
We conducted the test for one person at a time, with the users walking on their own from operation to destination. We first explained and then executed the procedure using the indoor navigation system, until the destination was reached.
While the subject was travelling, the sequence shown in Figure 3 was followed, based on the visible light ID and the geomagnetic values obtained from beneath the LED lights, and the subject was informed of the travel direction and distance. If the subject strayed from the route while travelling, the route was recalculated and the subject was re-informed.
4.2 Result of verification test
For the travel direction logs, we obtained the visible light ID and the geomagnetic values beneath the LED lights for each subject. The results, as shown in the graph of Figure 10, were identified the subject’s position within range of 1–2m and all corrected in the range from 66° to −59°. Essentially, the correction results on the graph are 0°, and the geomagnetic position values do not vary. However, when the subjects travelled with the smartphone suspended from a strap around their neck, making it swing more than anticipated, the position for which the geomagnetic value was obtained and the corrected value diverged, and hence errors occurred. There were further errors for each LED light position; with the smallest error range being for the position in which the visible light ID ‘031103’ was received, they ranged from 10.54° to 9.47°. We also established that there was an error variation for each subject. The person with the most stable error was subject F, with errors ranging from 13.89° to −16.62°. On the other hand, subject E had the highest variation ranging from 66° to −37°.
Test results showed that three people with low vision reached the goal. Three blind people arrived at the goal by checking the position of the wall along the route with their cane a few times.
After the test, we conducted interviews with all subjects.
These covered various issues, but for this article, we will limit the discussion to the guidance functions relating to position, travel direction, and distance (Table 6). People with low vision expressed opinions about the overall route and distance, while blind people voiced opinions about the travel direction and the need for a more accurate match with the timing of the guidance.
In this section, we will discuss the indoor navigation system, particularly the positional information and the travel direction functions. We will also discuss the results of the interviews after the test.
5.1 The positional information and the travel direction
We reduced the error range through geomagnetic corrections and provided more guidance accuracy for the travel direction. However, apart from the LED light ‘031103’, the lights had errors greater than equal to ±15°, and we could not consistently provide correct travel direction guidance. As previously mentioned, the reason lies, in addition to spurious noise, in the fact that the subjects travelled with the smartphones suspended from a strap around their necks, and that with the swinging, the obtained geomagnetic sensor values changed. It is conceivable that the smartphones swung excessively at the spot where the visible light ID ‘030204’ (which had a particularly large error) was received, because the subjects were close to where they made a few successive turns. To confirm this inference, the authors, after the test, travelled the course with the smartphone held still in their hands. As shown in the results in Figure 11, the geomagnetic errors settled in the 8° to −29° range. Realistically, it would be difficult to travel while keeping a smartphone still and holding a cane. We believe this can be dealt with by fixing an optical sensor to, for example, a belt, in a manner that would not obscure the sensor from visible light, and by keeping the smart phone in a pocket and transmitting using Bluetooth.
In addition, although this was not an issue with this experiment, there is the possibility of situations occurring where the magnetic field changes and the correction does not function because of floor alterations or the presence of magnetic materials in the actual space. To solve this problem, geomagnetic values need to be obtained on these occasions, which is a hurdle in actually applying the system practically.
5.2 The results of the interviews
Because an error in the direction equivalent to a one-hour difference in the clock position is a big problem for visually impaired people and especially the blind, we concluded that a higher directional accuracy is needed. In addition, because there is an individual variation in the guidance methods for the travel direction, a significant improvement could be made by employing a configuration that allowed the user to choose, apart from clock positions, modes for guidance using, for example, eight directions or angles.
On the other hand, as some visually impaired people walk fast, the timing of the spoken navigation needs to be matched as closely as possible to the travelling speed. We need to consider a configuration where guidance is given not at a bend, but approximately 1–4m before the bend. Moreover, a feedback sound is played when an LED light is correctly passed, but, because blind people usually feel uneasy about whether they are walking a route accurately, we also want to consider a configuration that provides a continuous feedback sound while travelling.
We developed and tested an indoor navigation system for visually impaired people using visible light communication that makes use of LED lights and a geomagnetic sensor integrated in a universally used smartphone. To support travel for visually impaired people, accurate guidance for the positional information and travel direction are needed, and we have confirmed that the positional and directional accuracy improves through visible light communication and geomagnetic value correction. Therefore, we have concluded that our approach will be effectual system for the visually impaired people. An issue to consider in the future is the need to establish an azimuth accuracy detection method.
Ministry of Health, Labour and Welfare: Ministry of Health, Labour and Welfare, “Report: About the voice-based navigation system for visually impaired people. 2009. , Online, Accessed Mar. 2012. (in japanese) http://www.mhlw.go.jp/bunya/shougaihoken/cyousajigyou/jiritsushien_project/seika/research_09/dl/result/08-09a.pdf
Ministry of Land, Infrastructure: Plan for the conduct of Ex-post Evaluation, 2009. , Online, Accessed Mar. 2012. (in japanese) http://www.mlit.go.jp/common/000056483.pdf
Kolodziej KW, Johan H: Local Positioning Systems LBS Applications and Services. Boca Raton, FL: CRC Taylor & Francis Group; 2006:87-164.
William S, Jeremiah S, John R: Magnetic field navigation in an indoor environment, Ubiquitous Positioning Indoor Navigation and Location Based Service (UPINLBS). Kirkkonummi, Finland; 14-15 October 2010:1-10.
Liu X, Makino H, Mase K: Indoor location estimation using visible light communication: practicality and expandability, 2010 International Conference on Indoor Positioning and Indoor Navigation, Abstract Volume. Switzerland: ETH Zurich; September 15-17 2010:407-408.
Makino H, Ito D, Nishimori K, Kobayashi M, Wakatsuki D: Pedestrian indoor positioning method using fluorescent light communication and autonomous navigation, 2010 International Conference on Indoor Positioning and Indoor Navigation, Abstract Volume. ETH Zurich: Switzerland; September 15-17 2010:403-404.
Muhammad Haris A, Valérie R, Gérard L: Magnetic field based heading estimation for pedestrian navigation environments, 2011 International Conference on Indoor Positioning and Indoor Navigation. Guimarães, Portugal; 21-23 September 2011:21-23.
Hung-Huan L, Yu-Non Y: WiFi-based indoor positioning for multi-floor environment, TENCON 2011–2011 IEEE Region 10 Conference. Bali, Indonesia; 21-24 November 2011:597-601.
Brown DWR, Dunn DB: Classification schemes of positioning technologies for indoor navigation, Southeastcon, 2011 Proceedings of IEEE. Nashville, USA; 17-20 March 2011:125-130.
Hui L, Houshang D, Pat B, Jing L: Survey of wireless indoor positioning techniques and systems. IEEE T. Syst. Man. Cy. C. 2007, 37(6):1067-1080.
Nakajima M: Path planning using indoor map data generated by the plan view of each floor, 25th International Cartographic Conference. Paris, France;3–8July 2011. ISBN:978-1-907075-05-6, CO-103
Nordin MJ, Ali AM: Indoor navigation and localization for visually impaired people using weighted topological map. J. Comput. Sci. 2009, 5(11):883-889.
The authors would like to thank Osaka Urban Industry Promotion Center, Panasonic Corporation and Osaka Municipal Association for the Welfare of the Visually Impaired.
The authors declare that they have no competing interests.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.