The Knowledge Science Laboratories at the Advanced Telecommunications Research Institute International (referred to below as ATR) is working on an "E-Nightingale Project" to develop a wearable system. The aim is to understand nurses' every day activities in order to prevent failures of communication and other potential incidents in the high-pressure medical workplace. Research and development for the project is currently underway with cooperation from Tokyo Women's Medical University and the Kansai Electric Power Hospital. The project requires a wearable main unit that is small enough so that it doesn't interfere with nurses' work, yet has a high level of processing capacity and can operate continuously for eight hours. For this reason, the project selected to use a Renesas SH-Mobile processor that is widely used in mobile phones.

Conducting ongoing research into knowledge-sharing systems based on an understanding of daily activities and their surrounding situations

The Advanced Telecommunications Research Institute International (ATR) was established in 1986 and opened its main laboratories in 1989 in Kansai Science City "Keihanna." Since its foundation ATR has worked actively on developing new communication technologies to support comfortable lifestyles in the future by engaging in fundamental and original research in the field of telecommunications. The role of its Knowledge Science Laboratories is to establish the fundamental technology to support our everyday processes of creating and sharing working knowledge for our everyday lives.

Those technologies are especially needed in applications in the hospital wards, where the medical accidents and incidents are one of the critical concerns. The E-Nightingale Project was thus started centering around the activities of the nurses in the medical facilities. The research is supported by the National Institute of Information and Communications Technology.

"Our research started with the aim of capturing and analyzing people's daily activities to understand how they spent the day . Nursing was selected as a specific area for the research, and we have developed a system for focusing on the activities of nurses," explained Dr. Kogure. The research team is now at the stage of conducting trials at actual medical facilities. They are working with Tokyo Women's Medical University and the Kanasai Electric Power Hospital as research partners.

Three technologies to be developed

The E-Nightingale Project is developing three fundamental technologies for [1]understanding activities and conditions, [2] constructing knowledge, and [3] provideing knowledge.The first technology involves recording and analyzing data on nurses' activities as they perform their duties. The second one consists of building a knowledge base based on the analysis results. The third one involves providing adequate pieces of knowledge with people who need them. The expected results from the project include a system for recording and analyzing nursing work, a system for just-in-time nursing advice, or a system for creating incident documentation videos.

The configuration of the E-Nightingale wearable system is shown in Figure 1. The Renesas SH-Mobile 3 application processor was selected to be the "brains" of a wearable main unit (see Photograph 1) that nurses attach to their waists or elsewhere on their body. The unit gathers real-time data and thus is a key part of the trial.

Attaching 3-axis accelerometers to four different parts of the body

The wearable main unit and small wireless accelerometers are designed to be attached to the nurse's body to record their activities as they perform their duties. The main unit supports both Bluetooth (2 channels) and wireless LAN communications. Its features include an audio data input and an SD card interface.

The small and slim wireless accelerometers (38mm x 39mm x 10mm: see Photograph 2) measure acceleration in three axes. They are attached to, for example, both upper arms, the left chest pocket, and the back of the waist. The acceleration data varies according to the task being performed. For example, taking medicine from a medicine cabinet may involve the arms moving a large vertical distance while the chest and waist movements remain horizontal.

The wearable main unit and small wireless accelerometers form a personal area network (PAN) that connects via Bluetooth. This allows the data from each sensor to be collected by the wearable main unit. This allows the data from each sensor to be collected and sent to a server over the wireless LAN by the wearable main unit.
"One of the major design issues of the data collection system is that it can be worn all day without getting in the nurse's way. That is why we choose to use this wireless personal area network," Dr. Kogure said.

Typical nursing activities might include working at a desk, dispensing medicine, fitting drips, walking to a patient's bedside, or moving patients by wheelchair. The project was able to classify these activities in accuracies of from 80 to 90% using features extracted from the nurses' movement data collected by the small wireless acceleromaters (see Figure 2).

Microphone can be used to record pulse and respiratory sounds, as well as speech

The nurses also wear a microphone to record all their conversations. "Many medical errors are the result of failures in communication and related problems. Therefore, we ask the nurses to explain what they are doing when something prompts an action. Then we record everything they say," explained Dr. Noma. To identify which nurses were talking to each other, the project also devised a proximity-detection technique. It estimates the distance between wearable main units or other Bluetooth devices located nearby by monitoring the receiving signal strength. This is the reason that the wearable main units have two Bluetooth modules.

The project also investigated using a special type of microphone, called a non-audible murmur (NAM) microphone. Developed by the Nara Institute of Science and Technology, this microphone is attached to the skin just below the ear and is capable of capturing not just ordinal speech, but also the sound of the wearer's pulse and breathing. "Analysis of the sound picked up by the NAM microphone has been shown to measure variations in pulse rate with a level of accuracy similar to that of a electrocardiogram. The amount of stress that nurses feel can be estimated to some extent by this method. The microphone also provides information such as breath timing, which is also closely related to these matters," Dr. Noma explained.

The researchers have also devised a method for recording which nurses are passing through which doors, and at what time. IrDA-based infrared entry sensors on either side of the door provide very detailed records of activity. The data can even show how the nurse went through a doorway; that is, whether the person walked through it or ran through it.

Gaining an overwhelming advantage from the performance and low power consumption of the SH-Mobile processor

An essential requirement in the development of the wearable main units is that they have to be able to operate for more than eight hours after being attached. The units also required a high level of processing performance, sufficient to receive data continuously from wearable sensors such as microphones and accelerometers and to pass this data on to the server via the wireless LAN. "Considering the conflicting requirements of high processing performance and low power consumption needed to execute a heavy processing load continuously for eight hours, selecting the Renesas SH-Mobile 3 device as the CPU was our decision. In other words, the high processing performance and low power consumption it provided gave us an overwhelming design advantage. The ability to operate in a mobile environment with a wireless LAN for eight hours is certainly remarkable," noted Dr. Kogure.

The project team was impressed by the features of the SH-Mobile 3 processor, including its wide range of on-chip peripheral functions, which eliminate the need for external chips, and the 32MB of internal memory, which provides more than enough memory space. "We also used an operating system and driver software from Renesas. The specification documents are in Japanese and an excellent development system is available that includes the HEW (High-performance Embedded Workshop) integrated development environment. Because HEW has a graphical user interface very similar to a PC, our development team had few problems familiarizing themselves with the system," commented Dr. Noma.

Microcontrollers in the Renesas H8 family were chosen for the small wireless accelerometers to meet the requirement for low power consumption. The sensors should be able to operate for the same length of time as the wearable main unit.

The E-Nightingale Project plans to add further functions that make full use of the processing capacity of the SH-Mobile device. They will use for processing the audio data in the wearable main unit to reduce the volume of communication data, and for storing data in the unit whenever it can’t make a wireless connection to the server. Having attracted attention for providing a conceptual model of a truly practical wearable computer, this important research project is close to helping make significant improvements at the front line of the medical workplace.

top

See Product Information