Sensors in Network (4) —An Example: Human Activity Monitoring System—

In the previous sessions,(1–3) a general overview of sensor networks was given. In this session, I will focus on a system for monitoring human activities that can be attached to the human body, similar to adhesive plasters, as an example of an application of a sensor network. Because this system consists of various sensors, a microcontroller, a RF module, and a power source, the system can be regarded as a typical example of a sensor network. The specific requirements for this system compared to other sensor networks consider their weight, size, and shape. When the system is attached to the human body, the weight, size, and shape of the entire system greatly affects the comfort felt by users. To improve the comfort for users (that is, reduce the discomfort experienced when the users wear the system), we have made an effort to not only reduce the size, thickness, and weight of the entire system as much as possible but also to use multiple small printed circuit boards (PCBs), instead of using one large PCB, mounted on a flexible substrate.


Introduction
In the previous sessions, (1)(2)(3) a general overview of sensor networks was given. In this session, I will focus on a system for monitoring human activities that can be attached to the human body, similar to adhesive plasters, as an example of an application of a sensor network. Because this system consists of various sensors, a microcontroller, a RF module, and a power source, the system can be regarded as a typical example of a sensor network. The specific requirements for this system compared to other sensor networks consider their weight, size, and shape. When the system is attached to the human body, the weight, size, and shape of the entire system greatly affects the comfort felt by users. To improve the comfort for users (that is, reduce the discomfort experienced when the users wear the system), we have made an effort to not only reduce the size, thickness, and weight of the entire system as much as possible but also to use multiple small printed circuit boards (PCBs), instead of using one large PCB, mounted on a flexible substrate.

Why is the System a Plaster-Type?
Recently, various wearable devices have attracted attention. In particular, healthcare-related wearable devices have been intensively proposed to address problems resulting from an aging population, a declining birth rate, and an increase in lifestyle-related diseases throughout the world. (4) There are several ways of wearing such devices, including (1) putting them in pockets or bags, or attaching them to clothes using a clip, (2) wearing them as accessories such as watches, bracelets, glasses, rings, and earrings, (3) being worn in the shape of clothes or belts, (4) by attaching similar to adhesive plasters, and (5) by being embedded into the body, possibly in the future. Our system shown here aims to continuously and constantly monitor the biosignals of humans, with high accuracy, during their daily activities, including sleeping and bathing; therefore, (1)-(3) are inappropriate for our system because watch-and bracelet-type devices cannot be used to accurately monitor the posture and the amount of exercise; and clothes-type devices are as yet unsuitable for measuring biosignals during bathing and sleeping. However, plaster-type devices can be used in daily life until they are taken off. Such attachable devices can be used to easily and directly measure vital data such as the results of an electrocardiogram (ECG), volume of sweat, and body surface temperature, among others. Moreover, they enable coercive monitoring of the elderly and of infants if attached to a site where they cannot be freely detached by the wearers themselves (e.g., the back). From these viewpoints, we have carried out research and development of a plaster-type system. Of course, watch-and bracelet-type devices are of value for use as fashionable, gadgetlike, simple health monitoring tools, and it is not my intention to deny the potential of these devices.

Emergence of Plaster-Type Devices
For five years since 2008, we had studied MEMS technologies and devices for plaster-type devices in the Exploratory Research for Advanced Technology (ERATO) project supported by the Japan Science and Technology Agency (JST). (5,6) As of 2008, there was no concept of bodyattachable health monitoring devices. Since then, however, we have made efforts to promote this concept, in addition to researching and developing such devices. Recently, plaster-type devices have appeared commercially, particularly those developed by venture companies such as SensiumVitals (by Sensium Healthcare, Toumaz Group), HealthPatch (by VitalConnect), Metria (Vancive Medical Technologies), Nuvant MCT (Corventis), V-Patch (Vpatch Medical Systems), ZIO XT Patch (iRhythm), and Vitalgram (AffordSENS). Although some devices have yet to obtain general acceptance, since they are still under development, increasing recognition of the utility of attachable devices requires attention. Many of the attachable devices are designed using commercially and commonly available MEMS sensors, RF modules, and batteries in the commercial market, and are equipped with the standard functions of sensor network nodes. In the ERATO project, we designed and evaluated, from an academic viewpoint, new MEMS combo sensors and dedicated peripheral ICs that function with extremely low power consumption, new power generation devices, and peripheral MEMS-related technologies using novel materials. At the same time, we have fabricated prototypes of such devices as demonstration models using today's commercially available products, including MEMS sensors and RF modules. (7) These prototype devices are not based on cutting-edge technology, but they can be particularly applied to commercially available products in a short time. The details of our demonstration models are described in the following sections. Figure 1 shows (a) the appearance of the plaster-type system and (b) how to attach the system to the human body. The system comprises three PCB sections: (1) a RF module, (2) sensors and a microcontroller, and (3) a power source, which are electrically wired on a flexible substrate (polyethylene). Permanent magnets are attached to the back of both the RF and power PCBs, which connect to ECG electrodes. The maximum size of the system including the flexible substrate is 20 × 160 × 4 mm 3 , and its weight is 6.6 g. Ultimately, it is desirable to realize the entire system as a combo system (monolithic integration of sensors, microcontroller and RF) and minimize the cost to a level comparable to that of disposable systems, a future goal of the ERATO project. Our current demonstration model uses commercially available products, and its hardware is relatively expensive because of assembly costs. Thus, the model is not disposable but is used in combination with consumable plaster (adhesive plaster) with ECG electrodes. The system is attached to the human body as follows: (1) attach a consumable adhesive plaster with ECG electrodes to the chest (or back, or arms), (2) bring the demonstration model into contact with the electrodes so that the magnets on the back of the system can be electrically and mechanically connected to the ECG electrodes on the adhesive plaster, and (3) cover the entire system with a waterproof permeable film. By this method, users can take a bath and sleep without discomfort while the system is in operation. Figure 2 shows a block diagram of the system. This system is one of the prototypes of our demonstration models that we have fabricated using different sensors, processors, and RF modules. In this example, a 2.4-GHz proprietary protocol (Gazell, by Nordic Semiconductor, developed as a low-power-consumption RF protocol), is used as the RF module. The measurement targets are ECG, triaxial acceleration, body surface temperature, humidity, atmospheric temperature, and atmospheric pressure. A 2032 type 75mAh Li-ion button battery is used for the power source, and the microcontroller is an 8 bit processor. In the demonstration, the ECG waveform and triaxial acceleration were transferred at a sampling rate of 125 Hz, whereas body surface temperature, atmospheric pressure, humidity, atmospheric temperature, and battery voltage were transferred at a sampling rate of around 0.5 Hz. A dedicated RF device, which communicates to the plaster system by Gazell protocol, is required to receive data. Figure 3 shows a real time PC display of the data transmitted from the demonstration model. A dedicated data communication device (base station) is connected to a USB connector. In a practical system, multiple base stations are installed within the area of human activities (for example, one base station per room) and networked with each other to enable the constant communication of the system with any of the base stations within the area and realize seamless data acquisition.

Our Plaster-Type System for Monitoring Human Activities (Demonstration Model)
Because the sampling rates for both ECG and triaxial acceleration are 125 Hz, interrupts are generated by an internal counter every 8 ms, and a total of 5 bytes of data (2 bytes for ECG and 1 × 3 axes = 3 bytes for acceleration) are obtained within the interrupt routine. The payload for the RF module is fixed at 27 bytes, and data for 4 samples (20 bytes) are collectively transmitted once every four interrupts. The remaining 7 bytes are used for transmitting low-rate sampling data such as body temperature and for simple-check data to detect the lack of data during RF communication.  To examine a different application of the demonstration system, our system was embedded into a female adhesive silicone bra (Bragel International, Inc., patented product). Figure 4 shows the appearance of the bra and the system worn by a person. Because the PCBs are sufficiently thin, they can be almost completely embedded into the bra by cutting the silicone of the bra. Very thin polyurethane wires are used to electrically connect the PCBs. Moreover, electrodes are formed on the back of the adhesive layer of the bra to obtain ECG. For the position of the electrodes shown in the figure, ECG waveforms with a high signal-to-noise (S/N) ratio can be obtained. According to the user, there was no sense of discomfort during wearing.
Details of the actual data obtained are found in the reports of Maenaka (6,7) and are not reported here. Here, the quality and amount of data that should be transferred are discussed. For example, if the above-mentioned unprocessed raw data (125 Hz sampling) are transferred to the system and recorded, the amount of data per a day will exceed 50 Mbytes. If 100000 people (only ~1% of the population of Japan) use the system, the amount of data will reach 5 Tbytes per day and 2 Pbytes (peta bytes) per year. If the data are recorded in high-readability text format, the amount will further increase severalfold. Such data are a component of so-called big data; and since the data on human activities are highly personal information, the most secure protection of confidentiality is required. Thus, the processing of data will be very difficult. The range of data on human activities we obtain is wide and includes heart rate, the amount of exercise, posture, body temperature, and ambient conditions. Because there have been few reports on data processing for human activity data, for example, discussions between obtained data and actual health conditions or future health prospects, the question of how to treat and effectively use the various forms of massive big data is an important research theme that should be explored in the future.

Applications of Data Compression
Some applications may require the direct transfer and recording of unprocessed raw data such as the above-mentioned high-rate sampling data (ECG and acceleration data). In general, however, greatly reducing the amount of data is hardly considered to affect the understanding of human activities. (8) For example, recording the heart rate (beats per minute), instead of using a 125-Hzsampled ECG, will reduce the amount of data to approximately one-hundredth. If the number of steps or the mean exercise intensity and posture for a certain period [1 min may be appropriate because the unit of the exercise intensity called the metabolic equivalent of task (MET) is usually calculated for every minute] are used, instead of raw data from acceleration sensors, the amount of raw data can be reduced to one-ten-thousandth or smaller. This will greatly reduce the energy required for RF transmission; the working capacity of the system will increase for the same battery, or alternatively, smaller batteries can be used for the same working capacity, thereby reducing the size and weight of the system. The energy consumed during data transmission through the Internet (network routers and servers) can also be reduced owing to the amount of data reduced.
The use of an IC dedicated for signal processing (Application Specific Integrated Circuit: ASIC) is beneficial for the routine processing of such data. Much less power is consumed during signal processing by dedicated ASIC than by a microcontroller using software. In the algorithms for determining the heart rate from ECG waveforms, and for determining the exercise intensity and the number of steps from acceleration data, the use of a dedicated digital IC can reduce the power consumption to one-hundredth or one-thousandth compared with that used in processing by software. (9,10) That is, data is converted by the dedicated IC, instead of the microcontroller, to elongate the sleep time of the microcontroller, which can greatly reduce the mean power consumption of the entire system. This method is illustrated in Fig. 5. Although the design and fabrication of dedicated ICs require expertise and initial cost, compact field-programmable gate arrays (FPGAs) with ultrasmall power consumption are now available on the market and can be used to establish a system at the initial stage.

Further Reduction of Power Consumption
It may be possible, for example, to expand the intervals for obtaining data such as the heart rate and exercise level during sleeping or resting. In this case, the entire system can be in the sleep mode. However, if the entire system is in the sleep mode, a trigger for waking up the system is necessary. To wake up the system from the sleep mode when the user starts moving from the resting mode, motion sensors need to be working. External power for this constant working mode will be unnecessary if the system is equipped with a self-power-generating sensor. For example, a piezoelectric thin film [e.g., lead zirconate titanate (PZT); electric charges, or electric energy, are generated upon application of stress] may be used as the movement sensor. By increasing its output to a voltage level sufficient to directly wake up the system, (11) the peripheral circuits can be completely switched off, i.e., a normally-off system can be realized. We call this sensor a trigger sensor. The proposal and development of such novel devices is of value in terms of marked reduction in power consumption.

Summary
I have described an attachable system for monitoring human activities as an example of a sensor network. The device shown in this session uses a dedicated RF module for data communication. This is because we assume that the application of the device aims to monitor persons, and that the base stations to receive data are arranged over the entire spatial range of human activities (e.g., bedroom, living room, bathroom) as the infrastructure. This enables direct communication between the monitoring system and base stations and ensures constant communication without users having to carry coordinators (3) (i.e., users carry only the monitoring system). There are other applications in Fig. 5. Method of reducing power consumption (using our device as an example): (a) raw data are transmitted, (b) the ECG is converted into the heart rate and the acceleration data are converted into the number of steps and the amount of exercise, which reduces the rate of data transmission and thereby the power consumption during transmission, and (c) the data are converted by the ASIC, while the microprocessor is in sleep mode, greatly reducing the power consumption. which the users will always carry a mobile terminal such as mobile phone with them. In this case, mobile phones can be used as the coordinators connected to the monitoring system via Bluetooth Low Energy, a standard communication specification of mobile phones. No special hardware, such as base stations, will be required, and data can be directly transferred from the monitoring system to the mobile terminals.
It is expected that the data automatically obtained from sensors will account for the majority of the information accumulated in the cloud in the future. Biological data are expected to be one of the major components of the data. As of now, a large amount of data have been automatically obtained from systems similar to our system, shown in this session. Is is necessary to discuss the direction of sensor networks taking into consideration how to use the massive data, and the future prospects of big data processing and artificial intelligence (AI) in the future, Internet of Things (IoT) society.