Online Human Daily Activity Recognition with Rechargeable Wearable Sensors

This paper describes an online human physical activity (PA) recognition system based on machine learning, using rechargeable wireless wearable sensors with body-energy harvesting. The entire system is introduced and described, including the wireless wearable sensor network with a control center (smartphone), a body-energy harvesting module as the power supply for the sensors, and a PA recognition model based on the random forest algorithm. Hardware design, software design, and algorithm design are described in detail. For the hardware design, there are two main parts: the low-power wireless wearable sensors using Bluetooth 4.0 BLE, and the energy harvesting module. Two experiments were carried out to validate the design. One was an energy harvesting and consumption experiment on the wearable sensors, which determined that harvesting human body energy to supply the wearable sensors is feasible. The other was online human PA recognition based on the rechargeable wearable sensors. According to the results, the body-energy harvesting module has almost no influence on the recognition accuracy, and using the human body movement energy harvesting system for online human PA recognition is practical.


Introduction
Physical activity (PA) is an increasingly important factor in maintaining human health. Therefore, scientific and effective human physical activity analysis is of great significance to human health. With the development of accelerometers, gyroscopes, and other portable sensor technologies, health monitoring systems based on wearable human physical activity sensors have gradually become the focus of scientific research and technological innovation.
The research groups of Professor S. S. Intille at Massachusetts Institute of Technology and Professor P. S. Freedson at the University of Massachusetts used neural networks and other machine learning algorithms to study the measurement of PA based on single acceleration sensors. (1,2) However, sometimes different types of activities, such as ordinary walking and weight-bearing walking, cannot be effectively distinguished by only one accelerometer signal. (3,4) To obtain better accuracy in measurement, numerous research teams have studied PA measuring methods based on various types of sensors. Gao et al. compared the recognition accuracy of six kinds of wearable system under the same condition, and found that a monitoring system based on multiple sensors can achieve better recognition accuracy even if it uses a relatively lightweight algorithm. (5) Therefore, with enough sensors and data, PA recognition could be calculated by different machine learning algorithms.
Most wireless wearable sensors need batteries to supply power. Batteries must be replaced when exhausted, which is the most urgent problem to be solved. Even when using rechargeable batteries, the need for frequent charging still means the batteries do not last for a long time. The research in this paper is about PA assessment using wearable sensors, and the energy produced when the human body moves. First, a body-energy harvesting module is required that collects body energy to provide a power supply for the sensors. When the human body begins to do some activity, the body-energy harvesting begins, and then the wireless sensors also begin to collect and transmit data. Then, the recognition system collects the sensor data and calculates the recognition results.
To address these problems, an online human PA recognition system with rechargeable wearable sensors was built. This paper describes the construction of a wireless wearable body sensor network based on the SensorTags of Texas Instruments (TI), which include several kinds of useful sensors and a low-power Bluetooth 4.0 BLE module. Then, a smartphone is used as a data logger and control center to receive data from the body sensor network, process the data, and exchange information with the online human PA recognition system. Bluetooth 4.0 BLE was selected as the communication protocol to construct the wireless wearable sensor network and communicate with the control center (smartphone). As the power supply of the wearable sensors, this paper describes a body-energy harvesting module that collects the energy produced by body movement as a power supply for the sensors.

System Overview
As shown in Fig. 1, the human daily activity recognition system has three components: a body-energy harvesting system, a wireless wearable sensor network, and an online daily activity recognition model.
It is very important to select an appropriate communication technology to construct a wearable wireless sensor network. Bluetooth 4.0 BLE is used as the communication protocol to construct this system. SensorTag of TI with Bluetooth 4.0 is selected as the basic unit of the wearable sensor network. A smartphone supporting Bluetooth 4.0 is selected as the control center. SensorTag and   Materials, Vol. 29, No. 9 (2017) 1355 the smartphone both support Bluetooth 4.0 BLE, so they can exchange data with each other at very low power consumption.

Sensors and
EH-C002 and CN3052A are the main chips in the body-energy harvesting system, working together with a rechargeable battery. KEH-C002 is a vibration power generation module, which can collect human body energy when it is worn on the body. The CN3052A chip, a special charge management chip, can use the energy generated by KEH-C002 to charge the battery to store energy. The battery can supply power to the wearable sensor network described in this paper.
Three wireless sensors attached to the chest, wrist, and ankle were used to acquire data. When a subject began to be active, body-energy harvesting collected energy and provided it to the power supply of the sensors. Sensors were able to collect the data of the PA and transmit it to the server. The online daily activity recognition model was running on the server. The server calculated the result of human PA recognition and sent the results to the smartphone for display on the screen.

Hardware design
The hardware of this system has two parts: a wireless wearable sensor network and an energy harvesting module.

Design of wireless wearable sensor network
For this system, it is very important to select an appropriate communication technology to construct the wireless wearable sensor network. At present, there are three kinds of wireless communication technologies commonly used: Zigbee, Wi-Fi, and Bluetooth. Taking the advantages and disadvantages of these three communication technologies and their ease of use into account, Bluetooth 4.0 is selected as the communication method. Bluetooth 4.0 combines Bluetooth communication technology, Bluetooth low-power technology, and Bluetooth highspeed technology into one. Bluetooth 4.0 is particularly suitable for indoor sensor use, and it has the advantage of low power requirement. More importantly, most smartphones support Bluetooth 4.0 and can receive the data transmitted from the sensors. Therefore, it is selected as the communication protocol of the wearable sensor network.
For the unit of the wearable sensor network, we chose TI's SensorTag. SensorTag is the industry's first Bluetooth low-power kit specifically developed for wireless sensor applications. It supplies a development kit for smartphone app developers. It is not only a reference design but can also be used as a development platform for smartphone accessories. SensorTag brings nearly all commonly used sensors together on a single board containing TI's thermometer, accelerometers, gyroscopes, humidity sensors, pressure sensors, and magnetometers. Its photograph is shown in Fig. 2.
TI's CC2541 is the main chip of SensorTag, responsible for signal collection, processing, and data transmission. TI describes the CC2541 as a power optimized, true system-on-chip (SoC) designed for Bluetooth low power consumption and proprietary 2.4 GHz application with advantages of high performance, low cost, low power consumption, and ease of development. CC2541 combines a radio-frequency receiver with outstanding performance, an enhanced industry-standard 8051 MCU, in-system programmable flash memory, 8 kB RAM, and many other powerful peripherals. CC2541 exchanges data with various types of sensors specifically through the HW I 2 C bus. It has three different power modes, which is ideal for systems requiring ultralow power consumption. (6) As Fig. 3 shows, CC2541 is equipped with 6 MEMS sensors: a gyroscope (IMU-3000), accelerometer (KTXJ9), magnetometer (MAG3110), hygrometer (SHT21), pressure sensor (T5400), and thermometer (TMP006). All sensor types selected are for smaller, low-cost patch devices. The I2C signal interface is used between the sensor module and the master chip. The sensors are connected to the same bus interface via separate enable signals. In addition, to minimize overall current loss in the circuit, all sensors are set to be disabled by default, so that they are in sleep mode before a test. Therefore, we chose a SensorTag integrated module based on Bluetooth 4.0 communication technology as the basic unit of the wearable sensor network after taking the circuit power consumption index, integration, and simplicity into consideration. Considering the practical value and economics of the sensor data, SensorTag is recommended to be worn on the chest, the dominant wrist, and the dominant ankle of the subject as shown in Fig. 1.

Design of energy harvesting module
There are a variety of problems with using a battery as the power supply for SensorTag. The replacement of the battery is the most urgent problem to be solved. The research in this paper addresses the daily activity of a person, and the body's movements can produce energy, which can be harvested to provide a power supply for the sensor. Therefore, a magnetic method was selected to collect the energy of the human body considered to provide the power supply for SensorTag. (7)(8)(9)(10) Finally, the KEH-C002 chip from the Beijing Micro-High-Core Technology Company of China was selected as a body-energy harvesting source. KEH-C002 is shown in Fig. 4. The KEH-C002 chip is a vibration power generation module, which can collect energy produced by body movements when worn on the body. The construction of KEH-C002 is shown in Fig. 5. (11) The performance parameters of the KEH-C002 chip are shown in Table 1. (12) However, the energy generated by the movement of the human body is small and fragmented, so a special charge management chip is needed to charge the battery before the energy in the battery can be used by SensorTag. The charge management chip used in this design is the CN3052A chip from Consonance Electronic Company. Its electrical parameters are shown in Table 2.   The charging process for a CN3052A chip is shown in Fig. 6. When the battery voltage is less than 3 V, the charger uses a small current (a trickle charge) to precharge the battery, where the current is constant and its value is 10% of the constant-current charge (500 mA). When the battery voltage exceeds 3 V, the charger uses a constant-current mode to charge the battery; the charging current is determined by the resistance between the ISET pin and GND RISET. According to the user's manual, the current value for constant-current charging is 500 mA. (13) When the battery voltage is close to the battery-side modulation voltage, the charging current is gradually reduced, and CN3052A enters the constant-voltage charging mode. We considered a charging link for the trickle charge. Consequently, when the chip reaches its operating voltage (4.35-6 V), the battery charges with a constant current of 50 mA. The entire system is shown in Fig. 7.

Design of embedded software for sensors
As temperature and humidity sensors, magnetometers, and pressure sensors have relatively low correlation with human behavior, our application relies mainly on the use of the accelerometer and gyroscope: two sensors which have a strong correlation. Therefore, we turned on the accelerometer and gyroscope and disabled the other sensors. In real life, the frequency of most human activities is not higher than 20 Hz, so in this study, in the information acquisition hardware module, the sampling frequency of the accelerometer was set to 30 Hz. In addition, the gyroscope sampling frequency was set to 30 Hz to reduce the difficulty of the synchronized processing of data from both the accelerometer and gyroscope. The data collection by the accelerometer and gyroscope in the hardware module and the main program for Bluetooth communication mainly use the IAR-embedded workbench (IAR Systems Software Inc.) integrated development environment (IDE) as a development tool. Based on the framework of the original Bluetooth 4.0 protocol in the CC2541 main chip, the specific needs of monitoring the daily activities of the human body were taken into account in designing and writing the main program. The overall operating logic of the program is shown in Fig. 8.

Smartphone software design (PA recognition feedback)
The human daily activity information collection module was run on TI's BLEv1.2 Bluetooth protocol stack and belonged to the BLE peripheral slave device in the Bluetooth 4.0 link. A smartphone with Android 4.3 as the BLE central device was used on the receiving side.
Establishing a reliable communication connection based on Bluetooth 4.0 and receiving the relevant sensor data collected were the main tasks performed by the control center. We needed only to follow the BLE to establish the connection process and call the appropriate system functions to complete the program development on the smartphone side. The program flow chart is shown in Fig. 9.

Server-side software design
The server-side main program flow diagram is shown in Fig. 10. The running program deployed on the server is mainly used to receive the original information set for daily activities, to analyze and extract the recognition feature quantities, and to use the random forest (RF) algorithm to dynamically recognize the human activities. The server-side interaction with the smartphone is mainly based on the hypertext transfer protocol (HTTP). Specifically, we used JSP/servlet-related technology through Tomcat 7 to build a relatively simple Web application server for the relevant information exchange.  In addition, to carry out offline data analysis and subsequent experimental research, the analytical raw data information was stored in the corresponding database table. The structure of the data table is shown in Table 3. When a given amount of sensor data for human behavioral activities accumulated, data preprocessing, recognition of feature extraction, and training of the classification model were carried out.

RF algorithm
RF (14) is chosen as the key algorithm for the PA recognition system, which is an ensemble learning algorithm for classification, as shown in Fig. 11. RF operates by building some decision trees at the training phase and outputting the class, which is the mode of the classification or regression of the individual trees. In the RF, every single tree depends on the values of a random vector, (15) which is sampled from the same distribution for all the trees in the forest. For the growing trees, the number of decision trees and the number of input variables chosen by each node are the most important variables. The training datasets for building every decision tree are obtained by sampling N (the number of cases in the original data) times at random. For classification, the input vector is presented to every decision tree in the forest. The final result is determined by the maximum voting among the trees. In this paper, according to the instances and features, we selected 10 as the number of trees in the forest and 7 as the number of random features.
Three sets of raw data were obtained from SensorTags on the wrist, chest, and ankle. These data were combined and seven different data sets were generated. (16)(17)(18) Mean, variance, standard deviation, skewness, and kurtosis were extracted as five time-domain features, and energy and entropy were extracted as two frequency-domain features. (19,20) These selected features were used as original input features for the model algorithm and selected once again in the  Fig. 9. The whole mobile-side program. Fig. 10. The server-side main program. internal RF algorithm. Then seven base classifiers with different weights were achieved for the ensemble learning, and instance-specific weighted majority voting was used to ensemble these base classifiers. Since the classifiers based on the different sensor sets usually have different recognition accuracies for the different PA types, it is not reasonable to assign a consistent weight for each classifier. Therefore, for each of the classifiers, according to its estimated confidence on the diverse PA types, the different weight of every single PA type can be obtained with 10-fold cross-validation. Then the final result is decided by the seven base classifiers using weighted majority voting. This instance-specific weighted majority voting considers the different identification accuracies of the different classifiers for different PA types and is more reasonable than the normal weighted majority voting. More details about this algorithm can be found in our previous work. (13,15)

Energy harvesting and consumption experiments with wearable sensors
TI's SensorTag CC2541 chip with Bluetooth 4.0 was selected as the basic unit of the wearable sensor network. CC2541 has 3 power modes: PM1, PM2, and PM3. (21) PM1 and PM2 are the normal work modes with power consumption of 23.9 µA. When SensorTag is in connection mode, it will turn to PM3 with power consumption of 0.4 µA.
The next step was to determine how much energy KEH-C002 could collect. In this experiment KEH-C002 was worn on the wrist of the experimenter, and the voltages were collected for different movements of the human body. By connecting KEH-C002 to the oscilloscope, the voltage could be measured. The setup of this experiment is shown in Fig. 12.
Four movements were simulated in the lab while wearing the body-energy harvesting module on the dominant wrist of subject. Connecting the body-energy harvesting module to the oscilloscope, the voltage data generated during these movements are shown in Fig. 13. The last step was assessing how much energy could be stored in the battery depending on CN3052A. Based on the preceding discussion, we considered our charging link for the trickle charge. The chip's operating voltage was 4.35-6 V, so when the chip reached the operating voltage, the battery charged with a constant current of 50 mA, which was 10% of the value of the constantcurrent charge. The data of voltage values from the oscilloscope were analyzed and calculated to obtain the result. The results are shown in Table 4.
The data in Table 4 show that strenuous movements can provide an amount of energy far greater than that consumed by the energy SensorTag. Because this energy can be stored in a battery, it can be used later. Therefore, it is shown that using this method to provide power for the SensorTag is workable.

Human PA recognition based on rechargeable wearable sensors
To verify the algorithm, a body sensor network with three rechargeable wearable sensors is used, and five PA experiments with four subjects are conducted.
A diagram of the experiment is shown in Fig. 14. The data sets were divided into training and test sets by random sampling. Training sets accounted for 80%, while test sets were the remaining 20%. The training data sets were split into seven data sets S(m) based on the position of the sensor on the body. The value of the T parameter was 10, and the final integrated classification model H final (x) was obtained by training according to the steps of the program shown in Fig. 8. The voting weight was decided by the accuracy of the movement recognition. Integrated fusion of the recognition results was carried out on this model based on the weighted majority voting method, which improved the final recognition performance significantly.  Due to the use of the body-energy harvesting module, we should consider if the body-energy harvesting module will influence the recognition accuracy. As the last experiment proved, the energy that the body-energy harvesting module stored is enough for the whole experiment. Thus, the main influence is the noise generated by the vibration of KEH-C002. A pair of contrasting experiments were carried out to investigate its influence on recognition accuracy. In one experiment, participants wore the wearable sensor network without the body-energy harvesting module, and in the other, the participants wore the wearable sensor network with the body-energy harvesting module.
There are four subjects in this experiment. Their basic information is shown in Table 5. Five kinds of daily activity were selected for the experiment: sitting, walking, walking upstairs, walking downstairs, and rope jumping. Each activity lasted 5 to 10 min. The results of this contrasting experiment are shown in Table 6. The recognition accuracy is the average value for the four subjects. Comparing the recognition accuracy of these two experiments, we can conclude that wearing the body-energy harvesting module has almost no influence on the recognition accuracy.
When subjects are sitting, the SensorTag signal is weak, but KEH-C002 does not vibrate significantly. Thus, there is almost no noise from KEH-002. When the subjects' actions are strong, the SensorTag signal is strong enough to ignore the noise from KEH-002. Thus, the noise from KEH-002 has almost no influence on the recognition accuracy.

Conclusion
A human daily activity recognition system based on the RF algorithm and a wireless wearable sensor network with a body-energy harvesting system was constructed in this study. We have provided an elaborate analysis of the hardware and software design of the entire system and the key algorithm. Energy harvesting from body movements and the consumption of energy by the wearable sensors were tested. The experimental results show that the body-energy harvesting system is reliable and practical. A pair of contrast experiments with 4 individuals and 5 PA types were carried out. The results show that the body-energy harvesting module has almost no influence on the recognition accuracy. The success of this experiment shows that harvesting the energy from the human body using new materials or devices can provide power to wearable sensors, thereby freeing us from replacing batteries or recharging batteries frequently.