Classification of Finger Movements for Prosthesis Control with Surface Electromyography

Surface electromyography (sEMG) signals can be used in the medical, rehabilitation, robotics, and industrial fields. In this paper, we assess a method of classifying finger movements for dexterous prosthetic hand control. The sEMG signals from five volunteers are recorded, and then pattern recognition is carried out by data preprocessing, feature extraction, and classification. The results show that high recognition accuracy can be achieved by time domain feature extraction and the use of an artificial neural network. To find the trade-off between the number of channels and the recognition accuracy, the number of channels is reduced, and it is found that the minimum number of channels required for high accuracy is seven, giving a recognition accuracy of 90.52%.


Introduction
In recent years, the research on surface electromyography (sEMG) prosthetic hands has been a hot topic in the fields of robotics and human-computer interaction. According to the National Center for Health Statistics, there are about 2 million amputees in the United States with an increase of 185000 new amputees every year. Among the amputations, wrist and hand amputations accounted for 10% of the upper limb amputations, while radial artery amputations accounted for 60% of the total number of wrist and hand amputations. Because sEMG signals can reflect the intention of movement before the actual movement occurs, (1)(2)(3) an sEMG prosthetic hand can help amputees to complete some tasks in daily life to improve their quality of life.
The decoding of sEMG signals generated by hand movements has achieved some results. (4) Kawano et al. used a support vector machine (SVM) to recognize six gestures. (5) Duan et al. used a discrete wavelet transform and wavelet neural network to analyze sEMG signals from three channels to classify six hand movements, and the classification accuracy reached 94.6%. (6) Motoche and Benalcázar used an artificial neural network (ANN) to analyze sEMG signals of five gestures collected by a Myo Armband in real time and achieved an accuracy of 90.7%. (7) Benalcázar et al. used the k-nearest neighbour algorithm (kNN) to recognize hand movements and achieved an accuracy of 86%. (8) Geng et al. used a convolutional neural network to recognize multiple gestures and achieved good results. (9) However, these studies mainly focused on the recognition of hand gestures. The dexterous control of a prosthetic hand also requires the recognition of the movement of a single finger. The control of finger movements is more challenging than that of gesture movements.
Some researchers have attempted to recognize finger movements controlled by dexterous hand prostheses. Jiang et al. classified six finger movements using four sEMG channels with wavelet transform characteristics. (10) Tenore et al. decoded 12 finger movements of five healthy subjects using 32 sEMG channels. (11) For an amputee, 19 sEMG channels were used, and only one time domain feature was extracted from each channel. A multilayer perceptron was used for classification. Kanitz et al. recognized 12 finger movements using time domain features derived from 16 unipolar electrodes with a genetic algorithm optimizer and an SVM classifier. (12) Castro et al. used five channels of sEMG signals to recognize six finger movements and achieved an accuracy of 97%. (13) Anam and Jumaily proposed a new extreme learning machine and a new dimension reduction method called the spectral regression extreme learning machine for finger movement recognition. (14) Five to eight finger movements were recognized using two-channel sEMG signals. In some of these studies, recognition performance was improved by increasing the number of sEMG channels, because a small number of channels can only recognize a small number of movements. However, owing to the limited forearm surface area of amputees, it is unrealistic to improve recognition performance by increasing the number of channels. (15) In addition, increasing the number of channels will also increase the complexity, weight, and cost of prosthetic hands. To be clinically relevant, ideal prosthetic hand control should be based on the least number of channels and the least computational complexity to achieve the highest recognition accuracy. (16) In this paper, the classification architecture of finger movements is studied for prosthesis control. The aim of the study is to improve the accuracy of recognition by carefully selecting the locations of electrodes, extracting more features, and comparing three pattern recognition schemes including SVM, kNN, and ANN. At the same time, the optimal scheme is used to solve the problem of selecting the minimum number of sEMG channels needed to achieve high recognition accuracy. Reducing the number of channels can avoid the information redundancy generated by a large number of channels.

Data acquisition
A total of five healthy subjects (four males and one female, aged from 22 to 26, all righthanded) volunteered for this study. The subjects were not trained before the test. At the beginning of the experiment, each subject sat in a comfortable chair and relaxed his/her arm.
The muscles of the human forearm are mainly divided into flexor and extensor. The anterior muscles composed of flexor are mainly located in the medial part of the forearm and are responsible for wrist, elbow, and finger flexion. The extensor mainly located in the lateral forearm is responsible for wrist and finger extension. Because most of the muscles responsible for finger movements are deep in the forearm, the acquisition of finger movement signals is more difficult than that of gesture movement signals. In this study, eight sEMG channels with paired self-adhesive Ag-AgCl electrodes were placed around the forearm. The distance between the electrodes was 24 mm. The bipolar electrodes can minimize crosstalk. Figure 1 shows the locations of the eight sEMG channels of the subjects. Before placing the electrodes, the forearm skin was wiped with alcohol to reduce skin effects, such as impedance, surface grease, and the dead cell layer.
A mini-DTS sEMG acquisition instrument from Noraxon (Noraxon, USA) with a sampling frequency of 1500 Hz was used for sEMG acquisition. The collected signals can be displayed in real time on the screen through wireless transmission, which is conducive to the observation of the collected sEMG signals during the experiment. The whole acquisition system is shown in Fig. 2.
The subjects were not trained for sEMG recording before the study and were told to perform 11 finger movements: thumb abduction, thumb extension, index finger extension, middle finger   extension, ring finger extension, little finger extension, thumb flexion, index finger flexion, middle finger flexion, ring finger flexion, and little finger flexion. Although the thumb itself can perform four movements, the muscles responsible for thumb adduction are located in the hand itself and cannot be decoded from the upper forearm, and only three thumb movements were included in the recognition. The 11 finger movements are shown in Fig. 3. The numbers below the figures represent the labels of the gestures. During the experiment, each subject sat in a chair in front of the computer to view all sEMG channel signals in real time while performing the actions. The subject's arms were fixed on the pillow, and a series of finger movements were produced with moderate and constant force. Each finger movement was maintained for 3 s with a rest for 5 s after each completion to avoid muscle fatigue. Each action was performed 36 times with 24 of them used for training and 12 used for testing.

Data processing
MATLAB 2018a (Mathworks, Natick, MA, USA) software was used for data processing. To find a better recognition method to evaluate the recognition performance, we compared three machine learning classifiers: kNN, SVM and ANN.
The raw signal obtained using the sEMG acquisition system is shown in Fig. 4. A fourthorder Butterworth band-pass filter, whose band-pass frequency ranged from 20 to 450 Hz, was used to filter the noise. The sEMG signals were generated 300 ms before limb movements. To ensure the real-time control of the prosthetic hand, the window length must be less than 300 ms. (6) Therefore, all recorded sEMG data were divided into overlapping windows with a length of 200 ms, and the increment between windows was 50 ms. Because the sEMG signal is recorded in the time domain, the time domain feature extraction is simple and the calculation time is short, which is conducive to the realization of real-time control, so the time domain feature extraction is one of the commonly used methods to extract signal features. (17) In our study, five time domain features are used for feature extraction: root mean square value (RMS), number of zero crossings (ZC), waveform length (WL), number of slope sign changes (SSC), and mean absolute value (MAV).
RMS is a characteristic associated with the frequency and amplitude of the sEMG signal. It represents the mean power of the sEMG signal and reflects muscle activity. The mathematical expression is  (1) where N represents the number of sample points in the window and f k is a data point in the channel.
ZC represents the number of times the signal passes through zero in a certain time. It can be used to observe the frequency of signal changes and display the number of signal symbols. The mathematical expression is where a represents the threshold, and WL is a characteristic reflecting amplitude. It is the cumulative length of the waveform over time and is related to the amplitude, frequency, and time of the waveform. The mathematical expression is SSC reflects the number of changes between positive and negative slopes. The mathematical expression is MAV reflects the characteristics of the amplitude and is equal to the average absolute value of the sEMG amplitude of N samples in the sliding window. The mathematical expression is In this study, we used the best recognition scheme determined in the experiment. Then, we determined the impact of the number of channels on the recognition performance by reducing the number of channels. The goal was to find the smallest number of sEMG channels that can achieve similar performance with all available channels. This will enable us to find the best trade-off between the recognition accuracy and the number of channels. A direct exhaustive search algorithm was not used to determine the number of channels in this paper because its calculation needs to investigate all possible channel combinations to reduce the number of channels, which leads to high computational load. On the contrary, the channel elimination algorithm used in the previous study (18) is less computationally intensive because it has the advantage of one-time recursive deletion of the worst-performing channel. During each iteration, the recognition accuracy was calculated and the channel that contributed the least to the recognition performance was deleted. Figure 5 shows the average recognition accuracy of the 11 finger movements with the three classifiers for the five subjects. It shows that the recognition performance of ANN and SVM is better than that of kNN, and the average recognition accuracy of kNN can only reach 80.81%. The recognition accuracy of ANN is slightly higher than that of SVM. The average recognition accuracy of SVM is 87.10%, whereas that of ANN is 91.10%.

Results
To better understand the recognition accuracy of the 11 finger movements by ANN, Fig. 6 shows the confusion matrix of subject A1. It can be seen that the 11 finger movements can be effectively recognized, and the recognition rate is mostly more than 90%. Figure 7 shows five subjects' recognition accuracies for different numbers of channels. This experiment illustrates the effect of the number of channels on recognition accuracy, so that we can use the least number of channels to recognize finger movements. From the figure, we can see that the recognition accuracy increases with the number of channels. When the number of channels is less than six, the recognition accuracy begins to decrease gradually with the decrease in the number of channels. When using seven channels, the recognition accuracy is more than 90%, and using seven channels achieves similar results to using eight channels. Therefore, Fig. 7 shows that there is no significant difference between the accuracy provided by seven sEMG channels and that provided by more sEMG channels.

Discussion
In this study, time domain features were extracted and the appropriate classifier was selected to provide the necessary information for recognizing a large number of movements. Three classifiers were evaluated and ANN achieved the best recognition results, whereas the recognition results of kNN were significantly worse than those of ANN and SVM. Compared with previous studies, this method has a higher N m /N ch ratio (where N m is the number of finger movements and N ch is the number of sEMG channels). Table 1 shows the results of previous studies and the N m /N ch ratio in the last column.
The recognition results are similar to those reported by Tenore et al. (11) who used one time domain feature and a multilayer perceptron pattern recognition method. For five intactedlimbed subjects, the accuracy was 93.3%. However, they used 19 sEMG channels to recognize 12 finger movements and increased the accuracy to 94.1% by increasing the number of channels to 32. In our research, we used five time domain features together with seven channels and achieved a recognition accuracy of 90.52%. Anam and Jumaily achieved an N m /N ch ratio higher than that in our work. However, they achieved a higher recognition accuracy of 95.67% with only five movements, and in the case of recognizing eight movements, the recognition accuracy decreased to 86.73%. (14)    In order to find the best seven channel locations, we followed the computational process of starting from multiple channels and then gradually removed the channel with the least contribution. Through the method, we confirmed that seven channels are suitable for finger movement recognition with an accuracy of 90.52%.

Conclusions
This paper presents a study on the use of multichannel sEMG to recognize finger movements for dexterous prosthetic hand control. We built an sEMG data set of five subjects. Through time domain feature extraction and ANN, 11 finger movements were recognized. The recognition accuracy reached 91.10%. Subsequently, the channel that contributed least to the recognition accuracy was successively eliminated to find the best trade-off between the number of channels and the recognition accuracy. It was found that the average recognition accuracy for seven channels is 90.52% with no significant improvement with more channels. Therefore, it is of great significance to prosthesis control that the high recognition accuracy of finger movements with a small number of channels and our future work will be focused on applying our results and conclusions to actuated prosthesis for real-time display.