Action recognition and control of mechanical simulated arm: electromyographic signal detection

. Electromyography (EMG) signal contains a large amount of human motion information, which can be used to classify human actions. In this study, based on the detection of surface electromyography (sEMG) signal, three actions were designed, the sEMG signal was collected by the EMG acquisition system. Four feature values, including root-mean-square value, average absolute value (MAV), wavelength, and Zero crossing point, were extracted from the signal. Then these values were taken as the input of Back-Propagation neural network (BPNN) to recognize different actions, thereby realizing the real-time control of mechanical simulated arm. The experiment found that the training time of the BPNN method designed in this study was short, 11.36 s, and the average recognition accuracy rate reached 92.2%. In the real-time control experiment of mechanical simulated arm, the recognition accuracy of different actions reached more than 90%, and the running time was short. The experimental results veri ﬁ es the effectiveness of the proposed method and make some contributions to the ef ﬁ cient control of the mechanical simulation arm.


Introduction
The recognition of human bioelectrical signals and the transmission of the recognition results to objects such as robots can enable robots to act according to their motional intentions. It is a new human-computer interaction mode. At present, the electroencephalogram (EEG) [1], electrocardiogram (ECG) [2] and electromyography (EMG) signals have attracted extensive attention of researchers. EMG signals not only are convenient to collect and mature in technology, but also can fully reflect the relevant information of muscle activity [3], which has a very wide range of applications in rehabilitation medicine, simulation robots, etc. [4]. Raj et al. [5] collected the surface electromyography (sEMG) signals of biceps brachii and triceps brachii muscles of subjects, realized the control of prosthetic limb by PID (Proportion, Integration and Differentiation) controller, and verified the effectiveness of the method experiments. Khan et al. [6] studied the application of EMG signals in speech recognition. They collected EMG signals from the neck of 10 healthy subjects and classified them by HMM classifier. The results showed that the average classification accuracy of Ka and Ga was 85% and 78%, respectively, and the accuracy of Kha and Gha was between 88% and 90%. Jung et al. [7] assessed muscle fatigue by EMG signals. They collected EMG signals of femur and gastrocnemius and found that realtime EMG signals of femur and gastrocnemius could be used to estimate muscle fatigue level. Yu et al. [8] studied the application of EMG signals in the diagnosis of cerebral palsy and found through spectral correlation density analysis that normal subjects and cerebral palsy subjects could be distinguished according to the difference of EMG signals. In this study, sEMG signals were collected from three actions using sensors, and the features values were extracted from the signals. Then different actions were recognized using Back-Propagation neural network (BPNN). The reliability of the method was verified through experiments. The present work makes some contributions to the improvement of the action recognition and control effect of mechanical simulated arm, which is conducive to the further development of the field of mechanical simulated arm.

EMG signal detection 2.1 EMG signal
Human action is realized by both skeletal and skeletal muscles. Skeletal muscles are controlled by neurons. When neurons excite, muscle cells depolarize and form action potential of muscle fibers. At that time, EMG signals can be obtained by attaching sensors to the corresponding parts and guiding and amplifying signals. There are two ways to collect EMG signals. One is to collect the surface EMG signals by sticking electrodes on the skin surface. The other is to collect the needle electromyography (nEMG) signals by penetrating the electrodes into the human motor muscle fibers. The interference on nEMG is small, the location is accurate, and it is easier to identify. However, it is seldom used in practice as it may cause injury to users. Therefore, sEMG signals were collected in this study.
sEMG signal is a kind of unstable weak signal which may differ because of collection sites and objects, and it has some characteristics: (1) weak: sEMG is very weak, with an amplitude generally between 0 and 5 mV; the sEMG of ordinary people is 60-300 uV during muscle contraction and 20-30 uV during relaxation, and the sEMG of limb disabled people will be weaker; (2) low frequency: the frequency of sEMG is low, basically below 1000 Hz; (3) alteration: the larger the muscle contraction force, the larger the amplitude of sEMG; (4) random [9]: when the surrounding environment changes, there will be some differences in sEMG.

Detection method of EMG signals
An EMG acquisition system (Delsys, USA) ( Fig. 1) was used to detect sEMG signals. The system has 16 wireless sensors connected to PC via USB. It is easy to operate and reliable, with a sampling frequency of 2000 Hz.
In order to collect sEMG signals, the site which would be pasted with electrode was wiped with alcohol before the experiment, and then the electrode was pasted. The experimental actions included action 1: middle finger and index finger open and the other three fingers grip; action 2: clench fist; action 3: five fingers open, as shown in Figure 2.
According to the relationship between hand movement and muscle, flexor carpi radialis, flexor carpi ulnaris, flexor digitorum superficialis and flexor digitorum profundus were selected as the bonding positions of sensors, as shown in Figure 3.

Feature extraction of EMG signals
In order to obtain good action recognition results and improve the control effect of mechanical simulated arm, it is necessary to extract some feature values with original sEMG signal information. Suppose the length of time window as N t , the length of increment window as N c , and the i-th sEMG signal as x i . The features extracted in this study included: ÀAverage absolute value ðMAVÞ: MAV ¼ ÀWavelength ðWLÞ: ÀZero crossing point ðZCÞ:

Action recognition based on back-propagation neural network
BPNN has been widely used in pattern recognition, image processing, etc. [10]. In this study, actions were recognized using BPNN. The structure of a BPNN is shown in Figure 4. BPNN learning mainly includes two stages, as follows: -Forward propagation It is assumed that the input layer has n nodes, X=(x 1 , x 2 , ..., x n ), i =1, 2, ..., n, j = 1, 2, ..., d, k = 1, 2, ..., m, the weight and threshold are w ij and b j , respectively, the hidden layer has d nodes, H = (h 1 , h 2 , ..., h d ), the weight and threshold between the input layers are w jk and b k , respectively, and the output vector is Y = (y 1 , y 2 , ..., y m ), then: ð5Þ -Reverse propagation The error between the output layer and hidden layer can be expressed as: The modified formula of weight and threshold can be expressed as follows: where n 0 stands for times of iterations, p stands for number of samples, and h stands for learning coefficient.
The error between the input layer and hidden layer can be expressed as: The modified formula of weight and threshold can be expressed as: When BPNN was used for action recognition, the number of nodes in the input layer was 4, including RMS, MAV, WL and ZC, and the output layer nodes was 3, including Action 1, Action 2 and Action 3. In order to determine the number of hidden layer nodes, hidden layers with 10, 15, 20 and 25 nodes were established. Through training, it was found that the error was the smallest when the number of nodes was 15. Therefore, the structure of BPNN used for action recognition was 4-15-3.

Action recognition and control based on sEMG signals 4.1 Action recognition results
Ten healthy college students were taken as the subjects for EMG signal detection. After pasting the sensor, the actions in Figure 2 were repeated 100 times, and each action was kept for 5 s. The collected sEMG signals were stored in the computer. The EMG signals collected by the sensor had some differences in waveform and amplitude, but it was difficult to describe these differences directly and specifically if only relying on pictures. Therefore, for the collected sEMG, feature values needed to be extracted according to the above method, including RMS, MAV, WL, and ZC. The feature values of different actions of a subject are shown in Table 1.
Seventy groups were randomly selected from each action as training samples, and the remaining 30 groups as testing samples. Support vector machine (SVM) [11], naive Bayesian (NB) [12] and BPNN model were trained using   Figure 5. It was found from Figure 5 that the training time of SVM was the longest, 28.13 s, while that of BPNN is the shortest, 11.36 s. It demonstrates that action recognition based on EMG signals had shorter training time, higher training speed and stronger applicability.
The action recognition results of different methods are shown in Table 2.
It was found from Table 2 that the recognition accuracy of SVM and NB was low, and the average accuracy was 71.1% and 73.3%, respectively. It showed that there were many recognition errors and the recognition effect was poor under these two methods. The accuracy of BPNN was 93.2%, 94.1% and 89.4%, respectively, in recognizing Action 1, 2 and 3, and the average accuracy was 92.2%, indicating that it had high reliability.

Control of mechanical simulated arm
Under the help of MATLAB and C# software, the sEMG training platform was established. The stable training accuracy (above 90%) was taken as the training completion condition. The trained model was sent to STM32 processor after the training completed. The processor output control commands and the mechanical simulated arm was controlled in real time. The real-time control flow of the mechanical simulated arm is shown in Figure 6.
The BPNN model trained in Section 4.1 was imported into STM32 processor for real-time control experiment. sEMG signals of three actions were collected from the 10 subjects in real time. Each action repeated 20 times, and there were totally 60 data. The real-time control results of the mechanical simulated arm were recorded, as shown in Table 3.
It was found from Table 2 that the recognition accuracy in the real-time control of mechanical simulated arm was very high, all above 90%, and the average accuracy rate reached 93.8%. In addition, experiments show that the average running time of the program was about 290 ms, the response was rapid, and the real-time performance was strong, indicating that the control method was reliable.

Discussion
Hand is one of the most flexible organs of the human body, and its damage and loss will seriously affect the normal life of the human body. With the progress of technology, the related research of mechanical simulated arm has made great progress. The mechanical simulated arm, combined with technologies such as information processing, biomedicine and computer control, can control the mechanical arm through the human body's own intention, which is a great help to improve the living quality of patients with hand loss. Human hand can complete various complex and flexible actions. In order to make the mechanical simulated   arm complete these actions according to its own intention, it is necessary to choose the appropriate control method. At present, the control methods of electroencephalogram, tendon, sEMG and other signals have been studied, but the acquisition of electroencephalogram and tendon signals is difficult, and their features are not as obvious as sEMG.
The mechanical simulated arm control based on sEMG can realize the exchange of different gestures. With the functions close to the actual hand, it can meet the needs of patients better. Therefore, further improving the accuracy of motion recognition based on sEMG and improving the control effect of the mechanical simulated arm are of great values to fields such as industry, rehabilitation medicine and sports medicine [13,14]. Firstly, the detection process of sEMG is introduced, and then four feature values were determined as input of action recognition model. BPNN, which is widely used in neural network, is used as the action recognition model in this study. Action recognition and real-time control experiments were carried out. The experiment demonstrated that the training time of BPNN was significantly lower than that of SVM and NB, indicating that BPNN model had high training speed and stronger applicability in real-time control of mechanical simulated arm. The comparison of recognition accuracy suggested that BPNN had higher recognition accuracy for different actions than SVM and NB, with an average accuracy rate of 92.2%, while the average accuracy of SVM and NB was only 71.1% and 73.3%. Finally, the experimental results of real-time control showed that the BPNN control method based on sEMG had a recognition accuracy higher than 90% and an average accuracy of 93.8%, short running time and fast response. Therefore it can fully meet the needs of mechanical simulated arm control and can be further applied and promoted in practice.
In this study, the motion recognition of mechanical simulated arm based on sEMG was studied, but there are still many problems that need to be solved: -The actions to be identified were relatively simple, and more complex actions can be selected for experiment; -More complex feature values need to be extracted to improve the recognition accuracy.

Conclusion
Based on the sEMG detection, the action recognition and control of the mechanical simulated arm was studied, the action recognition model was designed by combining with BPNN, and the action recognition and control experiment of the mechanical simulated arm was carried out. The results showed that: -The training time of BPNN was only 11.36 s, shorter than that of SVM and NB; -The average recognition accuracy of BPNN was 92.2%, which was higher than that of SVM and NB; -The average accuracy of BPNN in real-time control was 93.8%, and the average running time was about 290 ms.
The experimental results suggested that the action recognition and control method of the mechanical simulated arm designed in this study had high accuracy and fast response, which makes some contributions to the further improvement of the control effect of the mechanical simulated arm.