 Research
 Open Access
 Published:
Gesture recognition method based on a singlechannel sEMG envelope signal
EURASIP Journal on Wireless Communications and Networking volume 2018, Article number: 35 (2018)
Abstract
In the past, investigators tend to use multichannel surface electromyography (sEMG) signal acquisition devices to improve the recognition accuracy for the study of gesture recognition systems based on sEMG. The disadvantages of the method are the increased complexity and the problems such as signal crosstalk. This paper explores a gesture recognition method based on a singlechannel sEMG envelope signal feature in the time domain. First, we get the sEMG envelope signal by using a preprocessing circuit. Then, we use the improved method of valid activity segment extraction to find every valid activity segment and extract 15 features from every valid activity segment. Next, we calculate the absolute value of the correlation coefficient between each of the features and target values. After removing the feature with the smaller correlation coefficient, we reserve the 14 features. By the PCA dimensionality reduction algorithm, we transform the 14dimensional feature into 2dimensional feature space. Finally, we use the improved KNN algorithm and the soft margin SVM algorithm to complete the classification of five types of gestures. We obtain the gesture recognition rates of 75.8 and 79.4% by using the improved KNN algorithm and the soft margin SVM algorithm.
Introduction
Surface electromyography (sEMG) is the temporal and spatial superposition of faint bioelectrical signals generated by the muscle nerve cells during muscle contraction [1]. It is collected and recorded through the skin surface electrodes. Compared to conventional EMG signal acquisition that requires inserting a needle electrode into muscle tissue, the sEMG signal has the advantages of being noninvasive and providing the convenience of collection. The sEMG signal from the arm has been used in humancomputer interaction, rehabilitation training, and artificial prosthesis [2,3,4,5]. Because it can indirectly reflect the gestures of the person, it has extensive application and research value [6].
A Canadian company (Thalmic Labs) develops the world’s first Myo armband based on sEMG signal in 2013, which is used in the contactless humancomputer interactive device [7,8,9,10]. In 2015, Harbin Institute of Technology in China develops an imitation humanoid “dexterous hand” for the national lunar exploration project. One of the key technologies is gesture recognition based on sEMG [11, 12]. Previously, in the study of gesture recognition based on sEMG signal, investigators tend to increase the number of acquisition channels to improve the recognition rate [2]. Not only does this increase costs, but it also increases the complexity of the acquisition process and the number of data processing. Especially in application for commerce, multichannel has a high requirement for hardware. So it leads to a high cost and is hard to popularize. Besides, multichannel is easy to cause signal crosstalk problems [13, 14]. Moreover, previous investigators just study the raw sEMG property. Motivated by the aforementioned discussions, we investigate a new method.
In this paper, we develop and design the sEMG acquisition system and study the envelope property of the signals obtained. We put forward an improved method for extracting valid activity segment, and the method can be used to find with precision the starting point and ending point for each of the valid activity segments, which greatly reduce the noise interference and increase the precision of extraction. We extract 15 common features which include all sEMG envelop properties and analyze the correlation between every feature and five types of gestures. We find that the slope sign change (SSC) is unsuitable for the studying of sEMG envelope signal. However, according to previous research, SSC is an important feature to study of the raw sEMG signal. We also find that sEMG envelop signal can greatly reflect sEMG amplitude property but ignore its frequency property. These are the major differences between raw sEMG signal and sEMG envelop signal. Therefore, we only keep the other 14 features. However, there are still too many data for processing, which increase the burden of classifier. Furthermore, we use the PCA linear dimension reduction algorithm to transform 14dimensional features into 2dimensional. Finally, we apply three pattern recognition algorithms to classify the five types of gestures and find that the soft margin SVM is the best classifier with a recognition rate of 79.4%, while the improved KNN algorithm gives a gesture recognition rate of 75.8%. It turns out sEMG envelope property is an important way for the study of sEMG.
sEMG signal acquisition
Introduction of process of this experiment
We design the acquisition system to get the sEMG envelope signal. The sEMG envelope signal is the DC signal after the process of amplification, rectification, integration, and amplification again for the raw sEMG signal by the preprocessing circuit. Then, we use an improved method for extracting valid activity segments. After that, we extract 15 features and analyze the correlation. Next, we reduce the feature dimension by PCA dimension reduction. Finally, we compare the three pattern recognition algorithms. The whole process of the experiment is shown in Fig. 1.
Experimental methods
In this paper, five kinds of gestures are studied: bend the wrist up while flat, bend the wrist down while flat, bend the wrist up while in shake hand position, bend the wrist down while in shake hand position, and make a fist (as shown in Fig. 2). AgCl electrodes are placed in the position of the ulnar flexor carpi ulnaris of the right forearm (as shown in Fig. 3). The subjects are four college students, aged 23–25 years, two men and two women, all righthanded. Before the experiment, the skin of each subject’s right forearm is cleaned with soap [15]. The sampling frequency is set at 2000 Hz. Every gesture is repeated 60 times, and 300 samples are collected from each subject (as shown in Table 1). The typical waveform which corresponded with each of the five types of gestures is shown in Fig. 4.
The experimental algorithm
An improved method for extracting valid activity segment based on the moving square method
The purpose of valid activity segment extraction is to efficiently identify every valid activity segment to reduce the amount of processing data, to determine the starting and ending sampling points of every activity waveform, to distinguish from noise, and to lower the effect from noise interference. In this paper, an improved method is proposed based on the moving square method, as stated below:
① Assuming x_{ ki } represents the ith sampling point in kth activity segment, and within 32 ms after x_{ ki } there are 64 sampling points at the sampling frequency of f = 2000 Hz, then the average short time energy within 32 ms (window width, w = 64) is:
② In accord with our experience, we selected the appropriate thresholds as TH_{1} and TH_{2}. TH_{1} is the starting point threshold, and TH_{2} is the ending point threshold. When satisfying E_{ k } > TH_{1}, the first sampling point is recorded x_{ ks }. After x_{ ks } is recorded, the first sampling point is recorded x_{ ke } (e > s) when satisfying E_{ k } < TH_{2}.
③ Activity duration is recorded T, then:
④ From sampling point x_{ ks } to x_{ ke }, the average energy is recorded \( {\overline{E}}_k \), then
⑤ In accord with our experience, choosing TH_{3}, TH_{4} (TH_{3} < TH_{4}), and TH_{5} as the third, fourth, and fifth thresholds, respectively, when satisfying \( {\mathrm{TH}}_4>\mathrm{T}>{\mathrm{TH}}_3\kern0.5em \mathrm{and}\kern0.5em {\overline{\mathrm{E}}}_{\mathrm{k}}>{\mathrm{TH}}_5 \), the kth activity segment is valid. x_{ ks } is the real starting sampling point, and x_{ ke } is the real ending sampling point during the valid activity segment. Otherwise, the activity segment is regarded as noise and considered invalid.
Figure 5 shows the process of extraction of valid activity segment about the sEMG envelope signal corresponding to the five types of gestures. Figure 5a shows the short time energy (E_{ k }) distribution of the sEMG envelope signal when the window width w = 64. In order to improve the performance of antiinterference, TH_{1} is slightly higher than TH_{2}. Figure 5b shows the duration (T) of valid activity segment. According to the experiment results, we set TH_{3} = 500 ms and TH_{4} = 1800 ms. Figure 5c shows the average energy \( {\overline{E}}_k \) for five types of gestures. In Fig. 5c, we chose the appropriate threshold as TH_{5}. Figure 5d shows the extraction of valid activity segment for the sEMG envelope signal.
Feature extraction
The sEMG signal is a nonstationary random signal, but it can be regarded as a stationary random signal in short periods of time [2]. At present, the common feature extraction methods applied to raw sEMG signal are feature extraction methods based on timedomain statistical features, frequencydomain statistical features, timefrequency domain statistical features, and parametric model [11]. Timedomain features extract time structures in the sEMG signal [16,17,18,19]. Timedomain statistical features can intuitively reflect the amplitude characteristics of the sEMG signal, and its algorithm is easy to implement. It has the advantage of realtime and speed. In frequencydomain statistical features, investigators first need to take the Fourier transform of the timedomain sEMG signal and then extract power spectrum or spectral characteristics of the sEMG signal. But the method creates difficultly and complexity. In this experiment, because the sEMG envelope signal loses its frequency property, we only extract the features from timedomain and parametric model. In the experiment, we find that the trend lines of the envelopes corresponding to the five gestures are obviously different. Therefore, we extract 15 features of the sEMG envelope signal in the timedomain and the parametric model.
Feature extraction based on timedomain statistical features
Timedomain statistical characteristics mainly extracted the sEMG signal’s duration T, integral IEMG, mean absolute value (MAV), variance (VAR), standard deviation (s), average energy (E), maximum (MAX), slope sign change (SSC), skewness (Sk), and kurtosis (Ku). Assuming that {x_{ ki } i = 1, 2……n} represent all the sampling points in the kth valid activity segment, where n represents the total number of sampling points in the valid activity segment, x_{k1} represents the starting sampling point of this valid activity segment, and x_{ kn } represents the ending sampling point of this valid activity segment. In timedomain statistical features, the formulas are as follows:
① Time of duration for each valid activity segment T:
② Integral, IEMG:
③ Mean absolute value, MAV:
④ Variance, VAR:
⑤ Standard deviation, s:
⑥ Average energy, E:
⑦ Maximum, MAX:
⑧ Slope sign change, SSC:
⑨ Skewness as the index of data symmetry, Sk:
⑩ Kurtosis (Ku) describes the steep degree of data state, the formula is as follows:
where f is the frequency of sampling, s is the standard deviation of this valid activity segment, and μ is the mean.
Feature extraction based on parameter model
Although the sEMG signal is a nonstationary random signal, it has good stability in short time intervals and can be regarded as a piecewise stationary quasi stationary signal [11, 12]. Therefore, a mathematical model can be established for this property. According to the AR model, the jth sampling point in the kth valid activity segment is
where α_{ i } is the model coefficient, β(j) is the white noise, p is the order of the model, and p = 4 is appropriate according to the experimental results.
In this experiment, the AR model intercept (IT) and the fourthorder (p = 4) model coefficients α_{1}、α_{2}, α_{3}, and α_{4} of the AR model are chosen as the features. Finally, we extract 15 features in total.
The correlation analysis between features and five types of gestures
We calculate absolute value of the correlation coefficient between each of the 15 features and five types of gestures, respectively [16]. Calculating it as follows:
where n is the number of all samples, y_{ ij } represents the jth (1 ≤ j ≤ 15) feature of the ith sample, \( {\overline{y}}_j \) represents the mean of the jth feature in all samples, t represents the target value, and \( \overline{t} \) is the mean of the target value. Define the target value of the first gesture is 1, the target value of the second gesture is 2, the target value of the third gesture is 3, the target value of the fourth gesture is 4, and the target value of the fifth gesture is 5. Each final correlation coefficient is the mean absolute value of all experimenters’ correlation coefficient. Then, we illustrate with a bar graph (as shown in Fig. 6). In this paper, we obtain the sEMG envelope signal, and this is the DC signal after the raw sEMG signal is amplified, rectified, integrated, and amplified again. As a result, the sEMG envelope signal is different from the raw sEMG signal. So, we make the correlation analysis about the suitability of the classification for the five types of gestures. It can be seen from Fig. 6 that the absolute value of the correlation coefficient between SSC and the target value is the smallest and is approximately equal to zero. It is irrelevant. So SSC almost has no effect for classification of five types of gestures. We remove the SSC from all features. Lastly, we get the 14 features.
PCA linear dimensionality reduction algorithm
Highdimensional spatial data cannot be visually expressed, so we cannot choose the appropriate classifier subjectively. In addition, the higher the dimension means the greater the amount of data, the more complex the data for handling, and the higher the hardware requirements. In order to observe distribution of the sample data and simplify the task of classifier, we use the dimensionality reduction algorithm. PCA (principal component analysis) is the most commonly used linear dimensionality reduction algorithm. Through a kind of linear projection, this algorithm can make the data be mapped to lowdimensional space from highdimensional space. We assume that n dimensional vector w is the mapping vector, then make the mapping data variance maximum:
where m is the number of all samples, X_{ i } is the eigenvector which consists of all the features of every sample, and \( \overline{\boldsymbol{X}} \) is the average vector of all the features of all samples. W is a matrix containing all the mapping vectors as column vectors. After transformation, the following optimal projection matrix can be obtained:
where tr is the trace of the matrix, I is the unit matrix, and A is the covariance matrix of all the sample data. The optimal projection vector W^{′} is composed of eigenvectors that are column vectors which are determined by the front k largest eigenvalues of the sample data covariance matrix A, so that the transformed dimensionality reduction matrix can be obtained as follows:
where n (n > k) is the number of all original features. Finally, we get the kdimensional features. Experiments show that when feature matrix is reduced to 2dimensional (k = 2), the proximate recognition effect as the original data can be achieved with the least amount of data. After the 14dimensional feature matrix is reduced to 2dimensional feature matrix, the data distribution of five types of gestures is shown in Fig. 7. In Fig. 7, the horizontal axis x represents the first dimensional value and the vertical axis y represents the second dimensional value in the 2dimensional feature matrix. The legends first, second, third, fourth, and fifth represent the five kinds of gestures. It can be seen from Fig. 7 that every type within the five types of gestures is grouped together. According to the figure, we can select an optimal pattern recognition algorithm to achieve the optimal classification effect.
Pattern recognition algorithm
KNN algorithm based on weight optimization
In the traditional knearest neighbor (KNN) algorithm, in order to judge the category of unknown instances, all known instances are taken as references. The distance between each unknown instance and all known instances is calculated, and the knearest neighbor in known instance sets are selected. We obey the rules that the minority obeys the majority. And lastly, we make the unknown instances be classified into the category of the largest number of knearest neighbor samples. This traditional algorithm has obvious shortcomings. When the sample distribution is unbalanced, such as one of the samples is too large or dominant, the new unknown sample can easily be classified as the dominant or scattered category. In view of such a problem, this paper proposes an improved KNN algorithm, taking the reciprocal of distance as the weight. Assuming that the known set of instances of known categories is Ω = {X_{1}, X_{2}, X_{3}⋯⋯X_{ n }} and the set of instances of unknown categories to be classified is Ψ = {Y_{1}, Y_{2}, Y_{3}⋯⋯Y_{ m }}. The process is as follows:
① We calculate all distance d_{ ij } between each unknown instance Y_{ j } to be classified and all X_{ i } in Ω, respectively.
② In descending order of d_{ ij }, we select knearest neighbor known instances from Ω.
③ We calculate the k reciprocal of distances d_{ ij }, respectively, and make the reciprocal \( \frac{1}{d_{ij}} \) as the weight.
④ In these k known instances, we calculate the number of instances of each category as α and sum of the weights of each category as \( \beta =\sum \limits_{i=1}^{\alpha}\frac{1}{d_{ij}} \).
⑤ Next, we calculate the value of α × β as the weight of this category and name the value of α × β as θ.
⑥ Lastly, we classify the unknown instance into the category that has the highest weight θ.
Soft margin support vector machine algorithm
Support vector machine (SVM) is a supervised algorithm [20]. Its principle is to maximize the margin between a data set and the hyper plane separating two data sets. The soft margin SVM algorithm is to solve the linear nonseparable problem. Its idea is to find an n − 1 dimensional hyperplane in ndimensional space as a decision function and divide the data into two parts, each of which belongs to a category. Its model is as follows:
where w is the vector representing adaptive model parameters, C is the weight of the outlier, that is, the penalty factor. The larger the C means the greater the impact of the outlier for the target. ξ_{ i } is a slack variable, and m is the number of sample data points. y_{ i } is the label associated with a data point x_{ i }, and y_{ i } is − 1 or 1. x_{ i } is the vector representing a data point. b is the intercept of the hyper plane. After computing for formula (21), we get the hyper plane of classification for the two categories as follows:
In addition, it has been successfully applied to classification for multiclass.
Comparison of recognition results
The experiment finds that waveform features from the same gesture differ from one person to another [21], so we only need to train the sample data collected from different people in order to recognize the gestures from different people. Table 2 and Fig. 8 show the average recognition accuracy of the four subjects by three recognition algorithms. In order to verify the effect of PCA algorithm, we also get the recognition accuracy in the 14dimensional feature space. In Fig. 8, the legends first, second, third, fourth, fifth, and mean, respectively, represent the first gesture, the second gesture, the third gesture, the fourth gesture, the fifth gesture, and mean of five types of gestures. TKNN represents the traditional KNN algorithm, and IKNN represents the improved KNN algorithm. SSVM represents the soft margin SVM algorithm. Figure 8a shows the accuracy in 14dimensional feature space. Figure 8b shows the accuracy in 2dimensional feature space. It can be seen that the overall recognition accuracy in 14dimensional feature space and in 2dimensional feature space belong to the same order of magnitude, which illustrates the reliability of the PCA dimensionality reduction algorithm in this experiment. It can also be seen from Fig. 8 that the KNN algorithm based on weight optimization is better than the traditional KNN algorithm and that the soft margin algorithm is the best in this experiment.
Conclusions
This paper explores a gesture recognition method based on a singlechannel sEMG envelope signal. In the experiment, after the raw sEMG signal passed the preprocessing circuit, we obtain its envelope signal. We just use one channel to extract features from the sEMG envelope signal and get the about 80% average accuracy. It shows that the study of gesture recognition based on the sEMG signal can also be researched from the sEMG envelope signal. The property of sEMG envelope signal contains great information of gesture. If we ignore the problems caused by multichannel, we will get the higher recognition accuracy by using multichannel acquisition. So how to deal with the contradiction between these problems caused by multichannel acquisition and highaccuracy will be our next task in the future. This paper provides a reference value for future research, especially applications for prosthesis and biomimetic robot.
Abbreviations
 E :

Average energy
 IEMG:

sEMG signal integral
 IKNN:

Improved knearest neighbor
 IT:

Intercept
 KNN:

knearest neighbor
 Ku :

Kurtosis
 MAV:

Mean absolute value
 MAX:

Maximum
 PCA:

Principal component analysis
 s :

Standard deviation
 sEMG:

Surface electromyography
 Sk:

Skewness
 SSC:

Slope sign change
 SSVM:

Soft margin support vector machine
 SVM:

Support vector machine
 T :

Time of duration for each valid activity segment
 TH:

Threshold
 TKNN:

Traditional KNearest Neighbor
 VAR:

Variance
References
 1.
G Staude, W Wolf, Objective motor response onset detection in surface myoelectric signals. Med. Eng. Phys. 21, 449–467 (1999)
 2.
M RojasMartinez, MA Mananas, JF Alonso, Highdensity surface EMG maps from upperarm and forearm muscles. J. Neuroeng. Rehabil. 9(1), 1–17 (2012)
 3.
F Zhao, L Wei, H Chen, Optimal time allocation for wireless information and power transfer in wireless powered communication systems. IEEE Trans. Veh. Technol. 65(3), 1830–1835 (2016)
 4.
F Zhao, H Nie, H Chen, Group buying spectrum auction algorithm for fractional frequency reuses cognitive cellular systems. Ad Hoc Netw. 58, 239–246 (2017)
 5.
K Agamemnon, K Iris, MS Erden, et al., Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. J. Neuroeng. Rehabil. 14(1), 71 (2017)
 6.
K Lienhard, A Cabasson, O Meste, et al., Comparison of sEMG processing methods during wholebody vibration exercise. J. Electromyogr. Kinesiol. 25, 833–840 (2015)
 7.
Y Du, W Jin, W Wei, et al., Surface EMGbased intersession gesture recognition enhanced by deep domain adaptation. Sensors 2, 458–480 (2017)
 8.
F Zhao, B Li, H Chen, X Lv, Joint beamforming and power allocation for cognitive MIMO systems under imperfect CSI based on game theory. Wirel. Pers. Commun. 73(3), 679–694 (Nov 2013)
 9.
F Zhao, X Sun, H Chen, R Bie, Outage performance of relayassisted primary and secondary transmissions in cognitive relay networks. EURASIP J. Wirel. Commun. Netw. 2014(1), 60 (2014)
 10.
F Zhao, W Wang, H Chen, Q Zhang, Interference alignment and gametheoretic power allocation in MIMO heterogeneous sensor networks communications. Signal Process. 126, 173–179 (2016)
 11.
D Graupe, WK Cline, Functional separation of EMG signal via AMAR identification methods for prosthesis control purposes. IEEE Trans. Syst. Man Cyber 15, 252–260 (1975)
 12.
S Micera, AM Sabatini, P Dario, On automatic identification of upperlimb movements using smallsized training sets of EMG signals. Med. Eng. Phys. 22(8), 527 (2000)
 13.
L Mesin, S Smith, S Hugo, et al., Effect of spatial filtering on crosstalk reduction in surface EMG recordings. Med. Eng. Phys. 31(3), 374–383 (2009)
 14.
DA Winter, AJ Fuglevand, SE Archer, Crosstalk in surface electromyography: Theoretical and practical estimates. J. Electromyogr. Kinesiol. 4(1), 15–26 (1994)
 15.
L Hargrove, K Englehart, B Hudgins, A training strategy to reduce classification degradation due to electrode displacements in pattern recognition based myoelectric control. Biomed. Signal Process Control 3, 175–180 (2008)
 16.
A Stango, F Negro, D Farina, Spatial correlation of high density EMG signals provides features robust to electrode number and shift in pattern recognition for myocontrol. IEEE Trans. Neural Syst. Rehabil. Eng. 23, 189–198 (2015)
 17.
D Farina, N Jiang, H Rehbaum, et al., The extraction of neural information from the surface EMG for the control of upperlimb prostheses: Emerging avenues and challenges. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 797–809 (2014)
 18.
K Kim, D CHo, KJ Lee, et al., A realtime pinchtozoom motion detection by means of a surface EMGbased humancomputer interface. Sensor 15(1), 394 (2014)
 19.
A Phinyomark, M Phothisonothai, P Phukpattaranont, et al., Critical exponent analysis applied to surface EMG signals for gesture recognition. Metrology Meas. Syst. 18(4), 645–658 (2011)
 20.
MA Oskoei, H Hu, Support vector machinebased classification scheme for myoelectric control applied to upper limb. IEEE Trans. Biomed. Eng. 55, 1956–1965 (2008)
 21.
A Mapelli, BCZ Machado, LD Giglio, et al., Reorganization of muscle activity in patients with chronic temporomandibular disorders. Arch. Oral Biol. 9(72), 164–171 (2016)
Acknowledgements
Northeast Normal University and Changchun University of Science and Technology offer the experiment platform.
Funding
This work is supported by the National Natural Science Foundation of China under Grants 61370228 and 61471111 and Jilin Provincial Development and Reform Commission under Grant 2017c0331.
Author information
Affiliations
Contributions
YSW conceived and designed the experiments and wrote this paper. SLL and LZ helped to perform the analysis with constructive discussions. ZQC, CLC, and SWW contributed to the structuring and reviewing of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Correspondence to Shili Liang.
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 sEMG
 Gesture recognition
 Envelope signal feature
 Improved KNN algorithm
 Soft margin SVM