TY - JOUR

T1 - Research on Puppet Animation Controlled by Electromyography (EMG) in Virtual Reality Environment

AU - Tan, Yu Tong

AU - Zhou, Xu Feng

AU - Kong, Ling Zhi

AU - Wang, Xing Ce

AU - Wu, Zhong Ke

AU - Shui, Wu Yang

AU - Fu, Yan

AU - Zhou, Ming Quan

AU - Korkhov, Vladimir

AU - Gaspary, Luciano Paschoal

PY - 2019/10/1

Y1 - 2019/10/1

N2 - Quanzhou puppet is one of the intangible cultural heritages of China. It is the physical embodiment of traditional Chinese culture. However, the large size of the puppet and inconvenience to carry and manipulate directly makes it hard to reach a wider audience. In order to realize the effective inheritance and protection of Quanzhou puppet, this study designs a virtual real-line puppet animation scheme based on gesture recognition, builds a prototype system which uses MYO Armband EMG signal to control the generation of animation, and applies it in user experiment to verify the high accuracy and easy manipulation of the algorithm. Firstly, low-pass filtering and smoothing is used to process the original multi-channel EMG data. Secondly, after eight-channel EMG signal time-domain feature and time-frequency-domain feature extraction, the dimension of the feature vector is reduced to six by linear discriminator to eliminate the correlation between features and enhance the robustness of the algorithm. Thirdly, a multi-class support vector machine is constructed which uses feature vector to determine the result of gesture recognition. Experiments show that the average recognition accuracy of offline action is 95.59%, the average recognition accuracy of real-time action is 90.75%, and the gesture recognition is completed within 1.1 s. For the puppet task, two users task is designed: the common users and the expert users. In the common user study, the gestures recognition accuracy is high. In the aspects of user's willingness to use and easiness to learn, the performance of this system is significantly higher than real puppets manipulation. In the expert user study, user's acceptance and usability of the system are also highly evaluated. These two user tasks indicate the system meets the requirements of real-time and accuracy, and has good interactivity and interesting. Relevant research can be widely applied to similar systems, such as computer animation. It has practical significance for experiencing and protecting the puppet.

AB - Quanzhou puppet is one of the intangible cultural heritages of China. It is the physical embodiment of traditional Chinese culture. However, the large size of the puppet and inconvenience to carry and manipulate directly makes it hard to reach a wider audience. In order to realize the effective inheritance and protection of Quanzhou puppet, this study designs a virtual real-line puppet animation scheme based on gesture recognition, builds a prototype system which uses MYO Armband EMG signal to control the generation of animation, and applies it in user experiment to verify the high accuracy and easy manipulation of the algorithm. Firstly, low-pass filtering and smoothing is used to process the original multi-channel EMG data. Secondly, after eight-channel EMG signal time-domain feature and time-frequency-domain feature extraction, the dimension of the feature vector is reduced to six by linear discriminator to eliminate the correlation between features and enhance the robustness of the algorithm. Thirdly, a multi-class support vector machine is constructed which uses feature vector to determine the result of gesture recognition. Experiments show that the average recognition accuracy of offline action is 95.59%, the average recognition accuracy of real-time action is 90.75%, and the gesture recognition is completed within 1.1 s. For the puppet task, two users task is designed: the common users and the expert users. In the common user study, the gestures recognition accuracy is high. In the aspects of user's willingness to use and easiness to learn, the performance of this system is significantly higher than real puppets manipulation. In the expert user study, user's acceptance and usability of the system are also highly evaluated. These two user tasks indicate the system meets the requirements of real-time and accuracy, and has good interactivity and interesting. Relevant research can be widely applied to similar systems, such as computer animation. It has practical significance for experiencing and protecting the puppet.

KW - EMG signal processing

KW - Gesture recognition

KW - LDA (linear discriminant analysis)

KW - MYO armband

KW - SVM (support vector machine)

UR - http://www.scopus.com/inward/record.url?scp=85075570206&partnerID=8YFLogxK

U2 - 10.13328/j.cnki.jos.005786

DO - 10.13328/j.cnki.jos.005786

M3 - Article

AN - SCOPUS:85075570206

VL - 30

SP - 2964

EP - 2985

JO - Ruan Jian Xue Bao/Journal of Software

JF - Ruan Jian Xue Bao/Journal of Software

SN - 1000-9825

IS - 10

ER -