Sign Language Recognition for Sentence Level Continuous Signings

Abstract

It is no doubt that communication plays a vital role in human life. Most people consider communication as important as breathing for humans. However, there is a significant communication gap between hearing impaired people and others, because they use different techniques for their communication purposes which others cannot understand. These techniques are based on sign language, the main communication protocol among hearing impaired people. In this research, we propose a method to bridge the communication gap between hearing impaired people and others which translates sign language sentences into text. Most of the existing solutions, based on technologies such as Kinect, Leap Motion, Computer vision, EMG and IMU try to recognize and translate individual signs of hearing impaired people. The few approaches to sentence-level sign language recognition suffer from not being user-friendly or even practical owing to the devices they use. The proposed system was therefore designed to provide full freedom to the user. For this purpose, we employ two Myo armbands for gesture-capturing. Using signal processing and supervised learning based on a vocabulary of 49 words and 346 sentences for training with a single signer, we were able to achieve 75-80% word-level accuracy and 45-50% sentence level accuracy using gestural (EMG) and spatial (IMU) features for our signer-dependent experiment.