Sign Language Translation to Sinhala Language

Abstract

Sign language is the basic communication method between deaf people. Several dictionaries of words defined to make this communication possible. However, there is a communication problem between deaf and ordinary persons unless there is a translator available. In this thesis, design and development of a real time sign language translator is presented based on Sri Lankan sign language. So that the user will be able to perform sign based gesture in front the camera and system will display the meaning of the gesture using Sinhalese characters. In addition, this study focuses on the possibility of enabling the distance communication between deaf and ordinary user by integrating chat application with a 3d simulator of the sign. Several attempts were made previously to implement such translators based on models such as Hidden Markov Models or Artificial Neural Networks classifiers. In this thesis, Microsoft Kinect XBOX 360 camera is used as gesture observation device and an improved version of Dynamic Time Warping algorithm has been proposed for gesture matching purpose. A gesture identifier is generated for every frame by extracting skeleton information and DTW based nearest neighbor classifier has been developed for classification purpose. Microsoft Kinect Software development KIT has been used with Microsoft Visual Studio 2010 development environment, in order to implement the system prototype. System provided an overall recognition rate of 94.2% for a dictionary of fifteen signs in Sri Lankan sign language.