Gesture Based Android interface for Deaf persons

Abstract

Mobile communications today can protect and extend the personal freedom of people with disabilities as well as give them more confidence. They can use texting and the video communication available on mobile phones. The deaf prefer using sign language to communicate with each other rather than text. There are problems with the video quality when using real-time video communication available on mobile phones. The mobile video chat is only conversations between deaf callers, not for those between deaf and hearing callers. This dissertation looks at implementing a gesture-based interface for deaf persons which helps the deaf person to communicate with the rest of the world using sign language. However, vision-based hand tracking gesture recognition is an extremely challenging problem due to complexity of hand gestures, which are rich in diversities. This thesis proposes a new method to solve the problem of real-time vision-based hand tracking and gesture recognition. The processing steps include: gesture extraction, gesture matching and conversion to text. In the gesture recognition process, there are three contributions. The first is the new image representation called Integral Image which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a cascade. This structure achieves real-time performance and high classification accuracy. The system yields face detection performance comparable to the best previous systems, face detection proceeds at 15 frames per second.