Virtual “Valipilla” - Air Gesture Based Tool for Practicing the English Alphabet Writing

Abstract

The recent technological advances in vision based sensors are competent enough to capture the finger positions and movements. It allows computerized systems to capture air writing. This is a popular research area which combines both the fields of human computer interaction and machine learning. This research work proposes a vision based air gesture writing tool which guide primary level children to improve writing skills, specifically focusing on English Alphabet. Proposed methodology for the tool, included implementation of a virtual mouse, 2D virtual writing interface, commanding with static hand gestures and furthermore gesture written character recognition with a new type of user input called point cloud. ‘$P point cloud recognition framework’ is a template matcher which finds the point per point correspondence and uses simple geometric principles for matching. It is invariant to scale, number of strokes, stroke direction, stroke order and stroke type; which makes an ideal solution as a realtime classifier to recognize one character at a time. With the support of Leap Motion controller which is a low cost consumer vision sensor device for hand and finger motion gestures, a prototype was developed and evaluated as a concept of proof. Implemented prototype received an acceptable feedback from the user audience. The conducted evaluation clearly depicted that, by using a state of the art technology with the children in an attractive way for an educational activity can gain more engagement. By considering the final outcome on recognition speed, recognition accuracy, physical strain and other subjective measurements, it discovered that the selected methodology was a suitable approach for actual production of the tool. Finally it is believed that the work carried upon through this research work has facilitated for the betterment of the learning community as well as the human computer interaction and machine learning research community.