Mobile Phone based Awareness of Surroundings for the Visually Impaired

Abstract

Assistive technology helps improve the day to day activities for people with disabilities. One of the methods utilized by assistive technologists employs the use of mobiles. Mobile computing has evolved from being used as communication devices to multi-purpose mobile computing devices. Modern day smart phones are equipped with a set of assistive features that can be used by the handicapped for exploring the applications. This thesis explores the idea of a mobile application for the visually impaired to improve the surrounding awareness. It is inspired by the existing assistive technology such as scene describing mobile applications and mobile technology such as mobile sensors. One method explored in this thesis is to facilitate sensors to monitor the direction the visually impaired person is facing and automate the process of screen capture. Additional method is to provide manual screen capture by introducing hard tap on the „Device‟ as an alternative to tap on „Buttons‟ in the touch screen. This thesis utilizes a machine learning model that is a trained model for scene describing [1]. A proof of concept which runs on an IPhone containing MobileNet as the trained model is presented. The evaluation on the success of improving the awareness of a visually impaired person is carried out with a group of visually impaired undergraduate students