Tamil to Sinhala Augmented Reality Translation

Abstract

The latest generation of mobile phones comes with hardware powerful enough to develop innovative applications that allow users to directly interact with outside world. Augmented reality translation is a novel concept that interprets a source text on a natural scene, in a user's augmented view, re-display the text in a target language. It involves text detection in the natural image, text recognition, machine translation and re-display the translated text in augmented view. This research presents an approach to apply the augmented reality translation to Sinhala and Tamil languages. It investigates usage of language independent text detection algorithm called Stroke Width Transform (SWT) to detect Tamil text in natural images. Implementation of the proposed application has been done using Android technology combined with Tesseract OCR and Google Translate API. The Android application captures the natural image using the mobile camera and process the captured image to detect and recognize the Tamil text. Then the detected Tamil text will be translated to Sinhala text through the language translation service and overlay the Sinhala text on the augmented view of the mobile screen. Translated text will be cached locally and reuse in future to reduce network access. ICDAR 2003 dataset has been used to evaluate general text detection accuracy of the application. Set of collected natural images contains Tamil text used to evaluate accuracy of the Tamil to Sinhala augmented reality translation and analyze the source of failures.