Enhancing the Communication Experience for the Deaf and Hard-of-Hearing using AI Language Models

Abstract

Abstract Effective communication in healthcare is critical for accurate diagnosis and treatment, but deaf and hard-of-hearing (DHH) people experience substantial challenges in medical consultations. This problem grows more significantly in Sri Lanka, since there are few assistive tools accessible for Sinhala-speaking DHH patients. The identified research gap is the absence of accessible, real-time communication tools that enable seamless interaction between healthcare professionals and DHH patients in Sinhala. This research bridges the gap by developing a mobile application that enhances bidirectional communication in medical settings, addressing the user group’s preference for mobile solutions and their need for support in healthcare contexts. The study focuses on creating and evaluating a smartphone application that allows healthcare professionals to speak in Sinhala, which is then transcribed into text. The app provides three contextually relevant responses for the DHH patient, who can choose, modify, and confirm one, which is then turned back into voice for the doctor. Furthermore, the system accepts text-based input from the patient, allowing doctors to answer verbally. The evaluation uses a mixed-method approach, integrating quantitative indicators like mobile application transcription accuracy and answer generating speed with qualitative feedback from interviews with DHH individuals. Usability testing was conducted with Deaf/Hard-of-Hearing (DHH) people and communication facilitators to evaluate accessibility, efficiency, and user satisfaction. The findings show that the application enhances communication clarity and lowers misconceptions during medical consultations. DHH users provided positive feedback, emphasizing the application’s real-time response creation and ease of use. This research contributes to assistive technology for the DHH community in Sri Lanka by providing a realistic answer to a crucial healthcare accessibility issue.