Sign Language Recognition in Low-Resourced Languages

Abstract

1 Abstract Sign Language Recognition (SLR) systems are vital for bridging communication gaps between deaf and hearing communities, yet low-resourced languages like Sinhala Sign Language (SSL) face challenges due to limited training data. This thesis investigates the efficacy of cross-lingual transfer learning to enhance SLR accuracy for SSL, a language with scarce datasets. By pre-training Transformerbased models on a large Indian Sign Language (ISL) dataset and fine-tuning them on a 64-word SSL dataset, the study evaluates performance improvements in lowresource scenarios, simulating data scarcity with 2 to 6 instances per class. Results demonstrate that transfer learning significantly boosts accuracy with 2 or 3 instances per class, achieving up to an 8% improvement over models trained directly on SSL, though benefits diminish with 4 or 6 instances per class. The study also explores the impact of overlapping semantic and movement patterns between ISL and SSL, finding no conclusive advantage. Additionally, varying base model sizes (80 to 240 classes) showed no consistent effect on fine-tuning performance, suggesting further research is needed. This work contributes to the field by providing insights into transfer learning strategies for low-resourced SLR, offering methodologies applicable to other under-resourced sign languages, and highlighting the potential for developing accessible communication systems with minimal data. In conclusion, this study confirms that pre-training on high-resource sign languages like ISL can lead to meaningful improvements in recognizing signs from SSL, particularly in extreme low-resource conditions where only 2–3 video instances per sign are available. While the benefits of transfer learning diminish as more training data becomes available, this approach offers a promising pathway for developing effective SLR systems for underrepresented languages with limited datasets.