SWARALOKA ADAPTIVE MUSIC SCORE TRAINER FOR VISUALLY IMPAIRED

Abstract

Some inherited barriers which limit the human abilities can be surprisingly won through technology. Swar¯al¯oka focuses on defining a more reliable and a controllable model for visually impaired people to read and train Eastern music notations which are widely available in printed format. Music notation has become a major communication medium among musicians and is compulsory in Sri Lankan local Eastern music syllabus and even in Bhathkande music exams. In current situation, visually impaired people require third party assistance to convert visual music notations to a format readable for them. Hence, Swar¯al¯oka provides a controllable auditory interface (instead music Braille) for them to study and experiment on music independently. The main objective of the research was to identify and propose most readable and convenient format that visually impaired people can use to understand eastern music notations. Main research objective ran through three basic paths which explored most effective methods for recognizing music notations, converting music notations into a readable format (which is more convenient for visually impaired users) and defining qualities to be embedded in the interface. Briefing the whole methodology of Swar¯al¯oka, initially it pre-processes (de-skew and binarize) soft-copy of a printed music notation. Then the tabulated area, zones of notation rows and blobs within notation rows are detected. In order to extract composite notation symbols without losing the information embedded, we used a separate eroded clone of original image to detect boundaries in regions of interest and the original pre-processed image to extract content. A novel OMR method which analyze outer profiles of symbols was used for recognition of music notations. Depending on the OMR output, two intermediary files (one to define the sequence of music notes and the other to define the sequence of thaal symbols) were created and converted to its corresponding audio format. Output consists with synthesized swara singing with background thaal layers created using sound banks with fixed length audio chunks. Evaluation of the research was done with 16 students from Blind School, Ratmalane. Same environmental conditions were preserved throughout evaluation process. The evaluation basically consisted two phases in order to be able to compare between the effectiveness of existing manual method and the model proposed by Swar¯al¯oka. Relevant to the tested scenarios, the average translation time in manual method was 3 min. 7 sec., and the translation time of Swar¯al¯oka was 1 min. 31 sec. which is three times better compared to manual method. Users also could grasp the output format suggested by the research with a greater accuracy. According to the summary of user feedback, 81% of the users have mentioned that Swar¯al¯oka is the best method to use. Research approach proves that profile based recognition can be used effectively when there is limited number of characters to identify. Further it suggests that audio chunks in a sound bank with a fixed duration can be used to synthesize more naturalistic rhythmic singing. Users could effectively grasp information from concurrent audio layers, proving that concurrent auditory signals can be used to convey composite information more effectively compared to spatial tactile signs. The overall evaluation suggests that the most convenient format that visually impaired people can use to understand eastern music notations is the controllable audio format compared to Braille music notations. Post processing of music notation symbol recognition can be further enhanced by using a well tested probability matrix which previews probability of one note following another. More assistance in capturing music notation scripts will help the users to be more independent and convenient. The research has a potential to be extended in a way which assist users with feedback on his/her own singing.