A Language-Independent Method for Lyrics-Based Cover Song Identification Using Phoneme Transcriptions

Abstract

Abstract Research in music information retrieval has traditionally focused on audio-based methods for cover song identification, analyzing elements like chords, melody, and harmony. However, these methods face scalability issues and struggle with covers that significantly alter the original music. Recent studies have shifted towards text-based approaches, using metadata and lyrics, since lyrics often remain consistent across different versions of a song, making these systems more robust. Nonetheless, these approaches are typically limited by language dependency in Singing Voice Recognition (SVR). This thesis introduces a novel method for cover song identification that utilizes the phonetic transcriptions of lyrics. The approach is based on the premise that any spoken language can be transcribed into the phonetic transcriptions of International Phonetic Alphabet (IPA).We fine-tuned the XLS-R wav2vec 2.0 model using Connectionist Temporal Classification (CTC) to transcribe singing into IPA phonetic representations. Songs are then analyzed for similarity using the Levenshtein distance to identify cover versions. The study achieved a 40.41% improvement in multilingual phoneme recognition in singing voices compared to the baseline. However, the results for English cover song identification were below those of the state-of-the-art lyrics-based cover song identification methods. Nonetheless, our proposed system achieved a Mean Average Precision (MAP) of 0.513 for identifying cover songs in Sinhala, a language not previously seen by the model during training or fine-tuning. This demonstrates the potential of using phonetic transcriptions for language-independent, lyrics-based cover song identification.