Digital Pianoforte Tutor: A Computer based evaluator for Piano Playing

Abstract

Evaluating a piano playing has been the domain of music experts most of the time. If one (i.e. pianist or piano player) needs to receive precise evaluation on his/ her piano playing (i.e. piano performance) of a music piece given in sheet music (i.e. manuscript having music notation for the music piece), he/ she will have to meet a music teacher or some music expert who can give some useful feedback, pointing out the pianist of any mistakes of the playing, and areas that can be improved. Even though it is still not very prevalent, a computer aided evaluation on such piano performance could support a pianist to get an evaluation on his/ her playing, without having to visit a music expert each time. Digital Pianoforte Tutor (DPT) will be a computer aided evaluator that compares a recorded piano playing with the sheet music, and provide an evaluation that would be useful for the pianist (e.g. music student) to get an idea on how well he/ she performed, while also identifying his/ her mistakes in the piano playing in terms of note mismatches (i.e. incorrectly played music notes), note insertions (i.e. additionally played music notes) and note deletions (i.e. music notes missed). The sheet music in PDF format and the WAVE recording of the pianist?s piano playing will be the inputs to Digital Pianoforte Tutor (DPT). DPT will consist of the three main modules; Optical Music Recognition (OMR), Feature Extraction (FE) and Performance Evaluation (PE). The underlying research for this artefact, will aim in providing a number of evaluations, and assess the strengths and weaknesses of each of these evaluation methods, in order to identify which of these techniques would provide a useful performance evaluation for the pianist. The OMR module makes use of the Java Audiveris API to generate a MusicXML file from the PDF sheet music. In the FE module, musical feature extraction from the MusicXML file and MIDI or WAVE audio file will take place making use of one of the selected audio FE plugins and modules (e.g. Java jMusic API, MatLab MIRtoolbox). Once sufficient music features have been obtained in the FE module, the PE module will generate useful feedback to the pianist using the selected evaluation techniques (i.e. Note Alignment and Music Visualisation). The PE module was implemented in Java; while Note Alignment made use of Needleman Wunsch (NW) global alignment algorithm to identify playing mistakes, and Music Visualisation made use of piano roll notation and MuseScore digital score notation to give a visual musical representation. Keywords: optical music recognition, music feature extraction, music information retrieval, performance evaluation, music visualisation