Emotion-based Music Generation using Generative Models

Abstract

Abstract Music, as a universal language, has the power to evoke a wide range of emotions and profoundly influence human experiences. Emotion-based music generation has emerged as a compelling avenue for harnessing the expressive potential of music through computational methods. This study explored the e!ectiveness of di!usion models in generating emotion-based music. In this research, the generation of music utilised the mel spectrogram representation of audio as the input as it resembles how humans perceive music. A latent representation of this spectrogram was used to train the di!usion model for faster training. Concurrently, four emotion classes derived from Russel’s Valence Arousal Model were employed to condition the music generation process. A subjective evaluation, incorporating both model-generated and human-composed songs, was conducted to assess the perception of intended emotions by human evaluators. Among the four emotion types utilized to condition the generation, High Arousal - High Valence, High Arousal - Low Valence, Low Arousal - Low Valence and Low Arousal - High Valence, the highest successful identification by human evaluators was observed with High Arousal - High Valence which was 29%. Further analysis incorporating valence and arousal scores revealed that the model excelled in generating high-arousal music, with 71% of evaluators correctly identifying such music.