Title: Human-centered machine learning for music emotion recognition

Supervisors: Dr. Emilia Gómez Gutiérrez, Dr. Perfecto Herrera Boyer and Dr. Estefanía Cano (AudiosourceRE)

Jury: Dr. Arthur Flexer (Johannes Kepler University), Dr. Isabelle Hupont (European Commission Joint Research Center) and Dr. Blair Kaneshiro (Stanford University)

Abstract:
This doctoral thesis is focused on music in terms of emotion -- such algorithms compose the computational task of music emotion recognition (MER). MER evaluates emotionally relevant features from music, correlates them with certain emotions that could be perceived by or induced to a listener, and finally attempts to predict these emotions. The work attempts to frame the relevance of the MER task into two broad research questions: what for? and for whom?

In general, MER displays a need to incorporate contextual and individual data to eventually be able to effectively model the annotations that it tries to predict. Therefore, the main goal of this dissertation is to improve understanding of systems that place the human at the center of the MER system. The studies and experiments included here cover several subtopics for that goal: discovering interpretable/meaningful features for machine learning models, allowing and encouraging response diversity during dataset creation, selecting relevant music according to the background of the listener, enabling an evaluation feedback between the listeners and the MER model for evaluation and improvement, and guiding potential application scenarios with overall ethical principles.

As a consequence of our findings, we propose methodologies to incorporate a human-centric perspective in several stages of the MER task: acknowledging the complexity of creating a “ground truth” from subjective emotion annotations, incorporating properties and context from diverse listeners by using agreement as input to the algorithms, and evaluating possible risks of adopting a human-centric perspective for personalization purposes.

This thesis defense will take place in hybrid format. To attend online use this link (meeting ID 932 0051 6196). The microphone and camera must be turned off, and the online access will be unavailable after 30 minutes from the start of the defense.