MusAV: A dataset of relative arousal-valence annotations for validation of audio models

Authors

Bogdanov D, Lizarraga Seijas X, Alonso-Jiménez P, Serra X

UPF authors

BOGDANOV, DMITRY; LIZARRAGA SEIJAS, XAVIER EDUARDO; ALONSO JIMENEZ, PABLO; SERRA, XAVIER

Type

Conference proceedings

Book Authors

AA. VVV.

Book title

Proceedings 23rd International Society for Music Information Retrieval Conference (ISMIR 2022)

Publisher

ISMIR

Publication year

2022

Pages

650-658

ISBN

978-1-7327299-2-6

Abstract

We present MusAV, a new public benchmark dataset for comparative validation of arousal and valence (AV) regression models for audio-based music emotion recognition. To gather the ground truth, we rely on relative judgments instead of absolute values to simplify the manual annotation process and improve its consistency. We build MusAV by gathering comparative annotations of arousal and valence on pairs of tracks, using track audio previews and metadata from the Spotify API. The resulting dataset contains 2,092 track previews covering 1,404 genres, with pairwise relative AV judgments by 20 annotators and various subsets of the ground truth based on different levels of annotation agreement. We demonstrate the use of the dataset in an example study evaluating nine models for AV regression that we train based on state-of-the-art audio embeddings and three existing datasets of absolute AV annotations. The results on MusAV offer a view of the performance of the models complementary to the metrics obtained during training and provide insights into the impact of the considered datasets and embeddings on the generalization abilities of the models.

Complete citation

Bogdanov D, Lizarraga Seijas X, Alonso-Jiménez P, Serra X. MusAV: A dataset of relative arousal-valence annotations for validation of audio models. In: AA. VVV.. Proceedings 23rd International Society for Music Information Retrieval Conference (ISMIR 2022). 1 ed. Bengaluru: ISMIR; 2022. p. 650-658.