Starting date: April 2022

Project duration: 4 years

Project partners: SONY R&D and MTG-Universitat Pompeu Fabra, with the collaboration of HUMAINT project, Joint Research Centre, European Commission.

Team: PIs - Yuki Mitsufuji (SONY), Weihsiang Liao (SONY), Emilia Gómez (JRC and MTG) and Xavier Serra (MTG). PhD Student - Roser Batlle Roca

Abstract

Current advancements on the naturalness and expressiveness of machine learning models for music generation have raised questions about the legal, cultural and social impact that those systems may have, including the works of musicians, producers and the listening experiences. This research deals with the concept of trustworthiness in AI-powered music generation systems. The goal is to investigate how explainability and transparency can be used to inform listeners and other stakeholders about the degree to which AI technologies are involved in the music they listen to, and to relate generated music with the different components of the generation model, i.e. the dataset, algorithm (e.g. neural network architecture) and the different humans involved in the process (e.g. developer, dataset curator, composer of pieces used for training). The project will also investigate how we can characterize the originality of machine vs human-generated music by exploiting current knowledge on music similarity to assess the generated outcome with respect to the datasets used for training in terms of different facets such as melody, timbre, rhythm and musical  structure. Finally, the project will integrate knowledge and methods from machine learning explainability, music information retrieval, musicology and intellectual property.