Jordi Pons, currently a Research Scientist at Dolby Laboratories, and formerly a doctoral researcher with the Music Technology Group (MTG) at the UPF Department of Information and Communication Technologies (DTIC), receives one of the Premio Jóvenes Informáticos awards at the 20th edition of the Premios de Investigación Sociedad Científica Informática de España (SCIE) – Fundación BBVA 2020, “for his contributions in the field of music information retrieval, with special emphasis on the use of deep learning architectures for tagging sound and music signals, with which he has earned numerous scientific citations”, as expressed by the jury of the Awards, when it met via videoconference on 26 May last.

Auto-tagging of audio and music can increase the possibility of reusing many audio databases that remain virtually untagged

In his thesis defended on 15 November 2019 at UPF, Pons approaches the task of auto-tagging audio and music from the perspective of deep learning.  Auto-tagging of audio and music can increase the possibility of reusing many audio databases that remain virtually untagged.

In this context, in his work Pons addresses the following scientific questions:

• What deep learning architectures are best suited for (musical) audio signals?
• In which scenarios is it feasible for models of deep learning to directly process waveforms?
• How much data are needed to carry out research studies in deep learning?

To answer the first question, Pons proposes using musically motivated convolutional neural networks and evaluates various deep learning architectures for audio at a low computational cost. “Throughout our research, we find that the knowledge we had about music and audio can help us improve the efficiency, interpretability and performance of learning models based on spectrograms”, Pons explains.

Deep learning models that process waveforms (like SampleCNN) can achieve better results than those that process spectrograms

To answer the other questions, Pons continues to explain: “we study how SampleCNN, a deep learning model that processes waveforms, works when we have varying amounts of training data - from 25 thousand up to 1.2 million songs. In this study, we compare SampleCNN with an architecture based on spectrograms that is musically motivated”.

The experimental results obtained indicate that in scenarios where sufficient data are available, deep learning models that process waveforms (like SampleCNN) can achieve better results than those that process spectrograms.

Transfer learning and prototypical networks are useful strategies when training data are not abundant

Finally, in an attempt to find out how much data is needed to conduct research studies in deep learning, they also investigated whether a naive regularization of the solution space, prototypical networks, transfer learning, or combinations thereof, can foster deep learning models to better leverage a small number of training examples.

The results of the experiments indicate that transfer learning and prototypical networks are powerful strategies in such low-data regimes.

The Young Computer Science Researchers category of the Premios de Investigación Sociedad Científica Informática de España (SCIE) – Fundación BBVA, aim to distinguish relevant, innovative doctoral research projects. It seeks to recognize researchers who contribute to the generational replacement in this scientific and technical discipline and its study, strengthening and dissemination. It recognizes the creativity, originality and excellence of these researchers in their early professional years and aims to encourage pursuing their research.