List of results published directly linked with the projects co-funded by the Spanish Ministry of Economy and Competitiveness under the María de Maeztu Units of Excellence Program (MDM-2015-0502).

List of publications acknowledging the funding in Scopus.

The record for each publication will include access to postprints (following the Open Access policy of the program), as well as datasets and software used. Ongoing work with UPF Library and Informatics will improve the interface and automation of the retrieval of this information soon.

The MdM Strategic Research Program has its own community in Zenodo for material available in this repository   as well as at the UPF e-repository   

 

 

Back Slizovskaia O, Gomez E, Haro G. Correspondence between audio and visual deep models for musical instrument detection in video recordings. 18th International Society for Music Information Retrieval Conference (ISMIR17)

Slizovskaia O, Gomez E, Haro G. Correspondence between audio and visual deep models for musical instrument detection in video recordings. 18th International Society for Music Information Retrieval Conference (ISMIR17)

This work aims at investigating cross-modal connections between audio and video sources in the task of musical instrument recognition. We also address in this work the understanding of the representations learned by convolutional neural networks (CNNs) and we study feature correspondence between audio and visual components of a multimodal CNN architecture. For each instrument category, we select the most activated neurons and investigate existing cross-correlations between neurons from the audio and video CNN which activate the same instrument category. We analyse two training schemes for multimodal applications and perform a comparative analysis and visualisation of model predictions.

Additional material:

Link: https://ismir2017.smcnus.org/lbds/Slizovskaia2017.pdf

Related Assets