The audio processing software was created by the Music Technology Group (MTG) led by Xavier Serra

ESSENTIA is an audio analysis software library developed by the Music Technology Group (MTG) in the Department of Information and Communication Technologies (DTIC). It is modular, expandable, and easy to incorporate into any computer program. Distributed under an open software licence to promote its use in research, it is also available under proprietary licence for use in commercial applications. 

The library includes an extensive collection of audio-processing algorithms that enable the automatic retrieval of content properties, the statistical characterization of data, and calculations involving large sets of spectral descriptors. ESSENTIA also won the ACM Multimedia 2013 Open Source Software competition.

The MTG, set up more than twenty years ago, is a research group strongly committed to technology transfer that has sought to make open research compatible with having a social and industrial impact. ESSENTIA is a paradigmatic case in that it has generated a wide variety of new applications, both for commercial use and for research.

Freesound Labs

Launched in July 2015, Freesound Labs is a directory of projects, hacks, applications, research, and other initiatives that use content from Freesound, a free, collaborative, online platform that uses ESSENTIA and is based on Creative Commons licences. It is used in many artistic productions, including plays, ringtones, and feature films, as well as by prestigious research institutions worldwide.

Thanks to the Freesound API, computer applications have been developed able to directly access the content. According to Serra, it is ‘a collaborative platform for sharing sounds’.

AcousticBrainz, a high-impact project for the scientific community

The MTG and the MetaBrainz Foundation joined forces to lead AcousticBrainz, an initiative based on the contributions of users who use the ESSENTIA software to analyse music recordings. It is thus another source of the data that this software analyses.

To date, AcousticBrainz has analysed almost three million audio tracks. For each one, the tool generates data on basic characteristics, such as tonality or the number of beats per minute. These data are then used to extract other more important characteristics for the end user, such as the piece’s mood or genre. All of this information can be accessed through the AcousticBrainz website for use in research or computer applications.

‘We hope that AcousticBrainz will have a large impact on the scientific community. The data obtained will make it possible to take on new and interesting challenges that will give rise to innovative technologies’, Serra explained. Users can use the AcousticBrainz website to analyse their own music collections. The contribution of many users is key, as it will be what guarantees the tool’s success.

MORE INFORMATION

— Musical Technology Group (MTG): http://www.mtg.upf.edu/

— ESSENTIA: http://essentia.upf.edu/

— AcousticBrainz: http://acousticbrainz_org/

— MusicBricks: http://mtg.upf.edu/projects/musicbricks

— CompMusic: http://compmusic.upf.edu/

NEW PROJECTS TOGROW OUT OF ESSENTIA

MUSICBRICKS, promoting innovation in musical technology

In early 2015, the MTG launched ‘Musical Building Blocks for Digital Makers and Content Creators’, or MusicBricks, a new collaborative project within the framework of an innovative action financed by the European Commission under the Horizon 2020 programme and coordinated by the French company SIGMA Orionis SA.

The main goal of MusicBricks is to mature innovative technologies obtained through research to make them available and useful to the cultural and creative industries sector in Europe.

CAMUT, automatic analysis of Carnatic and Hindustani music

The group recently received one of the 45 Proof of Concept grants awarded by the European Research Council to researchers who have already been awarded a research grant from the European body. It will use the €150,000 grant to carry out the project ‘Culture Aware Music Technologies’ (CAMUT), which aims to promote the commercialization of some of the technologies developed within the framework of the ‘Computational Models for the Discovery of the World’s Music’ (CompMusic) project and bring technologies for the automatic analysis of Carnatic and Hindustani music to the Indian market.