Essentia - progress

With the Essentia API project, our goal is to develop a novel web API service for music audio analysis based on the outcomes of R&D in Music AI at the Music Technology Group. The funding from the María de Maeztu Innovation program is oriented to the following objectives, with the goal of advancing from TRL 4 to TRL 8:

• Develop our core technology: ML models and inference backend, scalability tests, and integration into cloud computing platforms.

• Create a user platform with a dashboard for the API usage and analysis results and product websites.

• Develop product positioning and sustainability plans.

• Conduct legal consultancy regarding our business operations, API Terms of Use, data management policy, and compliance with EU AI and software regulations.

From the start of the project and until May 2025, we made significant progress in the development of our core technology and user platform. In the area of core technology, we created new ML models tailored for auto-tagging tasks in industrially-relevant use cases, including a new music genre recognition model (500+ music genres, ready-to-use and validated by an industry partner) and mood/context recognition (200+ tags, currently work in process). We improved our backend ML inference server performance, scalability, and reliability in various aspects, addressing the queue task manager and support for longer audio files. On the user platform side, we created a new platform website with MVP functionalities for authentication and profile management, and an overview of analysis requests. In addition, we worked on design and implementation for the product website featuring online demos and a new visual identity (implementation currently in progress).

Specific activities:

• Improve the efficiency of the ML inference backend pipeline.

• Create new ML analysis models for music genre and mood, and context.

• Develop an MVP for the user platform.

• Perform initial market study and develop database of potential clients (with the support of mentors from the programs De la Ciència al Mercat 2023 and IdC Llavor 2024). 

• Design a product website.

• Dissemination at Sónar 2024 Project Area (presentation booth) and Barcelona Mobile Congress 2024 (networking).

• Pilot case study for the available ML analysis models with industry.

Future plans

• Gathering feedback from developer advocates (already contacted the first beta-testers)

• Migrating the MVP to an industry-scale cloud computing platform and performing scalability tests.

• Development of the payment system.

• Creation of new AI models for auto-tagging with industrially-relevant use-cases ( instrumentation, etc.) based on the input from market study and consultancy.