Performances as Highly Enriched aNd Interactive Concert eXperiences
PHENICX: Performances as Highly Enriched aNd Interactive Concert eXperiences
PHENICX was a STREP project coordinated by the MTG in collaboration with TU-Delft and funded by the European Commision (ICT-2011.8.2 ICT for access to cultural resources Grant agreement nr 601166).
In this project, academic researchers, music institutions, musicians and up-and-coming technology companies joined forces. The project mission was to make use of all the richness around classical music to build a whole new classical concert experience, supported by smart use of technology.
PHENICX lasted 36 months starting the first of February 2013 and the partner institutions involved were:
- Technische Universiteit Delft (TUD)
- Universitäet Linz (JKU)
- Stichting Koninklijk Concertgebouworkest (RCO)
- VideoDock BV (VD)
- Oesterreichische Studiengesellschaft Fuer Kybernetik (OFAI)
- Fundació Privada per a l'Escola Superior de Música de Catalunya (ESMUC)
- PHENICX prototype website (including access to phenicx integrated prototype): http://phenicx.com
- Project website with publications and documents: http://phenicx.upf.edu
The use of technology is not naturally common in classical concert contexts. Within PHENICX, this was acknowledged, and extra efforts were invested in developing solutions in such a way that they would be accepted, even amidst more traditional audiences. The new technology-enriched concert experiences are explicitly not meant to defy or fully replace traditional concert experiences. Instead, they aim to offer new engaging experiences on top of the traditional experience to those audiences who feel in need of this.
Next to this, by developing a technologically enriched concert experience, the project aimed to find ways to stretch the experience and engagement time span with classical concerts beyond the live experience in the hall. Enrichment also includes digital support for the preparation before the concert, as well as for further revisiting and re-experiencing after the concert. This way, audiences who attended a live concert will have the opportunity to ultimately develop deeper and longer engagement with the concert.
MTG's role and contributions:
The MTG was the main Project Coordinator of PHENICX project (P.I. Emilia Gómez). In terms of S&T contributions, our main contributions were related to:
- Music processing algorithms focusing on symphonic music material. Research outcomes: melody extraction, audio score alignment and informed source separation, audio+symbolic music processing, and computational modelling of conductor gestures.
- Multimodal datasets within the Repovizz framework, including multitrack audio recordings, video recordings, score information, automatically extracted descriptors and gesture information.
- Specific datasets for our target research tasks: Orchset, for melody extraction.
- Web prototypes: music visualization web prototypes accessible at http://repovizz.upf.edu/phenicx/
- MTG team (alph. order): Jordi Bonada, Juan J. Bosch, Julio J. Carabias-Orti, Carles Fernández, Emilia Gómez, Enric Guaus, Jordi Janer, Agustín Martorell, Óscar Mayor, Marius Miron, Alba Rosado, Álvaro Sarasúa.
- Videos: some videos illustrate our work in PHENICX.
Music visualizations during a concert:
Repovizz framework and PHENICX contributions to it:
Audio source separation demo:
Expressiveness in conducting gestures: