In music, ornaments are frills used to give expression to the piece of music being played. Now, technology developed by Sergio Giraldo and Rafael Ramírez, researchers with the Music Technology Group (MTG) of the Department of Information and Communication Technologies (DTIC) at UPF, allows computers to learn to play jazz music in all its expression through artificial intelligence techniques. 

To make this possible, the researchers studied the expressive resources used by jazz guitarists to convey emotions when performing. Then, this information was used to train computational models with the help of recordings made by a professional guitarist, so that the model was able to predict how to ornament a melody, i.e., with all its expressiveness. The result is a program that automatically generates jazz renditions with the expressiveness of an expert, and is available online.

Sergio Giraldo and Rafael Ramírez, members of the MTG’s Music and Machine Learning Lab present the details of the learning system to automatically generate expressive jazz performances in the Journal of Mathematics and Music and provide some examples in Machine Learning and Jazz, published by SoundCloud, a social sound platform where sounds can be created and shared. On this platform, the authors display the song “Yesterday” by Jerome Kern which was generated using the methodology that these researchers have been fine-tuning to perform expressive jazz guitar.

Computational modelling for jazz music using the guitar as a case study was the subject of the doctoral thesis read by Sergio Giraldo on 16 September 2016 at UPF, under the supervision of Rafael Ramírez. For his doctoral research, Giraldo obtained the audio recordings from the high level characteristics that he extracted from the musical scores and the corresponding expressive transformations (time, energy, ornamentation). Once each note had been characterized according to its description in the musical context and for its expressive deviations, Giraldo explored several machine learning techniques to induce predictive models.

Emotions in human-machine interaction are important in order to meet the different expectations of the users. The Music and Machine Learning Lab, directed by professor Rafael Ramírez-Meléndez, is working on the use of emotions in brain-computer interfaces in the field of music. They apply these technologies to investigate the benefits of combining music and brain-computer interfaces to improve users’ health and quality of life.

Reference works:

Sergio Giraldo, Rafael Ramírez (2016), “A Machine learning approach to ornamentation modelling and synthesis in jazz guitar”, Journal of Mathematics and Music, Vol. 10, No. 2, 107– 126, http://dx.doi.org/10.1080/17459737.2016.1207814.

Sergio Giraldo (2016), Computational modelling expressive music performance in jazz guitar: a machine learning approach, doctoral thesis read on 16 September at Pompeu Fabra University, under the supervision of Rafael Ramirez-Meléndez.