Back VOCALOID: from Hatsune Miku to Hibari Misora

VOCALOID: from Hatsune Miku to Hibari Misora

The singing voice synthesizer VOCALOID is the result of a long time collaboration between the MTG and Yamaha Corporation

27.01.2020

 

In 1997 The MTG and Yamaha Corporation signed a contract to collaborate in the development of different research and development projects. As a result of this successful and long collaboration there is a great number of patents (more than 40) as well as commercial products such as the popular VOCALOID which represents a new music industry paradigm.

VOCALOID is a singing synthesis technology developed and released by Yamaha in 2003 which has since gained wide recognition as a technology that can produce singing using virtual singers for music production.

The VOCALOID project is developed as part of our research in Voice and Audio Processing. Recently the project has been incorporating innovative methods of voice synthesis based on neuronal networks and deep learning. This approach allows to do things such as modeling a singer's timbre and expression from natural songs, or cloning voices from just a few minutes of recordings.

Various voice banks have been released for use with the VOCALOID synthesizer technology. Each is sold as "a singer in a box" designed to act as a replacement for an actual singer. The commercialization of the VOCALOID Spanish voices is done by Voctrolabs, spin-off of the MTG, which has some virtual singers such as Bruno, Clara and Maika.

Hatsune Miku, the most popular VOCALOID singer, performs live in Barcelona

 

In 2007 the Japanese company Crypton Future Media created the virtual singer Hatsune Miku using VOCALOID to generate her voice. The character is a manga style 16 year-old girl, animated by on-stage 3D projections. She became very popular in Japan, when fans started to create songs and videos using VOCALOID and sharing them online using the Japanese platform Nico Nico Douga . The community around Hatsune Miku plays a fundamental role for the success of  this phenomenon. Hatsune Miku performs in her shows songs created by her fans.

Tomorrow, January 28 2020, Hatsune Miku will perform live in Barcelona, at Sant Jordi Club.  In the same city where her voice was born.

NHK Special: Bringing Hibari Misora Back with AI

 

With the incorporation of deep learning methods, the joint research project VOCALOID currently can synthesize singing which includes the unique mannerisms and nuances of a predetermined vocalist in any language, with any melodies and lyrics.

The new VOCALOID, called VOCALOID:AI, has been used to reproduce the singing of Hibari Misora, a legendary Japanese vocalist. In collaboration with NHK (Japan Broadcasting Corporation), Yamaha reproduced Hibari Misora's characteristic singing voice and speech for a live performance of a new song to commemorate the 30th anniversary of her passing. Actual recordings of the artist's songs and speech made while she was still alive were used as machine learning data to reproduce her singing. Using cutting-edge 4K 3D video to reproduce her likeness, Hibari Misora took the stage and dazzled viewers with her rendition of the new song.

 

Media coverage

La Vanguardia (29/1/2020): Delirio virtual en el concierto de Hatsune Miku, un holograma sin vida

El Periodico (29/1/2020): Hatsune Miku, el holograma que se ha convertido en una estrella pop virtual

Cadena Ser (28/1/2020): Hatsune Miku, un holograma que llena estadios

 

 

Multimedia

Categories:

SDG - Sustainable Development Goals:

Els ODS a la UPF

Contact