Since December 2016, it has been possible to put a face and a voice to KRISTINA (answers and replies in Spanish, German and Polish), after presenting the first prototype of the human interface. KRISTINA is a research project funded by the European Union, within the Horizon 2020 Programme, whose goal has been to develop a technological solution to help overcome migrants’ language and cultural barriers in healthcare and geriatric services in the host countries.

Coordinated by  Leo Wanner, ICREA research professor of the Department of Information and Communication  Technologies (DTIC), participating among the nine project partners is the research group on Natural Language Processing (TALN), the Interactive Technologies research group (GTI), and the Cognitive Media Technologies group (CMTech), all from the DTIC, as well as the Spanish Society of Family and Community Medicine (semFYC), as well as companies, institutions, universities and scientific societies from five European countries (Germany, Spain, France, Greece and the Netherlands).

A virtual assistant the first prototype of which has been successfully tested

Project KRISTINA has been developing a virtual assistant in the field of healthcare that will enable getting over language barriers when dealing with users and, therefore, its priority application will be to attend to migrants who are struggling to overcome language barriers.

The first prototype was presented in November 2016 in Tübingen (Germany) and will be used for testing in as real situations as possible the prototype’s capacity for natural conversation. The intention is that in the future the assistant will be capable of holding a conversation by providing responses at colloquial speed, recognizing emotions in the voice, the face and body posture, as well as the cultural conditions of the person with whom it is talking and hold conversations in German, Spanish, Polish, Turkish and Arabic.

However, the prototype presented now only can understand and speak only German, Spanish and Polish and is able to establish a suitable response (among which are, for example, responding to specific questions about medical issues, advising on different health matters or reading an article). One of the aspects on which most work has been done on this prototype is that the assistant will generate a natural language output voice for each response.

A prototype that has already talked to potential users

One of the strengths of project KRISTINA is the coordination of interdisciplinary teams for its development. For this reason, within the European KRISTINA project, the technical partners are constantly working in different research areas to make the conversation more natural, improving the expressions and the ability to recognize and interpret human emotions and feelings. Thanks to the integration of the teams, project KRISTINA has already tested scientific developments on real citizens who are likely to receive its assistance. The Spanish Society of Family and Community Medicine (semFYC), the German Red Cross and EKUT (Eberhard Karls Universität in Tübingen) have been testing the first prototype with groups of end users (among whom are some elderly and caregivers with a migratory background).

Specifically, EKUT has undertaken to perform the analyses among an elderly population of Turkish origin in a context of geriatrics and dependency; the German Red Cross is evaluating the interaction between KRISTINA and a group of caregivers of Polish origin who work with people with dementia in Germany; and the SemFYC is responsible for the second pilot case, evaluating the information of immigrants from North Africa.

After the presentation of the first prototype, which is able to speak and respond in three languages, the researchers and developers are now focusing on the systems needed for KRISTINA to be able to interpret and execute human gestures and emotional interactions in a natural way, as well as a reaction to any input (verbal or gestural) expressed by the human interlocutor.

Overall viability of the project

Project KRISTINA has the support of the European Union, which is funding it entirely. This line of work emerged from the European Commission’s Action Plan on health online 2012-2020, which provides a road map to empower users, but also to link devices and technologies and invest in research into the personalized health and geriatric care of the future.

According to the e-health action plan 2012-2020 on health innovation for the 21^st century, presented by the European Commission, the potential of the e-health market is strong, despite the economic crisis that has slowed down a great deal of R&D&I. Specifically in the field of e-Health, it is estimated that the global telemedicine market has grown from $9.8 billion, in 2010, to $11.6 billion, in 2011, and it is expected to continue expanding until reaching 27,300 million dollars in 2016.