Narcís Parés, a member of the Cognitive Media Technologies Research Group, which is part of UPF's Department of Information and Communication Technologies, is working on a line of research called "full-body interaction". In his laboratory, he designs different applications based on such interaction with a view to studying the mediation of experience. He often uses artificial reality (nowadays a branch of virtual reality, despite having been a precursor thereof), as defined by Myron Krueger in 1973, since when it has been popularized and become hugely widespread, mainly in video games but also in other areas, such as applications for psychomotor rehabilitation.

As Parés explains, "the first artificial reality systems developed and marketed used the third-person interaction paradigm", i.e. a configuration in which the user sees a representation of their body interacting with virtual objects in a virtual  environment. "It's like a digital mirror", he continues. "The user interacts with objects indirectly through an added element, a silhouette, hence the term third-person interaction."

Another paradigm used in fullbody interaction is first-person interaction, which combines the physical and virtual environments. Parés's laboratory has employed both the paradigms in question to carry out projects and develop applications chiefly aimed at children, with the goal, in every case, of using artificial reality to improve people's quality of life.

Environments, music and activities for special needs

It was Parés himself who originally defined first-person interaction. He did so around 12 years ago, in the context of the MEDIATE research project, which was funded by the European Union and involved the development of an interactive environment for autistic children.

"We piloted the MEDIATE environment with 90 low-functioning autistic children between the ages of 7 and 12 from 4 European cities, Barcelona, London, Portsmouth and Hilversum in the Netherlands", states Parés. "The aim of the study was to give them a sense of control and agency in the MEDIATE environment, an interactive, multisensory space expressly designed as a play area, in which they played independently, safely and calmly."

The project's researchers found that more than 80% of the participants achieved a sense of control over the environment and that many of them also achieved a sense of agency (i.e. a feeling of connection between them and their surroundings), which are very important accomplishments where autistic children are concerned. The researchers concluded that their choice of interaction design, paradigm and technological configuration had proven to be sound, justifying and opening the door to further studies based on the technology.

Parés and his colleagues went on to design the SIIMTA project, a first-personartificial reality experience that enhanced the results of music therapy  sessions for children with autism.

The next step in applying the technology with first-person interaction consisted of creating an interactive slide incorporating artificial reality, with the aim of compensating for sedentary lifestyles among children. "The project's main objective was to prevent health problems, such as those stemming from obesity, during children's development, through play with special interactive media", says Parés. "The research is at the experimental stage at the moment", he adds. "Depending on its results, we hope to be able to incorporate it into physical play activities on a regular basis, with a view to improving children's quality of life."

Pico's Adventure: a video game that encourages social interaction

One of the greatest difficulties autistic children face is communicating with other people. Asking for help, initiating social interaction and expressing emotions are abilities that they have to acquire through exercises and therapy.

The Cognitive Media Technologies Research Group has worked with the Specialized Developmental Disorder Unit at Sant Joan de Déu Hospital in Barcelona to create Pico's Adventure , a full-body interaction video game featuring third-person interaction.

Pico's Adventure is intended to nurture social interaction in autistic children through a series of collaborative play activities. "Our first experimental studies show that it's effective as a complement to conventional therapies", Parés reports. "The game's technology is based on the use of the Kinect device, which is available for Windows platforms", he remarks.

A helping hand from an alien

Pico's Adventure comprises four stages of play which promote social interaction in autistic children. The environment in which the game takes place  includes a friendly alien who has landed on Earth due to a problem with his spaceship. This character accompanies children in a series of adventures designed to foster specific forms of behaviour, such as asking for help, taking turns and giving other children instructions.

Initial experimental studies conducted with children diagnosed with autism have demonstrated the video game's effectiveness in terms of promoting social interaction among them, which suggests that it could become a significant complement to conventional types of therapy for the disorder. It has been developed as part of a European research project entitled "Motion-based adaptable playful learning experiences for children with motor and intellectual disabilities (M4All)", which, in turn, is part of the European Commission's Lifelong Learning Programme.

More information:
- Pico's Adventure is freely available from http://lab4.ccp.upf.edu/?page_id=72
- The Cognitive Media Technologies Research Group's website: http://cmtech.upf.edu