[September 2016-Ongoing]

Background

Autism Spectrum Condition (or Disorder) is still very unknown. As its name depicts it covers a very broad range of conditions that makes it difficult to define and deal with. Important research is being undertaken in genetics to try to understand its origin, cause and typification but no major advances have been made yet.

In the meantime, ASC people need pragmatic solutions to help them in a number of aspects of their daily lives​, such as social interaction. ICT has become an important source of intervention and therapeutic tools in the last 10 years. There is a complete lack of sharing of data from trials of ICT tools for ASC​. This data could be useful to many researchers to compare results and to build research in different directions from that same data​. Within the available ICT tools Embodied Interaction is increasingly showing its potential in ASC. Data from these tools is multimodal in nature and is hence complex to store and analyze.​

Project Objectives

This project will create a large­scale publicly available reference database of multimodal data from sessions of ASC children and youngsters using ICT therapy and intervention tools. ASC­ME­OR will be based on REPOVIZZ, a multimodal online database and visualization tool developed by MTG. This proposal will foster the widespread use of the public database and associated analytics tools​. ASC­ME­OR will already start housing data from the embodied ICT experience designed and developed by CMTech in a previous project funded by RecerCaixa (IN­AUTIS­TIC). New interaction sessions will be undertaken and stored during the 2 years of this proposal.

This project will also create a set of data analytics tools for analyzing the public database for extracting powerful information from users’ interactions. This information will also include biometric feedback to assess users’ internal states which could help explain changes in the anxiety, as well as other parameters, of children with ASD when socializing with a partner. This is especially useful because children with autism are sometimes not able to express their feelings.

Proposed Methodology

We will propose an approach to model social engagement and interaction initiation using machine-learning techniques in Lands of Fog (second version). We plan to obtain data from 3 sources: psycho-physiological measurements, behavior data from video coding & questionnaires, logs from system management events. We will obtain a model of the relations between all these parameters. We will create models for different levels of social interaction dimensions. We will evaluate the accuracy of different models by performing feature selection and exploring different algorithms. Finally, we will test our models on unseen data and obtain a quantitative evaluation, as well as user qualitative evaluations. We will propose several applications that include a wearable to collect reliable physiological data, a system for detection of anxiety levels in full body interaction environments, an online database tool for collaborative sharing of multimodal data.

Related Publications

RepoVizz - Data repository and visualization tool for structured storage and user-friendly browsing of multi-modal recordings