Multimodality is an interdisciplinary approach, derived from socio-semiotics and aimed at analyzing communication and situated interaction from a perspective that encompasses the different resources that people use to construct meaning.

Description:

Multimodality is an interdisciplinary approach, derived from socio-semiotics and aimed at analyzing communication and situated interaction from a perspective that encompasses the different resources that people use to construct meaning. At a methodological level, multimodal analysis provides “concepts, methods and a framework for the collection and analysis of visual, aural, embodied and spatial aspects of interaction and environments” (Jewitt, 2013). For understanding embodied meaning-making in Full-Body Interactions Learning Environments, we employed this methodological approach to analyze users’ in situ interaction with the system. 

Benefits:

  • Analyse both the affordances offered by the system (e.g. its physical configuration, the available physical/digital objects, etc.) as well as the embodied forms of interacting with it (e.g. the paths that users follow to explore the environment, the variations and repetitions in their sensorimotor enactments, their usage of the available physical/digital elements, their focus of attention and their reciprocal proxemics and social relations).
  • By fully acknowledging and taking into account embodiment, constituted an appropriate and consistent research method to better understand meaning construction and learning in this kind of environments
  • Provides relevant insights to guide Design-Based Research processes and defining design improvements

In the pictures, we show two examples of different techniques for the graphical transcriptions for Multimodal Analysis.

Related Publications:

Laura Malinverni, Marie-Monique Schaper, and Narcis Pares (2018). Multimodal methodological approach for participatory design of Full-Body Interaction Learning Environments. Qualitative Research. DOI

Marie-Monique Schaper, Maria Santos, Laura Malinverni, Juan Zerbini Berro and Narcis Pares. (2018). Learning about the past through situatedness, embodied exploration and digital augmentation of cultural heritage sites. International Journal of Human-Computer Studies, 114: 36-50. DOI

Laura Malinverni, Joan Mora-Guiard, and Narcis Pares. (2016). Towards methods for evaluating and communicating participatory design. Int. J. Hum.-Comput. Stud. 94), 53-63. DOI