What is the project about?

The most widespread technological configurations for Augmented/Mixed Reality systems are based on smartphones and tablet devices and on the interaction paradigm known as Window-on-the-World (WoW). This paradigm provides the user with augmented information on the surrounding physical space by using the device’s screen to merge digital contents with the user’s view of the physical surroundings.

In contrast to this approach, we identify a novel and emerging conceptual paradigm to design ARMR applications for mobile technologies: the World-as-Support (WaS) interaction paradigm. This paradigm is based on projective augmented reality configurations which allow the users to augment the physical surroundings by intelligently projecting the digital information on the physical world. The World-as-Support (WaS) paradigm combines projective augmented reality (PAR) configurations with the possibilities of embodied interaction. Users can use portable devices to augment their physical surroundings by projecting the digital information on the physical world and use the physical world to interact with digital technologies. 

Which goals does the project have within this research?

To understand the potential of the Window-on-the-World (WoW) and World-as-Support (WaS) paradigms we compared their usage in an ARMR application aimed at supporting the collaborative improvisation of site-specific narratives by children.

For the study we employed the “EspaiStory” application and tested under two conditions:

1. Window-on-the-World (WoW): Based on an Android tablet, the system provided a view of the physical environment through the tablet screen as a live feed from the camera of the tablet. The contents created by the children were displayed on the tablet screen as an additional layer
2. World-as-Support (WaS): Based on the combination of an Android tablet and a Philips PicoPix PPX3414 pico-projector bundled in a box and case. The system allowed children to carry around the device and project their own contents directly on the physical environment.

Participants: 35 children (8-9 years old)

Procedure: Children created their own character and subsequently the explored the school telling the story of the character

Data analysis:  To analyze children interaction with the device we employed a multimodal analytical approach focusing on the following resources paths of exploration of the space, interaction with the device, gaze directionality, gestures, group disposition, verbal utterances. From these data we modeled different perfiles related to the ways of using the space and the ways of interacting with others

Results:

Image 1: Modeling the ways of using the space

Image 2: Using the space: Comparing the two conditions

2) Interacting with others

Image 3: Modeling ways of interacting with others

Image 4: Interacting with others: Comparing the two conditions

Lessons learned:

• The two interfaces strongly shaped social relationship
• The two interfaces promoted different ways of inhabiting and making sense of the physical/digital environment

Related Publications:

Laura Malinverni, Cristina Valero, Marie-Monique Schaper, and Narcis Pares. (2018). A conceptual framework to compare two paradigms of augmented and mixed reality experiences. Proceedings of the 17th ACM Conference on Interaction Design and Children. (IDC ’18). ACM, New York, NY, USA, 7-18. DOI= http://doi.org/10.1145/3202185.3202750

Laura Malinverni, Julian Maya, Marie-Monique Schaper and Narcis Pares. (2017). The World-as-Support: Embodied Exploration, Understanding and Meaning-Making of the Augmented World. In Proceedings of the 35th CHI Conference on Human Factors in Computing Systems (CHI ’17). DOI=http://dx.doi.org/10.1145/3025453.3025955

Marie-Monique Schaper, Maria Santos, Laura Malinverni, Juan Zerbini Berro and Narcis Pares. (2018). Learning about the past through situatedness, embodied exploration and digital augmentation of cultural heritage sites. International Journal of Human-Computer Studies. DOI=http://dx.doi.org/10.1016/j.ijhcs.2018.01.003

Marie-Monique Schaper, Maria Santos, Laura Malinverni and Narcis Pares. (2017). Co-Designing Virtual Heritage Experiences for Archaeological Sites based on the novel AR Paradigm World-as-Support ACM Celebration of Women in Computing, WomENCourage Conference 2017. See conference abstract 

Maria Santos, Marie-Monique Schaper, and Narcis Pares. 2017. Moving through the past: design and evaluation of a full-body interaction learning environment for a public space. In Proceedings of the XVIII International Conference on Human Computer Interaction (Interacción '17). ACM, New York, NY, USA, Article 54, 3 pages. DOI= https://doi.org/10.1145/3123818.3133833

Marie-Monique Schaper, Maria Santos, Laura Malinverni and Narcis Pares. (2017). Towards the Design of a Virtual Heritage Experience based on the World-as-Support Interaction Paradigm. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). DOI=http://dx.doi.org/10.1145/3027063.3053089