Demos
Here you can find examples of some of the most famous multisensory illusions, in videos that have been produced by our laboratory. Specifically, the ventriloquist illusion and the McGurk illusion arise from audio-visual interaction. We also provide detailed instructions to produce The Rubber-hand and the Pinocchio illusions yourself, but no video this time. This is because to experience these illusions you need the touch sense, something we can't do from the videos. Please make sure to follow the instructions, have fun, and like us on youtube!
Ah, and by the way. We have also included a link to a whole library of typical examples of cognitive psychology experiments, here: Psytoolkit
The ventriloquist illusion: Vision attracts the apparent origin of sounds
What is it?
In this demonstration, you will have to pay close attention to whether the sounds come from, the left or the right. The demonstration works best on headphones with the left and right channels in place (usually marked with an L and an R at the headset). You can also use stereo loudspeakers that are at least at a 45 degree angle left and right of your head. If you experience the illusion, you will perceive the sounds as coming from different places depending on whether or not you watch the screen.
Why does it happen?
Our everyday life experience involves seeing and hearing the sounds of objects all the time. Our brains have learned to exploit the correlation between acoustic and visual sources of events (crossing a street involves being attentive to any sign, visual or acoustic, of approaching cars), and tends to attribute a common origin when they are correlated in time. Here, we created this strong temporal correlation to fool the system into attributing common origin, hence ‘re-aligning’ the perceived origin of the sounding ‘pings’ to the seen location of the ball as it hits the virtual paddle.
The trick
To produce this video-clip, we have misaligned the origin of the sound (e.g., left headphone) with the location of the ball as it hits the paddle (e.g., right side of screen). This misalignment is often reconciled by re-aligning the perceived origin of the sound, hence creating the illusion. If you close your eyes, the sounds are heard from their correct place. This ventriloquist effect is more common than you would think, and it helps us keep some coherence between sound and sight.
|
Ventriloquism is more common that one might think, and it can helps us to focus our attention on a common point through the sight and hearing. For this to happen, sounds and images must be correlated. |
The McGurk-MacDonald illusion: When we can hear the movements of the lips
What is it?
In this video-clip, you will experience the McGurk-MacDonald illusion. Basically, you will hear things differently depending on whether or not you see the speaker’s lips.
Why does it happen?
Speech perception is mostly an auditory phenomenon. However, our brain decodes the speech signal in terms of an “articulatory” code, that is, by reconstructing the sequence of articulations the speaker made to pronounce the sounds (tongue, mouth, and other articulatory organs). Some of these articulations in fact involve actions that can be clearly seen if we watch the speaker talking (e.g., closing the lips to pronounce the sound ‘p’, or opening the mouth to pronounce ‘k’ sounds). When there is a mismatch between the heard and the seen articulation (as it is the case in these demonstrations), our brain sometimes arrives at a compromise between the two sources, leading to this interesting illusion.
How we did make the videos?
These videos require dubbing the image of the speaker as she pronounces a sound with posterior articulation (such as ga, ki, or ke) onto sounds with frontal articulation (such as ba, pe or pi). You can take a look at the individual sounds and silent articulations used for these demonstrations, at the end of the video-clip.
Demo videos of the McGurk illusion
|
The McGurk effect requires dubbing the video of a speaker articulating a sound, (e.g., ba) with the sound of another sound (e.g., ga). You can even create this effect live, when two very well coordinated persons. One should repeat "aba aba aba..." aloud while the other one synchronizes her lips articulating "aga..." but in silence. It depends on who you watch you will hear one syllable or another. |
Rubber-hand and the Pinocchio: Altered body representations
What is it?
In the rubber-hand illusion (RHI) we can come to feel as if a fake hand is part of ourbody, and even localize tactile sensations in this hand. In the Pinocchio-nose illusion we have the vivid sensation that our nose is longer than normal.
 |
Why does it happen?
Our body representation, that is, the mental representation of the position and extension of the distinct parts of the body, is formed by the integration of information from several sensorly modalities such as the propioception (muscles position), vision and touch. Normally, these sensory informations are congruent, and our brain tends to combines them in an efficient way. However, this type of "body" illusions capitalize on disparities between vision and propioception/touch information about hand-location (rubber-hand) or between propioception and touch in the case of Pinocchio-nose. This phenomenon proves that our body representation is more plastic than we could think initially. This plasticity is useful to adapt to changes in our body size and relative proportions (e.g., during growth), or to facilitate the use of tools and other external components to the body (e.g., drive a car).
More information
In Spanish
E. Azañón y S. Soto-Faraco. El espacio a través del tacto. Mente y Cerebro, n. 49. Julio/Agosto 2012
Web
In English
- Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press;
- S. Soto-Faraco, C. Spence, D. Lloyd and A. Kingstone, (2004). Moving multisensory research along: Motion perception across sensory modalities. Curr. Dir. Psychol. Sci., 13, 29–32
- McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746-748. ;
- Botvinick, M. & Cohen, J. (1998) Rubber hands “feel” touch that eyes see. Nature 391(6669):756.