Education and Brain: Teaching from Neuroscience

Here you can find examples of some of the most famous multisensory illusions, in videos that have been produced by our laboratory. Specifically, the ventriloquist illusion and the McGurk illusion arise from audio-visual interaction. We also provide detailed instructions to produce The Rubber-hand and the Pinocchio illusions yourself, but no video this time. This is because to experience these illusions you need the touch sense, something we can't do from the videos. Please make sure to follow the instructions, have fun, and like us on youtube! 

Ah, and by the way. We have also included a link to a whole library of typical examples of cognitive psychology experiments, here: Psytoolkit

The ventriloquist illusion: Vision attracts the apparent origin of sounds

What is it?

In this demonstration, you will have to pay close attention to whether the sounds come from, the left or the right. The demonstration works best on headphones with the left and right channels in place (usually marked with an L and an R at the headset). You can also use stereo loudspeakers that are at least at a 45 degree angle left and right of your head. If you experience the illusion, you will perceive the sounds as coming from different places depending on whether or not you watch the screen. 

Why does it happen?

Our everyday life experience involves seeing and hearing the sounds of objects all the time. Our brains have learned to exploit the correlation between acoustic and visual sources of events (crossing a street involves being attentive to any sign, visual or acoustic, of approaching cars), and tends to attribute a common origin when they are correlated in time. Here, we created this strong temporal correlation to fool the system into attributing common origin, hence ‘re-aligning’ the perceived origin of the sounding ‘pings’ to the seen location of the ball as it hits the virtual paddle.

The trick

To produce this video-clip, we have misaligned the origin of the sound (e.g., left headphone) with the location of the ball as it hits the paddle (e.g., right side of screen). This misalignment is often reconciled by re-aligning the perceived origin of the sound, hence creating the illusion. If you close your eyes, the sounds are heard from their correct place. This ventriloquist effect is more common than you would think, and it helps us keep some coherence between sound and sight.

Demo video of the ventriloquist illusion: What you have to do?

In this demonstration, you will have to pay close attention to whether the sounds come from, the left or the right. The demonstration works best on headphones with the left and right channels in place (usually marked with an L and an R at the headset). You can also use stereo loudspeakers that are at least at a 45 degree angle left and right of your head. If you experience the illusion, you will perceive the sounds as coming from different places depending on whether or not you watch the screen. 

 

The McGurk-MacDonald illusion: When we can hear the movements of the lips

What is it?

In this video-clip, you will experience the McGurk-MacDonald illusion. Basically, you will hear things differently depending on whether or not you see the speaker’s lips.  

Why does it happen?

Speech perception is mostly an auditory phenomenon. However, our brain decodes the speech signal in terms of an “articulatory” code, that is, by reconstructing the sequence of articulations the speaker made to pronounce the sounds (tongue, mouth, and other articulatory organs). Some of these articulations in fact involve actions that can be clearly seen if we watch the speaker talking (e.g., closing the lips to pronounce the sound ‘p’, or opening the mouth to pronounce ‘k’ sounds).  When there is a mismatch between the heard and the seen articulation (as it is the case in these demonstrations), our brain sometimes arrives at a compromise between the two sources, leading to this interesting illusion. 

How we did make the videos?

These videos require dubbing the image of the speaker as she pronounces a sound with posterior articulation (such as ga, ki, or ke) onto sounds with frontal articulation (such as ba, pe or pi). You can take a look at the individual sounds and silent articulations used for these demonstrations, at the end of the video-clip. 

Demo videos of the McGurk illusion

Demo video - The McGurk-MacDonald illusion: When we can hear the movements of the lips (Part 1)

What you have to do?

This video-clip shows several demonstrations using simple meaningless syllables. In each, you will be asked to pay attention to what you hear the person in the screen say. You will be asked to hear the syllable three times while watching the video normally (audio-visual condition) and then, hear the syllable three more times but with your eyes closed (audio-only condition). You will realize that you *hear* different things depending on whether you *see* the speaker or not.

 

Demo video - The McGurk-MacDonald illusion: When we can hear the movements of the lips (Part 2)

What you have to do?

There are four demonstrations with words. In each, you will be asked to pay attention to the word you hear the person in the screen say. Each word is repeated 6 times: The first 3 times you will be asked to hear the word while watching the video normally (audio-visual condition) and the next 3 to hear the word again but with your eyes closed (audio-only condition). You will realize that you *hear* different words depending on whether you *see* the speaker or not. There are 3 words in Spanish and 1 in English. At the end of the clip, we try the effect with a whole sentence, but the result is more modest, since the dubbing procedure is very challenging.

 

 

Rubber-hand and the Pinocchio: Altered body representations

What is it?

In the rubber-hand illusion (RHI) we can come to feel as if a fake hand is part of ourbody, and even localize tactile sensations in this hand. In the Pinocchio-nose illusion we have the vivid sensation that our nose is longer than normal.

Why does it happen?

Our body representation, that is, the mental representation of the position and extension of the distinct parts of the body, is formed by the integration of information from several sensorly modalities such as the propioception (muscles position), vision and touch. Normally, these sensory informations are congruent, and our brain tends to combines them in an efficient way. However, this type of "body" illusions capitalize on disparities between vision and propioception/touch information about hand-location (rubber-hand) or between propioception and touch in the case of Pinocchio-nose. This phenomenon proves that our body representation is more plastic than we could think initially. This plasticity is useful to adapt to changes in our body size and relative proportions (e.g., during growth), or to facilitate the use of tools and other external components to the body (e.g., drive a car).

Photo gallery

More information

In Spanish

E. Azañón y S. Soto-Faraco. El espacio a través del tacto. Mente y Cerebro, n. 49. Julio/Agosto 2012

Web

In English

  • Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press;
  • S. Soto-Faraco, C. Spence, D. Lloyd and A. Kingstone, (2004). Moving multisensory research along: Motion perception across sensory modalities. Curr. Dir. Psychol. Sci., 13, 29–32
  • McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746-748. ;
  • Botvinick, M. & Cohen, J. (1998) Rubber hands “feel” touch that eyes see. Nature 391(6669):756.