Integrating sensory modalities in the domain of time

One classic problem in multisensory integration is the natural asynchrony of information transmission in different sensory modalities. Differences in physical differences in speed of light and sound, as well as physiological differences in transduction and neural transmission times prevent that events that occurred physically synchronous at source, will arrive together in the brain. Think of the extreme example of thunder and lighting, or the more subtle but annoying delays in live TV broadcasts over satellite. There are perceptual mechanisms that are capable of handling these sensory discrepancies that have revealed a surprising flexibility of our perception to temporal asynchronies across modalities. For example, our window of perceived simultaneity is hundredths of milliseconds wide. Also, if exposed to a constant  cross-modal asynchrony during a short period of time (in the order of minutes), the width of this window, and even its center, can change. This means that our perception of cross-modal synchrony changes as a function of recent experience. What current research has yet to answer is, how these temporal binding processes occur amidst the sensory chaos of real complex environments. One important question of interest in this project is to provide an answer to this question by understanding that attention plays in this temporal adaptation mechanisms.

Selecting sensory modalities in the domain of time

In order to process the incoming information stream of our everyday life, not only binding of related information within and across our senses is important, but also filtering and selecting relevant from irrelevant information, based on modality attributes. Aside from the well studied case of selection by spatial attention (such as when we expect events at particular spatial locations and not others), an important type of selection is attention to time. We are sometimes able to predict (and therefore prepare for) the moment at which relevant information may appear, whereas at other times important events occur in a completely unexpected fashion. Our interest is how these temporal selection mechanisms operate in scenarios that contain events in various sensory modalities, that is, multisensory environments. This is important because it is how temporal attention mechanisms must work in everyday life scenarios.  In order to do so, we are utilizing psychophysical paradigms and neuroimaging methods (EEG). The goal of this project is thereby to link the behavioural outcome of temporal attention with underlying prestimulus changes in neural oscillatory activity. 

Representative publications:

Mühlberg, S., Oriolo G., & Soto-Faraco S. (2014).  Cross-modal decoupling in temporal attention. European Journal of Neuroscience.