Judith Ciarrussta presented her research on how babies learn to associate visual information with the sound of native and non-native syllables.

Infant neural adaptation to native versus foreign audio-visual speech perception

Poster D91 in Poster Session D, Wednesday, October 25, 4:45 - 6:30 pm CEST, Espace Vieux-Port

Judit Ciarrusta¹, Nuria Sebastian Galles¹; ¹Universitat Pompeu Fabra

Despite the immature cortical infrastructure newborn infants already show a preference for speech-like stimuli (Werker & Gervain, 2012). Throughout the first postnatal year, linguistic input helps the brain with significant structural and functional reorganization (Werker & Hensch, 2015). The increase in power of neural oscillations in faster frequency ranges, such as the gamma band appearing around 6 months, is an example of functional reorganization (Pivik et al., 2019). Higher gamma power has been associated with native language processing both in adults and infants in the auditory modality (Ortiz-Mantilla et al., 2013; Peña & Melloni, 2012). However, adult studies have shown visual speech can travel to temporal cortices even before auditory input arrives to cause a phase reset and induce oscillatory patterns necessary for audio-visual speech integration (Zhou et al., 2020). It is therefore likely that beyond native language auditory input, visual cues are also modulating oscillatory responses to speech. In fact, 4-month-old infants can discriminate among languages based on visual facial information, further highlighting the multimodal nature of speech input (Weikum et al., 2007). Adult studies have shown gamma oscillations are reflective of multisensory integration (Karthik et al., 2021), suggesting multisensory input in infants might boost an oscillatory pattern that further helps with the acquisition of native language. Thus, we hypothesized young infants will show a gamma band response to native audio-visual language phonemic contrasts in frontotemporal cortices indicative of both native language processing and multisensory integration. In this project, we designed an audio-visual oddball paradigm to test 6 to 8-month-old Spanish and/or Catalan native infants’ neural responses to native and foreign phonemic contrasts. For the pilot, we tested 11 infants (6 female) with an average age of 7.25 months from Barcelona, Spain. For the experimental procedure, the standard stimuli consisted of the Catalan/Spanish native contrast /da/ and the deviant stimuli consisted of the English contrast /tha/. The infants watched a total of 288 trials (1.75s each, 72 deviant), adding up to a total time of 10 min. To measure brain responses, we used a 128-sensor HydroCel Geodesic Sensor EEG Net and impedances were kept below 100 kΩ where possible. Data pre-processing was carried out using the APICE pipeline (Fló et al., 2022) and manually inspected to remove any remaining artefacts using EEGLAB software. Evoked response potential (ERP) analysis was carried out using EEGLAB software and time-frequency analysis was carried out using Fieldtrip. As expected, pilot ERP results in central electrodes showed a stronger response to deviant relative to standard stimuli. However, the increase in signal potential commenced after mouth movement onset rather than after auditory onset. Time-frequency analysis in left frontotemporal electrodes showed higher gamma power to native phonemic contrast relative to non-native phonemic contrast. A complex pattern of higher gamma power was observed around the onset of mouth movement onset. Although preliminary, we speculate the increase in signal potential and higher gamma power observed when the mouth movement starts might be related to the phase reset in auditory cortices triggered by preceding visual cues observed in adults.

Topic Areas: Language Development/Acquisition, Multisensory or Sensorimotor Integration