Research
Research lines
The interaction and coordination between prosody, gestures and meaning
It is well known that one of the fundamental aspects of human communication is the ability to use prosody and gestures, in addition to syntactic and lexical structures. Visual modality is very relevant, for example, in the alternation of word turns or in the expression of emotions, in the degree of confidence of the answers and in the structuring of discourse. It has been shown that there is a strong dependence between gestures and prosody. Recent research has shown that the most significant part of the gesture (called the stroke) is usually coordinated with the most prominent part of the speech (the stressed syllable or through intonation). But it is not yet clearl what are the prosodic anchors to which gestures are coordinated. This line of research has two objectives. On the one hand, it aims to analyze the bodily communicative signals that speakers send to each other when interacting, either through gestures, head movements, or facial expressions, that is to say, to study gestures in coordination with intonation and speech (conventional meaning). On the other hand, this line of research aims to investigate what the coordination patterns between prosodic and gestural structure are.
The benefits of prosody and gesture in language production and processing
It is well-known that an appropriate use of prosodic and gestural strategies is fundamental in language processing, and that these strategies can improve the understanding of the message expressed by the speaker. This project investigates the benefits of prosody and gesture (as well as the joint integration of gesture and prosody) in cognitive processing as well as in second language learning tasks. The general objective of this line of research is to determine how various prosodic and gestural strategies that we use in audiovisual messages (a) promote perceptual improvements and modify the listener's attention and memory levels, and (b) produce beneficial aspects in general learning tasks and the acquisition of first and second languages. This research line also investigates the communicative value of prosody and gestures that accompany speech (co-speech gestures) in populations with neurological developmental pathologies characterized by difficulties in the production of verbal and non-verbal language and in the patterns of social integration (such as in populations with specific language disorder, with autism spectrum disorder or with Asperger syndrome).
The development of prosody and gestures to convey meaning in child speech
The acquisition of prosody and gestures is known to begin early in language development. When babies are still in the womb, they can already identify the mother's voice and develop a sensitivity for recognizing the rhythmic and intonational patterns of their native language. Babies are also sensitive to visual stimuli from other human beings. For example, right after birth babies imitate the facial expressions of adults. We have evidence that children, from the first words, produce intonation patterns for different purposes. In recent years, research has shown that prosody and gesture together make an essential contribution to the pragmatic meaning of an utterance and play a crucial role in the listener's pragmatic decoding of sentences. However, our knowledge of the development of children's prosody and gestures, and their relationship to the acquisition of meaning, is still very limited. This line of research aims to contribute to filling this gap. The central hypothesis is that gestures and prosody help children significantly in the acquisition of pragmatic meaning. One of the aims of this research is to investigate how children use intonation and gestures in relation to learning different semantic meanings.
The cognitive processing of prosody and gesture
A question that has captured the interest of cognitive psychologists, media researchers, and linguists is how the brain processes audiovisual messages and, in particular, speech. Research on this subject has shown that the way someone says something is just as important as what they say. If the appropriate use of prosodic and gestural strategies is essential in the communicative process, it is possible to conclude that the study of the perception and processing of prosody and gestures is fundamental to understanding how the human cognitive system processes messages. Therefore, the general objective of this line of research is to determine how the different prosodic and gestural strategies applied to audiovisual messages cause differences in perception and modify the attention and memory levels of the listener.
Els nostres projectes d'investigació són finançats per fonts públiques, que inclouen: Agència de Gestió d'Ajuts Universitaris, Ministeri de Ciència i Innovació (MINECO) i German Research Foundation.
