Gesturing when speaking is something we all do, whether consciously or unconsciously, and is part of human communication. We usually use our hands, move our head, raise our eyebrows, point with our fingers, describe shapes with our hands and arms, and even turn our body. Gestures are many and varied and they play a very important role in learning a language and creating knowledge.

Researchers from the Prosodic Studies Group (GrEP) at the Department of Translation and Language Sciences at Pompeu Fabra University, together with the Department of Subject-Specific Education at the University of Girona, have published in the journal Frontiers in Psychology a study of how children use gestures to mark information structure in narrative discourse.

Information structure refers to how a speaker packages information so that the recipients of the message gain shared knowledge and how the speaker updates that knowledge by adding new information that moves communication forward. This packaging can be done simply verbally (the use of nouns vs pronouns), but also with intonation (emphasizing a new element) or with gesture.

The results show that at older ages, non-referential gestures, which do not visualize the definition of some aspect of speech, were far more numerous than referential ones.

When we speak we make gestures, which can be classified into two major groups. Referential gestures are the ones that visually represent what we are explaining, or the ones that are used to define a key word or concept for the discourse that is being constructed. The others are non-referential gestures that do not visually represent the content of what we are saying, but are also used to highlight important information and structure the discourse.

For example, a referential gesture would be to draw a ball with the hands when mentioning the word “ball”, or when saying about driving a car, moving the hands as if turning the steering wheel. A non-referential gesture would be to simply move the hands up and down rhythmically or raise the eyebrows and move the head. These movements do not express the specific meaning of the verbal content. They are often made by politicians during their speeches to emphasize important points.

The authors, together with the ICREA researcher Pilar Prieto, from UPF (Principal Investigator of the GrEP and project director), and Ingrid Vilà-Giménez, of the University of Girona, used the Audiovisual corpus of Catalan children’s narrative discourse development (collected by Vilà-Giménez) which contains 332 narratives interviewing 83 children from various schools in Girona at two stages of their lives: when they were aged between 5 and 6 years and when they were between 7 and 9 years old.

On this basis, they analysed the information structure of children’s discourse at three levels using the linguistic information structure annotation (LISA) guidelines, highlighting the elements that update shared knowledge:

- focus, or the general segment that reveals the new information that moves discourse forward

- comment, or the segment that adds information about an established topic

- referents, which may be new entities or entities that can be inferred by context.

The results show that at older ages, non-referential gestures, which do not visualize the definition of some aspect of speech, were far more numerous than referential ones. “When they are younger, they use these gestures to mark new referents”, explains Júlia Florit-Pons, joint principal author of the article. “The use of many more non-referential gestures indicates that children aged 7 to 9 know how to structure discourse in a very similar way to an adult. What changes is the use they make of gestures: while younger ones use gestures that represent what they are saying, older ones use them to emphasize what they are saying and to mark the elements that move discourse forward”.

The results of this study strengthen the idea that communication is multimodal and not purely verbal, as prosody and gestures contribute to both the semantic and pragmatic meaning of discourse.

Patrick Louis Rohrer, also joint principal author of the study, explains that the main goal was to understand the role of non-referential gestures in the creation of the information structure as children grew, to understand what role they play in children’s language development. Until then, there had only been studies that looked at how these elements were marked textually or in prosody (intonation, pitch, tone, etc.) but not so much multimodally including gestures. Our results suggest that at this age, children use gestures in a more abstract way, indicating progress in their cognitive development.

Florit-Pons works with children and the multimodal development of gestures and their relationship to language while Rohrer works with data on adults, seeking ways to label (annotate) gestures. “There is no single annotation system that serves the main gestural theories as explained by McNeill or Kendon. Hence we join forces to obtain one that takes into account not only gestures but also all aspects of communication, such as syntax, lexicon and prosody”.

The results of this study strengthen the idea that communication is multimodal and not purely verbal, as prosody and gestures contribute to both the semantic and pragmatic meaning of discourse.

Reference article:

Patrick Louis Rohrer, Júlia Florit-Pons, Ingrid Vilà-Giménez, Pilar Prieto. Children Use Non-referential Gestures in Narrative Speech to Mark Discourse Elements Which Update Common Ground. Frontiers of Psychology, January 2022. https://doi.org/10.3389/fpsyg.2021.661339