Articles and book chapters Articles and book chapters

Return to Full Page

Linguistic generalization and compositionality in modern artificial neural networks

  • Authors
  • Baroni, Marco
  • UPF authors
  • BARONI ., MARCO;
  • Type
  • Scholarly articles
  • Journal títle
  • Philosophical Transactions of the Royal Society. B: Biological Sciences
  • Publication year
  • 2020
  • Volume
  • 375
  • Number
  • 1791
  • Pages
  • 20190307-0
  • ISSN
  • 1471-2970
  • Publication State
  • Published
  • Abstract
  • In the last decade, deep artificial neural networks have achieved astounding performance in many natural language-processing tasks. Given the high productivity of language, these models must possess effective generalization abilities. It is widely assumed that humans handle linguistic productivity by means of algebraic compositional rules: are deep networks similarly compositional? After reviewing the main innovations characterizing current deep language-processing networks, I discuss a set of studies suggesting that deep networks are capable of subtle grammar-dependent generalizations, but also that they do not rely on systematic compositional rules. I argue that the intriguing behaviour of these devices (still awaiting a full understanding) should be of interest to linguists and cognitive scientists, as it offers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality, and it might lead to new insights into the less systematic generalization patterns that also appear in natural language. This article is part of the theme issue `Towards mechanistic models of meaning composition¿.
  • Complete citation
  • Baroni, Marco. Linguistic generalization and compositionality in modern artificial neural networks. Philosophical Transactions of the Royal Society. B: Biological Sciences 2020; 375(1791).
Bibliometric indicators
  • 11 times cited Scopus
  • 12 times cited WOS
  • Índex Scimago de 3.051 (2019)