Articles and book chapters
Linguistic generalization and compositionality in modern artificial neural networks
- Authors
- Baroni, Marco
- UPF authors
- BARONI ., MARCO;
- Type
- Scholarly articles
- Journal títle
- Philosophical Transactions of the Royal Society. B: Biological Sciences
- Publication year
- 2020
- Volume
- 375
- Number
- 1791
- Pages
- 20190307-0
- ISSN
- 1471-2970
- Publication State
- Published
- Abstract
- In the last decade, deep artificial neural networks have achieved astounding performance in many natural language-processing tasks. Given the high productivity of language, these models must possess effective generalization abilities. It is widely assumed that humans handle linguistic productivity by means of algebraic compositional rules: are deep networks similarly compositional? After reviewing the main innovations characterizing current deep language-processing networks, I discuss a set of studies suggesting that deep networks are capable of subtle grammar-dependent generalizations, but also that they do not rely on systematic compositional rules. I argue that the intriguing behaviour of these devices (still awaiting a full understanding) should be of interest to linguists and cognitive scientists, as it offers a new perspective on possible computational strategies to deal with linguistic productivity beyond rule-based compositionality, and it might lead to new insights into the less systematic generalization patterns that also appear in natural language. This article is part of the theme issue `Towards mechanistic models of meaning composition¿.
- Complete citation
- Baroni, Marco. Linguistic generalization and compositionality in modern artificial neural networks. Philosophical Transactions of the Royal Society. B: Biological Sciences 2020; 375(1791).
Bibliometric indicators
- 11 times cited Scopus
- 10 times cited WOS
- Índex Scimago de 3.051 (2019)