Articles and book chapters Articles and book chapters

Return to Full Page

Permutation equivariant models for compositional generalization in language

  • Authors
  • Gordon, J.; Lopez-Paz, D.; Baroni, M.; Bouchacourt, D.
  • UPF authors
  • BARONI ., MARCO;
  • Authors of the book
  • -
  • Book title
  • Proceedings of ICLR 2020 (International Conference on Learning Representations)
  • Publication year
  • 2020
  • Pages
  • -
  • ISBN
  • ISBN-2725
  • Abstract
  • Humans understand novel sentences by composing meanings and roles of core language components. In contrast, neural network models for natural language modeling fail when such compositional generalization is required. The main contribution of this paper is to hypothesize that language compositionality is a form of group-equivariance. Based on this hypothesis, we propose a set of tools for constructing equivariant sequence-to-sequence models. Throughout a variety of experiments on the SCAN tasks, we analyze the behavior of existing models under the lens of equivariance, and demonstrate that our equivariant architecture is able to achieve the type compositional generalization required in human language understanding.
  • Complete citation
  • Gordon, J.; Lopez-Paz, D.; Baroni, M.; Bouchacourt, D.. Permutation equivariant models for compositional generalization in language. In: -. Proceedings of ICLR 2020 (International Conference on Learning Representations). 1 ed. 2020.