Articles and book chapters Articles and book chapters

Return to Full Page

"Show me the cup": reference with continuous representations

  • Authors
  • Boleda G, Baroni M, Padó S
  • UPF authors
  • BOLEDA TORRENT, GEMMA; BARONI ., MARCO;
  • Authors of the book
  • Gelbukh, A. (ed.)
  • Book title
  • International Conference on Computational Linguistics and Intelligent Text Processing
  • Publisher
  • Springer
  • Publication year
  • 2018
  • Pages
  • 209-224
  • ISBN
  • 978-3-319-77116-8
  • Abstract
  • One of the most basic functions of language is to refer to objects in a shared scene. Modeling reference with continuous representations is challenging because it requires individuation, i.e., tracking and distinguishing an arbitrary number of referents. We introduce a neural network model that, given a de nite description and a set of objects represented by natural images, points to the intended object if the expression has a unique referent, or indicates a failure, if it does not. The model, directly trained on reference acts, is competitive with a pipeline manually engineered to perform the same task, both when referents are purely visual, and when they are characterized by a combination of visual and linguistic properties.
  • Complete citation
  • Boleda G, Baroni M, Padó S. "Show me the cup": reference with continuous representations. In: Gelbukh, A. (ed.). International Conference on Computational Linguistics and Intelligent Text Processing. 1 ed. New York: Springer; 2018. p. 209-224.