Articles and book chapters Articles and book chapters

Return to Full Page

Understanding Visual Scenes

  • Authors
  • Silberer C, Uijlings J, Lapata M
  • UPF authors
  • Type
  • Scholarly articles
  • Journal títle
  • Natural Language Engineering
  • Publication year
  • 2018
  • Volume
  • 24
  • Number
  • 3
  • Pages
  • 441-465
  • ISSN
  • 1351-3249
  • Publication State
  • Published
  • Abstract
  • A growing body of recent work focuses on the challenging problem of scene understanding using a variety of cross-modal methods which fuse techniques from image and text processing. In this paper, we develop representations for the semantics of scenes by explicitly encoding the objects detected in them and their spatial relations. We represent image content via two well-known types of tree representations, namely constituents and dependencies. Our representations are created deterministically, can be applied to any image dataset irrespective of the task at hand, and are amenable to standard NLP tools developed for tree-based structures. We show that we can apply syntax-based SMT and tree kernel methods in order to build models for image description generation and image-based retrieval. Experimental results on real-world images demonstrate the effectiveness of the framework
  • Complete citation
  • Silberer C, Uijlings J, Lapata M. Understanding Visual Scenes. Natural Language Engineering 2018; 24(3): 441-465.
Bibliometric indicators
  • 1 times cited Scopus
  • 1 times cited WOS
  • Índex Scimago de 0.315(2018)