UPF-Cobalt Submission to WMT15 Metrics Task

  • Authors
  • Fomicheva, Marina; Bel Rafecas, Núria; Da Cunha Fanego, Iria; Malinovskiy, Anton
  • UPF authors
  • DA CUNHA FANEGO, IRIA; BEL RAFECAS, NÚRIA;
  • Authors of the book
  • Màrquez, L. (ed.)
  • Book title
  • Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015)
  • Publisher
  • Association for Computational Linguistics
  • Publication year
  • 2015
  • Pages
  • 373-379
  • ISBN
  • 978-1-941643-32-7
  • Abstract
  • An important limitation of automatic evaluation metrics is that, when comparing Machine Translation (MT) to a human reference, they are often unable to discriminate between acceptable variation and the differences that are indicative of MT errors. In this paper we present UPF-Cobalt evaluation system that addresses this issue by penalizing the differences in the syntactic contexts of aligned candidate and reference words. We evaluate our metric using the data from WMT workshops of the recent years and show that it performs competitively both at segment and at system levels.
  • Complete citation
  • Fomicheva, Marina; Bel Rafecas, Núria; Da Cunha Fanego, Iria; Malinovskiy, Anton. UPF-Cobalt Submission to WMT15 Metrics Task. In: Màrquez, L. (ed.). Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015). 1 ed. East Stroudsburg PA: Association for Computational Linguistics; 2015. p. 373-379.