Atrás LUTEST

Language Understanding Test Sets

Language Understanding Test Sets
LUTEST objective is the creation of test sets and an evaluation methodology that provide significant evidence about the linguistic generalization capabilities of deep learning methods applied to natural language processing.

LUTEST objective is the creation of test sets and an evaluation methodology that provide significant evidence about the linguistic generalization capabilities of deep learning methods applied to natural language processing.  For the last years, there have been different works on building test sets and evaluation methods for the purpose of assessing the language understanding capabilities of deep neural models and what information they select and encode. However, there is much work to be done yet, in particular from a linguistically-motivated perspective. LUTEST attempts to base the assessment of deep language model's generalization capabilities on the linguistics-grounded hypothesis that, if actually generalizing, whatever difference in the representations of two same meaning, but different in structure sentences, this difference will show up the same in a significant number of sentence pairs which exhibit the same structure-transformational phenomenon, despite of any lexical variation.

From 1/6/2020 to 31/5/2023

Investigadores principales

Núria Bel
Project PID2019-104512GB-I00 funded by the Ministerio de Ciencia e Innovación (Spain)