The goal of this project is to push for a breakthrough in language technologies with respect to visual signals, with a special focus on Sign Languages (SLs), in which visual signals carry the entire communicative burden. To do so, it will be based on novel interdisciplinary collaborations between specialists in computer vision and machine learning (to extract relevant features from visual communicative signals, discover patterns in the data, and quantitatively assess them for recognition and synthesis), linguists specialised in multimodal communication and sign language (to construct predictive linguistic theories based on low-level patterns found in the data), and specialists in computer graphics (to produce virtual characters with sign language and multimodal conversational abilities).

UPF is uniquely positioned to make groundbreaking contributions to this enterprise, with world-class experts in all relevant fields – computer vision, machine learning, multimodal communication, sign language, and computer graphics. The present project will bring these researchers from different UPF departments together and will moreover establish a long-term collaboration with an interdisciplinary team of computer scientists and linguists at the University of Amsterdam who share the same ambitions.