How should the quality of technological and artificial intelligence tools be evaluated? Is it only necessary to consider whether the products they generate are adapted to user needs or should other factors related to their social or environmental impact be weighed up? On Monday, 24 February, the UPF Poblenou campus hosted a seminar to reflect on these issues, focusing on the evaluation of technologies applicable to the field of translation and linguistics.

Joss Moorkens, an associate professor at the School of Applied Languages and Intercultural Studies at Dublin City University tackled the matter. Moorkens is also an expert in translation ethics and technologies, which is why he was the keynote speaker at the seminar, organized by the IULATERM (Lexicon and Technology) research group of the UPF Department of Translation and Language Sciences.

An issue of growing interest given the major impact of technology on the labour market and its effects on the environment

Moorkens presented his ideas for evaluating technologies, and particularly those applicable to the field of translation and linguistics, taking ethical criteria and social and environmental sustainability into account. The subject is arousing increasing social interest for several reasons, mainly due to the major impact of technology and AI on jobs in the field of translation and linguistics and the need to reverse climate change. It should be taken into account that technology and AI tools have a huge environmental impact. For example, experiments to train and improve AI natural language processing applications generate a carbon footprint seven times greater than a person’s average yearly consumption, as Moorkens explained during his presentation.

Evaluating technology based on three pillars: people, planet and performance

In this context, the expert outlined a proposal to evaluate translation and linguistic technologies that takes an existing model for applying ethical criteria to business as a reference, the so-called “Triple bottom line”. This model was devised in 1997 by Johan Elkington, one of the world’s leading experts on sustainability in the business world, and three decades later Moorkens has adapted it to the translation technology sector.

The original model took into account three factors for assessing ethics in business: the impact on people’s well-being, the impact on the planet, and its benefits, not only from an economic, but also from a social or a cultural point of view. Moorkens also proposes taking into account the impact of technologies on the situation of professionals with a variety of profiles (translators, people who work for technological platforms in the sector, people who interpret or transcribe speeches, etc.) or their ecological footprint, beyond the performance of the applications. 

Moorkens stressed that the evaluation of technology cannot be limited to applying metrics on the quality or accuracy of translations, but must consider factors related to efficiency and sustainability. For the expert, most current research focuses on the creation of higher performing technologies and ignores the social and environmental side, pushing AI to evolve in the wrong direction.

Joss Moorkens (Dublin City University): “Delegating to Big Techs means losing control over the evolution of these tools”

In this regard, he warned that research in translation technologies is currently led by the big technology companies and it is relying on research centres and universities to regain the prominence they had had years ago. “Delegating to Big Techs means losing control over the evolution of these tools”, he warned.

Moorkens was also critical of the current technology assessment model: “There is a general tendency to focus on the advantages of technology and to ignore the negative aspects”. He also questioned whether certain metrics -with biased measurement systems- should have such an impact when making decisions with a social impact.

Moorkens not only criticized the fact that these evaluation models do not take social and environmental criteria into account, but he also questioned whether they take into account the technical shortcomings that translation technologies still have. Although they have improved in recent years -better interpreting the meaning of a word in its context, for instance- they still have significant their shortcomings.

To improve current evaluation models and promote the sustainable development of translation technologies, Moorkens endorsed seeking a balance between enhancing the performance of these applications and preserving the future of the planet. To this end, he considered that there is a need to raise awareness among the different stakeholders in the translation technology sector (technology developers, translation professionals and agencies, users, etc.) so that they take ethical criteria into account. He also urged future graduates and professionals in the industry to include these criteria in the evaluation of technology.