eTranslation excels at WMT 2019

eTranslation amongst top ranking engines in a competition with over 150 other machine translation systems

Screen capture of eTranslation's browser user interface

Quick facts

• Event: The fourth Conference on Machine Translation (WMT 2019)
• Organisation: European Commission’s CEF Telecom Automated Translation programme with DGT, DG CNECT and DIGIT
• Objective: To experiment and validate ideas for eTranslation’s future development
• Outcome: Promising results in all categories participated
• CEF Building Block: eTranslation

eTranslation tops the charts

This year’s Conference on Machine Translation (WMT 2019) took place in Florence, Italy, between August 1-2, 2019. During these two days, 153 systems from more than 40 groups competed on translation tasks and for the first time, the European Commission’s eTranslation solution was among them. Despite the strong competition, eTranslation achieved very good results in all featured translation categories, including French to German, Russian to English, English to German and English to Lithuanian.

For the European Commission’s Directorate-General for Translation (DG T), the WMT was an opportunity to challenge its translation engines in order to improve the services offered to European Union institutions and Member State public authorities. The primary objectives of the WMT are to evaluate the state of the art in machine translation, to disseminate common test sets and public training data, and to refine evaluation and estimation methodologies for machine translation.

From novice to notable

eTranslation, formerly a novice in machine translation competitions, proved to have very promising capabilities. The tool ranked second in French to German, third in Russian to English, and placed among the top ten in English to German and in English to Lithuanian. The pan-European effort to break down language barriers have paid off, as eTranslation compared favourably with many global corporate giants, research centres and leading universities.

What makes eTranslation's results even more remarkable, is that it had very few computational resources to spare for the competition. Detailed information on eTranslation’s submissions, including information on data preparation, trainings and translation results, are available at the link below. The complete WMT 2019 findings are also available for download.

WMT 2019 eTranslation submission

WMT 2019 findings

How was the assessment done?

To prepare for the WMT, the eTranslation team customised the machine translation engine to focus on translating news and more generic web content. This is because eTranslation used by public administrations to communicate across borders is specialised in formal, technical and academic language styles and subjects, such as legislation, public health, finance and cultural heritage. Therefore, the system had to be tweaked to handle the everyday language of journalists.

The test data for the 2019 translation task was selected from online news sources, with the exception of the French-German set. The latter consisted of translating news texts between French and German (in both directions) on the topic of the European elections. To achieve higher quality results in this task, the eTranslation team used a method called ‘guided topic modelling’ in order to select additional training material for the machine translation system. As in previous years, translations were produced in specific for the challenge. 

All systems were evaluated using the so-called ‘direct assessment’ procedure, where human assessors rate the machine translation output on a scale from 0 to 100. In order to ensure comparability of results, system parameters were tuned with a common set of parallel and monolingual corpora for training translation models and development sets.

What’s next?

By participating in the WMT, the eTranslation team was able to experiment and validate ideas in a competitive environment. The results and findings of the competition will go towards the development of new machine translation engines, as the eTranslation team envisages further expanding to new languages and domains. These new developments are supported by the European Language Resource Coordination (ELRC) that facilitates the collection of language resources under the Connecting Europe Facility - Automated Translation programme.