Analysis of translation errors and evaluation of pre-editing rules for the translation of English news texts into Spanish with Lucy LT
Abstract
In this paper we study the effect of pre-editing rules on the quality of the translations produced by the MT system Lucy LT when translating English news texts into Spanish. We carried out an error annotation of the first 200 segments of the News Crawl: articles from 2014 corpus and devised a set of 8 pre-editing rules. The application of these rules to a different set of segments from the same corpus results in a reduction of the word error rate of about 11%.
Keywords
machine translation, patterns of translation errors, pre-editing rules, Lucy LT, machine translation evaluation, error annotation, error analysis, Multidimensional Quality MetricsReferences
Burchardt, A. & Lommel, A. (2014). Practical Guidelines for the Use of MQM in Scientific Research on Translation Quality <http://www.qt21.eu/downloads/MQM-usage-guidelines.pdf> [Download on 26 February 2015].
EMNLP 2015: Tenth Workshop on Statistical Machine Translation. (2015). News Crawl: articles from 2014 [corpus] <http://www.statmt.org/wmt15/training-monolingual-news-crawl/news.2014.en.shuffled.gz> [Download on 8 March 2015].
Forcada, M. L. et al. (2011). “Apertium: a free/open-source platform for rule-based machine translation”. Machine Translation, 25(2), 127-144.
Görög, A. (2014). “Quantifying and benchmarking quality: the TAUS
Dynamic Quality Framework”. Tradumática, 12, 443-454 <http://revistes.uab.cat/tradumatica/article/view/n12-gorog2/pdf> [Download on 8 March 2015].
Hutchins, J. (1996). Evaluation of machine translation and translation tools. Norfolk: University of East Anglia.
Koby, G. S. et al. (2014). “Defining Translation Quality”. Tradumática, 12, 413-420. <http://revistes.uab.cat/tradumatica/article/view/n12-koby-fields-hague-etal/pdf> [Download on 8 March 2015].
Kohl, J. R. (2008). The Global English Style Guide: Writing Clear, Translatable Documentation for a Global Market. Cary (NC): SAS Institute Inc.
Lommel, A. & Melby, A. K. (2014). Multidimensional Quality Metrics (MQM) Issue types. Version 0.2.0 (2014-08-19) <http://www.qt21.eu/mqm-definition/issues-list-2014-08-19.html> [Download on 26 February 2015].
Lommel, A. et al. (2014). “Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics”. Tradumática, 12, 455-463 <http://revistes.uab.cat/tradumatica/article/view/n12-lommel-uzskoreit-burchardt/pdf> [Download on 12 March 2015].
Lommel, A. et al. (2015a). Multidimensional Quality Metrics (MQM) Definition. Version 0.3.0 (2015-01-20) <http://www.qt21.eu/mqm-definition/definition-2015-01-20.html> [Download on 26 February 2015].
Lommel, A.et al. (2015b). Multidimensional Quality Metrics (MQM) Definition. Version 0.9.3 (2015-06-16) <http://www.qt21.eu/mqm-definition/definition-2015-06-16.html> [Download on 30 June 2015].
Lommel, A. et al. (2015c). Multidimensional Quality Metrics (MQM) Issue types. Version 0.5.1 (2015-06-16) <http://www.qt21.eu/mqm-definition/issues-list-2015-05-27.html> [Download on 30 June 2015].
Plitt, M. & Masselot, F. (2010). “A Productivity Test of Statistical Machine
Translation Post-Editing in a Typical Localisation Context”. The Prague
Bulletin of Mathematical Linguistics, 93, 7-16 <https://ufal.mff.cuni.cz/pbml/93/art-plitt-masselot.pdf> [Download on 2 November 2015]. See also <http://langtech.autodesk.com/productivity.html>.
Sánchez-Martínez, F. (2012). “Motivos del creciente uso de la traducción automática seguida de posedición”. Tradumática, 10, 150-156 <http://revistes.uab.cat/tradumatica/article/view/24/PDF> [Download on 8 March 2015].
TAUS. (2011a). Adequacy/Fluency Evaluation <https://www.taus.net/knowledgebase/index.php?title=Adequacy/fluency_evaluation> [Download on 4 March 2015].
TAUS. (2011b). Adequacy/Fluency Process Guide <https://www.taus.net/knowledgebase/index.php?title=Adequacy/fluency_process_guide> [Download on 4 March 2015].
TAUS. (2011c). Dynamic Quality Framework <https://www.taus.net/knowledgebase/index.php?title=Dynamic_Quality_Framework> [Download on 4 March 2015].
TAUS. (2011d). Error Typology <https://www.taus.net/knowledgebase/index.php?title=Error_typology_evaluation> [Download on 4 March 2015].
Thicke, L. (2011). “Improving MT results: a study”. Multilingual (Jan/Feb 2011), 22(1), 37-40 <http://lexworks.com/cms/wp-content/uploads/2012/07/Improving_MT_Results_Multilingual_01-11.pdf> [Download on 15 March 2015].
Van der Meer, A. (2015). “DQF and MQM Harmonized to Create an Industry-Wide Quality Standard” <https://www.taus.net/think-tank/news/press-release/dqf-and-mqm-harmonized-to-create-an-industry-wide-quality-standard> [Download on 30 June 2015].
Published
Downloads
Copyright (c) 2016 Julia Mercader-Alarcón, Felipe Sánchez-Matínez
This work is licensed under a Creative Commons Attribution 4.0 International License.
