Traducció automàtica neuronal anglès-català: tecnologia punta, qualitat i productivitat
Resum
Els recents i importants canvis i avenços tecnològics han consolidat la traducció automàtica (TA) com un actor clau a tenir en compte en el món dels serveis lingüístics. En molts casos, és fins i tot un actor essencial a causa de les limitacions de pressupost i temps. Últimament, la recerca en TA ha rebut molta atenció i se n’ha augmentat l’ús per part d’usuaris professionals i aficionats. De tota manera, la recerca s’ha centrat principalment en les combinacions lingüístiques amb grans quantitats de corpus disponibles en línia (per exemple, anglès-castellà). La situació de les llengües minoritàries o no oficials a un estat, com el català, és diferent. Aquest estudi analitza el nou motor de traducció automàtica neuronal de codi obert de Softcatalà i el compara amb el Google Traductor i l’Apertium en la combinació lingüística anglès-català. Tot i que els desenvolupadors de motors de traducció automàtica fan servir mètriques automàtiques per avaluar-los, l’avaluació humana continua sent la pràctica de referència, tot i el cost que implica. Per mitjà de les eines TAUS DQF, s’ha avaluat la qualitat de la traducció (en termes de classificació relativa, adequació i fluïdesa) i la productivitat (comparant els temps d’edició i les distàncies) amb la participació d’11 avaluadors. Els resultats mostren que el traductor de Softcatalà ofereix una qualitat i productivitat majors que els altres motors analitzats.
Paraules clau
traducció automàtica, tecnologies de la traducció, avaluació humana, català, qualitat de la traducció, avaluació de la qualitatReferències
Ahrenberg, L. (2017). Comparing Machine Translation and Human Translation: A Case Study. In: The Proceedings of the Workshop on Human-Informed Translation and Interpreting Technology (HiT-IT): Varna, Bulgaria, september 2017. Shoumen, Bulgaria: Incoma, pp. 21–28. <https://doi.org/10.26615/978-954-452-042-7_003>. [Accessed: 20221209].
Barrault, L.; Biesialska, M.; Bojar, O.; Costa-jussà, M. R.; Federmann, C.; Graham, Y.; et al. (2020a). Findings of the 2020 Conference on Machine Translation (WMT20). In: Proceedings of the Fifth Conference on Machine Translation (WMT), online, November 19-20, 2020. Stroudsburg, PA: Association for Computational Linguistics, pp. 1–55. <https://aclanthology.org/2020.wmt-1.1.pdf>. [Accessed: 20221209].
Barrault, L.; Bojar, O.; Bougares, F.; Chatterjee, R.; Costa-jussà, M. R.; Federmann, C.; et al. (eds.) (2020b). Proceedings of the Fifth Conference on Machine Translation (WMT), online, November 19-20, 2020. Stroudsburg, PA: Association for Computational Linguistics. <https://aclanthology.org/volumes/2020.wmt-1/>. [Accessed: 20221209].
Barrault, L.; Bojar, O.; Bougares, F.; Chatterjee, R.; Costa-jussa, M. R.; Federmann, C.; et al. (eds.). (2021). Proceedings of the Sixth Conference on Machine Translation, online, November 10-11, 2021. Stroudsburg, PA: Association for Computational Linguistics. <https://aclanthology.org/2021.wmt-1.0.pdf>. [Accessed: 20221209].
Bentivogli, L.; Bisazza, A.; Cettolo, M.; Federico, M. (2016). Neural versus Phrase-Based Machine Translation Quality: a Case Study. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas November 1-5 2016. Stroudsburg, PA: Association for Computational Linguistics, pp. 257–267. <https://doi.org/10.18653/v1/D16-1025>. [Accessed: 20221209].
Bojar, O.; Chatterjee, R.; Federmann, C.; Graham, Y.; Haddow, B.; Huck, M.; et al. (2016). Findings of the 2016 Conference on Machine Translation. In: Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers: Berlin. Germany, August 11-12, 2016. Stroudsburg, PA: Association for Computational Linguistics, pp. 131–198. <https://doi.org/10.18653/v1/W16-2301>. [Accessed: 20221209].
Briva-Iglesias, V., (2021). Traducción humana vs. traducción automática: análisis contrastivo e implicaciones para la aplicación de la traducción automática en traducción jurídica. Mutatis mutandis: revista latinoamericana de traducción, v. 14, n. 2, pp. 571–600. <https://doi.org/10.17533/udea.mut.v14n2a14>. [Accessed: 20221209].
Callison-Burch, C.; Fordyce, C.; Koehn, P.; Monz, C.; Schroeder, J. (2007). (Meta-) evaluation of machine translation. In: StatMT'07: Proceedings of the Second Workshop on Statistical Machine Translation: Prague, June 2007. Stroudsburg, PA: Association for Computational Linguistics, pp. 136–158. <https://doi.org/10.3115/1626355.1626373>. [Accessed: 20221209].
Castilho, S.; Doherty, S.; Gaspari, F.; Moorkens, J. (2018). Approaches to Human and Machine Translation Quality Assessment. In: Moorkens, J.; Castilho, S.; Gaspari, F.; Doherty, S. (eds.). Translation Quality Assessment: From Principles to Practice. Cham: Springer. (Machine Translation: Technologies and Applications; 1), pp. 9–38. <https://doi.org/10.1007/978-3-319-91241-7_2>. [Accessed: 20221209].
Castilho, S.; Moorkens, J.; Gaspari, F.; Calixto, I.; Tinsley, J., Way, A. (2017a). Is Neural Machine Translation the New State of the Art? The Prague Bulletin of Mathematical Linguistics, n. 108 (June), pp. <109–120. https://doi.org/10.1515/pralin-2017-0013>. [Accessed: 20221209].
Castilho, S.; Moorkens, J.; Gaspari, F.; Sennrich, R.; Sosoni, V.; Georgakopoulou, P.; et al. (2017b). A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators. In: Kurohashi, S.; Fund, P. (ed.). Proceeddings of Machine Translation Summitt XVI: Research Track, pp. 116-131. <https://aclanthology.org/2017.mtsummit-papers.10/>. [Accessed: 20221209].
Cronin, M. (2012). Translation in the Digital Age. 1st ed. Milton Park, Abigdon [etc.]): Routledge. <https://doi.org/10.4324/9780203073599>. [Accessed: 20221209].
EUATC (2020). 2020 European Language Industry Survey launched. <https://euatc.org/industry-surveys/2020-language-industry-survey-launched/>. [Accessed: 20221209].
Faes, F. (2016). Disruption Turns 10 as Google Translate Comes of Age. Slator. <https://slator.com/disruption-turns-10-google-translate-comes-age/>. [Accessed: 20221209].
Forcada, M. L.; Ginestí-Rosell, M.; Nordfalk, J.; O’Regan, J.; Ortiz-Rojas, S.; Pérez-Ortiz, J. A.; et al. (2011). Apertium: a free/open-source platform for rule-based machine translation. Machine Translation, n. 25, pp. 127–144. <https://doi.org/10.1007/s10590-011-9090-0>. [Accessed: 20221209].
Freitag, M.; Foster, G.; Grangier, D.; Ratnakar, V.; Tan, Q.; Macherey, W. (2021). Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation. Transactions of the Association for Computational Linguistics, n. 9, pp. 1460-1474. <https://doi.org/10.1162/tacl_a_00437>. [Accessed: 20221209].
Freitag, M.; Grangier, D.; Caswell, I. (2020). BLEU might be Guilty but References are not Innocent. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), online, pp. 61-71. <https://doi.org/10.18653/v1/2020.emnlp-main.5>. [Accessed: 20221209].
Görög, A. (2014). Quantifying and benchmarking quality: the TAUS Dynamic Quality Framework. Revista Tradumàtica: tecnologies de la traducció, n. 12, pp. 443-454. <https://doi.org/10.5565/rev/tradumatica.66>. [Accessed: 20221209].
Graham, Y.; Baldwin, T.; Moffat, A.; Zobel, J. (2013). Continuous Measurement Scales in Human Evaluation of Machine Translation. In: Proceedings of the 7th Linguistics Annotation Workshop and Interoperability with Discourse: Sofia, Bulgaria, August 8-9 2013. Stroudsburg, PA: Association for Computational Linguistics, pp. 33-41 <https://aclanthology.org/W13-2305.pdf>. [Accessed: 20221209].
Kocmi, T.; Federmann, C.; Grundkiewicz, R.; Junczys-Dowmunt, M.; Matsushita, H.; Menezes, A. (2021). To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation. In: Proceedings of the Sixth Conference on Machine Translation (WMT), November 10-11, 2021. Stroudsburg, PA: Association for Computational Linguistics, pp. 478-494. <https://aclanthology.org/2021.wmt-1.57.pdf>. [Accessed: 20221209].
Koehn, P.; Monz, C. (2006). Manual and automatic evaluation of machine translation between European languages. In: StatMT'06: Proceedings of the Workshop on Statistical Machine Translation, New York City, June 2006. Stroudsburg, PA: Association for Computational Linguistics, pp. 102-121. <https://doi.org/10.3115/1654650.1654666>. [Accessed: 20221209].
Läubli, S.; Castilho, S.; Neubig, G.; Sennrich, R.; Shen, Q.; Toral, A. (2020). A Set of Recommendations for Assessing Human–Machine Parity. In: Language Translation. Journal of Artificial Intelligence Research, v. 67, pp. 653-672. <https://doi.org/10.1613/jair.1.11371>. [Accessed: 20221209].
Läubli, S.; Sennrich, R.; Volk, M. (2018). Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31-November 4, 2018. Stroudsburg, PA: Association for Computational Linguistics, pp. 4791–4796. <https://doi.org/10.18653/v1/D18-1512>. [Accessed: 20221209].
López-Pereira, A. (2019). Traducción automática neuronal y traducción automática estadística: percepción y productividad. Revista Tradumàtica: tecnologies de la traducció, n. 17, pp. 1-19. <https://doi.org/10.5565/rev/tradumatica.235>. [Accessed: 20221209]
Martín-Mor. A.; Piqué, R.; Sánchez-Gijón, P. (2016). Tradumàtica: Tecnologies de la traducció. EUMO, Vic.
Moorkens, J. (2022). Ethics and machine translation. In: Kenny, Dorothy (ed.). Machine translation for everyone: Empowering users in the age of artificial intelligence. Berlin: Language Sciencie Press. (Translation and Multilingual Natural Language Processing; 18), pp. 121-140.
Moorkens, J.; Castilho, S.; Gaspari, F.; Doherty, S. (eds.) (2018). Translation Quality Assessment: From Principles to Practice. Cham: Springer. (Machine Translation: Technologies and Applications; 1). <https://doi.org/10.1007/978-3-319-91241-7>. [Accessed: 20221209].
O’Brien, S. (2012). Towards a dynamic quality evaluation model for translation. The Journal of Specialised Translation, n. 17 (January), pp. 55-77. <https://www.jostrans.org/issue17/art_obrien.pdf>. [Accessed: 20221209].
Papineni, K.; Roukos, S.; Ward, T.; Zhu, W.-J. (2001). BLEU: a method for automatic evaluation of machine translation. In: ACL'02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, pp. 311-318. <https://doi.org/10.3115/1073083.1073135>. [Accessed: 20221209].
Pitman, J. (2021). Google Translate: One billion installs, one billion stories. Google. <https://blog.google/products/translate/one-billion-installs/>. [Accessed: 20221209].
Popović, M. (2015). chrF: character n-gram F-score for automatic MT evaluation. In: Proceedings of the Tenth Workshop on Statistical Machine Translation. Stroudsburg, PA: Association for Computational Linguistics, pp. 392–395. <https://doi.org/10.18653/v1/W15-3049>. [Accessed: 20221209].
Rei, R.; Stewart, C.; Farinha, A. C.; Lavie, A. (2020). COMET: A Neural Framework for MT Evaluation. In: Proceedings of the 2020 Conference on empirical Methods in Natural Language Processing (EMNLP), online. Stroudsburg, PA: Association for Computational Linguistics, pp. 2685-2702. <https://aclanthology.org/2020.emnlp-main.213/>. [Accessed: 20221209].
Snover, M.; Dorr, B.; Schwartz, R.; Micciulla, L.; Makhoul, J. (2016). A Study of Translation Edit Rate with Targeted Human Annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers. Assocaition for Machine Translation in the Americas, pp. 223-231. <https://aclanthology.org/2006.amta-papers.25.pdf>. [Accessed: 20221209].
Snover, M.; Madnani, N.; Dorr, B.; Schwartz, R. (2009). TER-Plus: paraphrase, semantic, and alignment enhancements to Translation Edit Rate. Machine Translation, n. 23, pp. 117–127. <https://doi.org/10.1007/s10590-009-9062-9>. [Accessed: 20221209].
Tillmann, C.; Vogel, S.; Ney, H.; Zubiaga, A.; Sawaf, H. (1997). Accelerated DP based search for statistical translation. <https://www.isca-speech.org/archive_v0/archive_papers/eurospeech_1997/e97_2667.pdf>. [Accessed: 20221209].
Toral, A. (2020). Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019. In: Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. European Association for Machine Translation, pp. 185–194. <https://aclanthology.org/2020.eamt-1.20.pdf>. [Accessed: 20221209].
Wu, Y.; Schuster, M.; Chen, Z.; Le, Q. V.; Norouzi, M.; Macherey, W.; et al. (2016). Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. Cornell University. <https://doig.org/10.48550/arXiv.1609.08144>. [Accessed: 20221209].
Ye-Yi Wang; Acero, A.; Chelba, C. (2003). Is word error rate a good indicator for spoken language understanding accuracy. In: 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721), pp. 577–582. <https://doi.org/10.1109/ASRU.2003.1318504>. [Accessed: 20221209].
Publicades
Descàrregues
Drets d'autor (c) 2022 Vicent Briva-Iglesias
Aquesta obra està sota una llicència internacional Creative Commons Reconeixement 4.0.
