Avaluació humana de traducció automàtica neuronal i informe de progrés anual: estudi de cas del castellà al coreà

Ahrii Kim, Carme Colominas Ventura

Resum

Aquest article proposa la primera avaluació de traducció automàtica neuronal en la combinació lingüística espanyol-coreà. Per fer-ho s'han aplicat quatre mètodes d'avaluació humana: l'avaluació directa, la comparació a través de la classificació dels segments i l'anàlisi del temps i de l'esforç de postedició del text traduït automàticament (en anglès, MTPE), i un mètode d'avaluació semiautomàtica.El motor detraducció automàtica neuronal utilitzat ha estat Google Translate, en concret en el seu domini de notícies. Després de ser avaluat per sis traductors professionals es constata que el motor augmenta el rendiment en un 78% i la productivitat en un 37%. A més, el 40,249% dels resultats del motor es modifiquen amb un interval de 15 mesos, de manera que mostra un índex de millora del 11%.

Paraules clau

Traducción automàtica neuronal, TAN, avaluació de TA, TAPE, postedició de traducció automàtica, traducció espanyol-coreà

Text complet:

PDF (English)

Referències

Bahdanau, D.; Cho, K.; Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate, in: CoRR, Accepted for oral presentation at the International Conference on Learning Representations (ICLR) 2015. .

Bengio Y.; Ducharme, R.; Vincent, P. (2003). A neural probabilistic language model, in: Journal of Machine Learning Research, v. 3, pp. 1137-1155. .

British Neuroscience Association (2003). Science of the brain: an introduction for young students. . Last updated: 2003. Page consulted on date: 07.05.18.

Brown, P.; Cocke, J.; Della Pietra, S.; Della Pietra, V., Jelinek, F.; Mercer, R.; Roossin, P. (1988). A Statistical Approach to French/English Translation, in: Proceedings of the 2nd International Conference on Theoretical and Methodological Issues in Machine Translation of Natural Languages (12-14 June 1988). Center for Machine Translation, Carnegie Mellon University, Pittsburgh, Pennsylvania, the United States of America. .

https://doi.org/10.1007/978-94-009-3117-6_27

Chang, A. (2017). Analysis of the Current Development of Machine Translation and Interpretation in Korea: Focusing on Korean-Chinese Language Pairs, in: The Journal of Translation Studies v. 18, n. 2, pp. 171-206. .

Cho, K.; Merriënboer, B.; Gülcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, in: Proceedings of EMNLP 2014, Doha, Qatar, October: Association for Computational Linguistics. .

https://doi.org/10.3115/v1/D14-1179

Choi, H.; Lee, J. (2017). A study on the evaluation of Korean-English patent machine translation - Focusing on KIPRIS K2E-PAT translation, in: Interpretation & Translation, v. 19, n. 1: pp. 139-178. .

https://doi.org/10.20305/it201701139178

Devlin, J.; Zbib, R.; Huang, Z.; Lamar, T.; Schwartz, R.; Makhoul, J. (2014). Fast and Robust Neural Network Joint Models for Statistical Machine Translation, v. 1: pp. 1370-1380. .

Domingos, P. (2015). The Master Algorithm. Basic Books, 1st edition.

Görög, A. (2014). Quality evaluation today: the Dynamic Quality Framework, in: Proceedings of Translating and the Computer 36: ASLING: Proceedings. Geneva: Tradulex, pp. 155-164. .

Groves, D.; Schmidtke, D. (2009). Identification and analysis of post-editing patterns for MT. .

Kang, B.; Lee, J. (2018). The Operating Principles of Neural Machine Translation and the Accuracy of Translation - Focusing on the Chinese-Korean Translation, in: The Journal of Chinese Language and Literature, v. 73, pp. 253-295. .

https://doi.org/10.46612/kjcll.2018.09.73.253

Ki, Y. (2018). An Analysis of Errors by sentence pattern in translating Korean sentences into Chinese by Machine Translation - focus on Naver Papago machine translation and Google machine translation, in: Chinese Studies, v. 74, pp. 3-32. .

https://doi.org/10.18077/chss.2018.74..001

Kim, A. (2015). Reordering of SOV-SVO Pairs in Statistical Machine Translation: In Relation to the Korean Language. Masters thesis at Universitat Pompeu Fabra.

Kim, A. (2019). Neural Machine Translation Evaluation & Error Analysis in a Spanish-Korean Translation. Doctoral thesis at Universitat Pompeu Fabra. Retrieved from https://repositori.upf.edu/handle/10230/42853.

Kim, S.; Lee, H. (2017). A Study on Machine Translation Outputs - Korean to English Translation of Embedded Sentences, in: The Journal of Mirae English Language and Literature, v. 22, n. 4, pp. 123-147. .

Koponen, M. (2012). Comparing human perceptions of post-editing effort with post- editing operations, in: Proceedings of the 7th Workshop on Statistical Machine Translation. Montreal, Canadá: Association for Computational Linguistics, pp. 181- 190. .

Korean Citation Index . Page consulted on date: 15.03.20.

Plitt, M.; Masselot, M. (2010). A Productivity Test of Statistical Machine Translation, in: The Prague Bulletin of Mathematical Linguistics, v. 93, pp. 7-16. .

Russell, J.; Norvig, P.; Canny, F.; Malik, M.; Edwards, D. (1995). Artificial Intelligence: a Modern Approach. Vol 2, Englewood Cliffs: Prentice Hall.

Schwenk, H. (2007). Continuous space language models, in: Computer Speech & Language, v. 21, pp. 492-518. .

https://doi.org/10.1016/j.csl.2006.09.003

Skadina, I.; Pinnis, M. (2017). NMT or SMT: Case Study of a Narrow-domain English-Latvian Post-editing Project, in: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Vol 1: Long Papers), pp. 373 - 383. .

Snover, M.; Dorr, B.; Schwartz, R.; Micciulla, L.; Makhoul, J. (2006). A study of translation edit rate with targeted human annotation, in: Proceedings of the 7th Biennial Conference of the Association for Machine Translation in the Americas (AMTA-2006). Cambridge, Massachusetts: Association for Machine Translation in the Americas.

Snover, M.; Madnan,i N.; Dorr, J.; Schwartz, R. (2009). Fluency, adequacy, or HTER? Exploring different human judgments with a tunable MT metric, in: Proceedings of the Fourth Workshop on Statistical Machine Translation, pp. 259-268. .

https://doi.org/10.3115/1626431.1626480

Song, H.; Park, S. (2020). Korean Part-of-speech Tagging Based on Morpheme Generation, in: ACM Trans. Asian Low-Resour. Lang. Inf. Process. n. 19, v. 3, Article 41 (January 2020), 10 pages. .

Sutskever, I.; Vinyals, O.; Le, Q. (2014). Sequence to Sequence Learning with Neural Networks, in: Proceedings of the Neural Information Processing Systems, Vol 2 (NIPS'14), MIT Press, Cambridge, MA, USA, pp. 3104 - 3112. .

Tatsumi, M. (2009). Correlation Between Automatic Evaluation Metric Scores, Post-Editing Speed, and Some Other Factors. .

TAUS (2010). Machine Translation Post-editing Guidelines. . Last updated: 2010. Page consulted on date: 04.08.17.

Weaver, W. (1949). "Translation", Reprinted in Locke, W.; Booth, A. (eds.) Machine Translation of Languages: Fourteen Essays, Cambridge, Massachusetts: Technology Press of the Massachusetts Institute of Technology: pp. 15-33.

Wu, Y.; Schuster, M.; Chen, Z.; Le, Q.; Norouzi, M.; Macherey, W.; Krikun, M.; Cao, Y.; Gao, Q.; Macherey, K.; Klingner, J.; Shah, A.; Johnson, M.; Liu, X.; Kaiser, L.; Gouws, S.; Kato, Y.; Kudo, T.; Kazawa, H.; Stevens, K.; Kurian, G.; Patil, N.; Wang, W.; Young, C.; Smith, J.; Riesa, J.; Rudnick, A.; Vinyals, O.; Corrado, G.; Hughes, M.; Dean, J. (2016). Google's neural machine translation system: Bridging the gap between human and machine translation, in: CoRR. .

Zhechev, V. (2014). "Analysing the Post-Editing of Machine Translation at Autodesk," in O'Brien, S.; Balling, L.; Carl, M.; Simard, M.; Specia, L. (eds.) (2014), Post-editing of Machine Translation: Processes and Applications: Cambridge Scholars Publishing.

Mètriques darticles

Carregant mètriques ...

Metrics powered by PLOS ALM
Copyright (c) 2020 Ahrii Kim, Carme Colominas Ventura
Llicència de Creative Commons
Aquesta obra està subjecta a una llicència de Reconeixement 4.0 Internacional de Creative Commons