Mesura de l'esforç cognitiu en la post-edició: Un estudio de seguiment ocular que compara traductors professionals i estudiants
Resum
Aquest estudi de seguiment ocular compara l’esforç cognitiu de post-edició de 25 professionals i 27 estudiants en post-editar la traducció automàtica neuronal (NMT, per les sigles en anglès) i la traducció automàtica estadística (SMT, per les sigles en anglès) de l’anglès al castellà. Els resultats no mostren diferències significatives en el temps de post-edició ni en la durada de les fixacions entre els grups o els sistemes de TA, però revelen una reducció en la durada de les fixacions amb NMT per a tots dos grups.
Paraules clau
seguiment ocular, post-edició, traducció automàtica neuronal, TAN, esforç cognitiu, temps de post-edicióReferències
Alves, F., Koglin, A., Mesa-Lao, B., Martínez, M. G., de Lima Fonseca, N. B., de Melo Sá, A., Gonçalves, J. L., Szpak, K. S., Sekino, K., & Aquino, M. (2016). Analysing the impact of interactive machine translation on post-editing effort. In M. Carl, S. Bangalore, & M. Schaeffer (Eds.), New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB (pp. 77–94). Springer Cham. <https://doi.org/10.1007/978-3-319-20358-4_4>. [Accessed: 20241212]
Aziz, W., Castilho, S., & Specia, L. (2012). PET: A tool for post-editing and assessing machine translation. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), 3982–3987. <http://www.lrec-conf.org/proceedings/lrec2012/pdf/985_Paper.pdf>. [Accessed: 20241212]
Balling, L. W. (2008). A brief introduction to regression designs and mixed-effects modelling by a recent convert. Copenhagen Studies in Language, 36, 175–192. <https://doi.org/10.1075/ts.21013.sil>. [Accessed: 20241212]
Balling, L. W., & Hvelplund, K. T. (2015). Design and statistics in quantitative translation (Process) research. Translation Spaces, 4(1), 170–187. <https://doi.org/10.1075/ts.4.1.08bal>. [Accessed: 20241212]
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1). <https://doi.org/10.18637/jss.v067.i01>. [Accessed: 20241212]
Bundgaard, K. (2017). (Post-)Editing - A workplace study of translator-computer interaction at TextMinded Danmark A/S. [Doctoral Thesis, Aarhus University].
Carl, M., Gutermuth, S., & Hansen-Schirra, S. (2015). Post-editing machine translation: Efficiency, strategies, and revision processes in professional translation settings. In A. Ferreira & J. W. Schwieter (Eds.), Psycholinguistic and Cognitive Inquiries into Translation and Interpreting (pp. 145–174). John Benjamins Publishing Company. <https://doi.org/10.1075/btl.115.07car>. [Accessed: 20241212]
Castilho, S., Moorkens, J., Gaspari, F., Sennrich, R., Way, A., & Georgakopoulou, P. (2018). Evaluating MT for massive open online courses. Machine Translation, 32(3), 255–278. <https://doi.org/10.1007/s10590-018-9221-y>. [Accessed: 20241212]
Daems, J. (2016). A translation robot for each translator? A comparative study of manual translation and post-editing of machine translations: Process, quality and translator attitude. [Doctoral Thesis, Ghent University].
Daems, J., Vandepitte, S., Carl, M., & Jartsuiker, R. J. (2016). The effectiveness of consulting external resources during translation and postediting of general text types. In M. Carl, S. Bangalore, & M. Schaeffer (Eds.), New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB (pp. 111–133). Springer Cham. <https://doi.org/10.1007/978-3-319-20358-4_6>. [Accessed: 20241212]
Guerberof Arenas, A. (2008). Productivity and quality in the post-editing of outputs from translation memories and machine translation. Localisation Focus - The International Journal of Localisation, 7(1), 11–21.
Guerberof Arenas, A. (2014). The role of professional experience in post-editing from a quality and productivity perspective. In S. O’Brien (Ed.), Post-editing of Machine Translation: Processes and Applications (pp. 51–76). Cambridge Scholars Publishing.
Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y. J., Afify, M., & Awadalla, H. H. (2023). How good are GPT models at machine translation? A comprehensive evaluation. <http://arxiv.org/abs/2302.09210>. [Accessed: 20241212]
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures (1st ed.). Oxford University Press.
Hvelplund, K. T. (2011). Allocation of cognitive resources in translation: An eye-tracking and key-logging study. [Doctoral Thesis, Copenhagen Business School].
Hvelplund, K. T. (2014). Eye tracking and the translation process: Reflections on the analysis and interpretation of eye-tracking data. In R. Muñoz Martín (Ed.), MonTI. Monografías de Traducción e Interpretación (Special Issue, pp. 201–223). Publicaciones de la Universidad de Alicante. <https://doi.org/10.6035/monti.2014.ne1.6>. [Accessed: 20241212]
Hvelplund, K. T. (2016). Cognitive efficiency in translation. In R. Muñoz Martín (Ed.), Reembedding Translation Process Research (pp. 149–170). John Benjamins Publishing Company. <https://doi.org/10.1075/btl.128.08hve>. [Accessed: 20241212]
Hvelplund, K. T. (2017). Translators’ use of digital resources during translation. Hermes (Denmark), 56, 71–87. <https://doi.org/10.7146/hjlcb.v0i56.97205>. [Accessed: 20241212]
ISO 18587. (2017). Translation services — Post-editing of machine translation output — Requirements. Geneva: International Organization for Standardization. Retrieved from <https://www.iso.org/obp/ui/en/#iso:std:iso:18587:ed-1:v1:en>. [Accessed: 20241212]
Jia, Y., Carl, M., & Wang, X. (2019). Post-editing neural machine translation versus phrase-based machine translation for English–Chinese. Machine Translation, 33(1–2), 9–29. <https://doi.org/10.1007/s10590-019-09229-6>. [Accessed: 20241212]
Jiménez-Crespo, M. A., & Casillas, J. V. (2021). Literal is not always easier: Literal and default translation, cognitive effort, and comparable corpora. Translation, Cognition and Behavior, 4(1), 100–125. <https://doi.org/10.1075/tcb.00048.jim>. [Accessed: 20241212]
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354. <https://doi.org/10.1037/0033-295X.87.4.329>. [Accessed: 20241212]
Koglin, A. (2015). An empirical investigation of cognitive effort required to post-edit machine translated metaphors compared to the translation of metaphors. Translation & Interpreting, 7(1).
Koponen, M. (2016). Is machine translation post-editing worth the effort? A survey of research into post-editing and effort. Journal of Specialised Translation, 25, 131–148.
Koponen, M., Aziz, W., Ramos, L., & Specia, L. (2012). Post-editing time as a measure of cognitive effort. AMTA Workshop on Postediting Technology and Practice, 47(3), 11–20.
Koponen, M., & Salmi, L. (2017). Post-editing quality: Analysing the correctness and necessity of post-editor corrections. Linguistica Antverpiensia, New Series: Themes in Translation Studies, 16(16), 137–148. <https://doi.org/10.52034/lanstts.v16i0.439>. [Accessed: 20241212]
Koponen, M., Salmi, L., & Nikulin, M. (2019). A product and process analysis of post-editor corrections on neural, statistical and rule-based machine translation output. Machine Translation. <https://doi.org/10.1007/s10590-019-09228-7>. [Accessed: 20241212]
Krings, H. P. (2001). Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes (G. S. Koby, Ed.). The Kent State University Press.
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest Package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13). <https://doi.org/10.18637/jss.v082.i13>. [Accessed: 20241212]
Lacruz, I., Denkowski, M., & Lavie, A. (2014). Cognitive demand and cognitive effort in post-editing. In S. O’Brien, M. Shimard & L. Specia (Eds.), Proceedings of the 11th Conference of the Association for Machine Translation in the Americas (pp. 73–84). Association for Machine Translation in the Americas.
Läubli, S., Amrhein, C., Düggelin, P., Gonzalez, B., Zwahlen, A., & Volk, M. (2019). Post-editing productivity with neural machine translation: An empirical assessment of speed and quality in the banking and finance domain. In Proceedings of Machine Translation Summit XVII: Research Track (pp. 267–272). <http://arxiv.org/abs/1906.01685>. [Accessed: 20241212]
Lehr, C., & Hvelplund, K. T. (2020). Emotional experts: Influences of emotion on the allocation of cognitive resources during translation. In R. Muñoz Martín & S. L. Halverson (Eds.), Multilingual Mediated Communication and Cognition (pp. 44–68). Routledge. <https://doi.org/10.4324/9780429323867-3>. [Accessed: 20241212]
Mann, H. B., & Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50–60. <https://doi.org/10.1214/aoms/1177730491>. [Accessed: 20241212]
Massardo, I., van der Meer, J., O’Brien, S., Hollowood, F., Aranberri, N., & Drescher, K. (2016). MT postediting guidelines. TAUS Signature Editions.
Mellinger, C., & Hanson, T. (2017). Quantitative Research Methods in Translation and Interpreting Studies. Routledge. <https://doi.org/10.4324/9781315647845>. [Accessed: 20241212]
Mellinger, C. D., & Hanson, T. A. (2018). Order effects in the translation process. Translation, Cognition & Behavior, 1(1), 1–20. <https://doi.org/10.1075/tcb.00001.mel>. [Accessed: 20241212]
Moorkens, J. (2018). Eye tracking as a measure of cognitive effort for post-editing of machine translation. In C. Walker & F. M. Federici (Eds.), Eye Tracking and Multidisciplinary Studies on Translation (pp. 55–70). John Benjamins Publishing Company. <https://doi.org/10.1075/btl.143>. [Accessed: 20241212]
Moorkens, J., O'Brien, S., da Silva, I., Fonseca, N., & Alves, F. (2015). Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, 29, 267–284. <https://doi.org/10.1007/s10590-015-9175-2>. [Accessed: 20241212]
Muñoz Martín, R. (2014). Situating translation expertise: A review with a sketch of a construct. In J. W. Schwieter & A. Ferreira (Eds.), The Development of Translation Competence: Theories and Methodologies from Psycholinguistics and Cognitive Science (pp. 2–56). Cambridge Scholars Publishing.
Nitzke, J. (2019). Problem-solving activities in post-editing and translation from scratch: A multi-method study. In New Empirical Perspectives on Translation and Interpreting (Translatio). Language Science Press. <https://doi.org/10.5281/zenodo.2546446-5>. [Accessed: 20241212]
Nunes Vieira, L. (2015). Cognitive effort in post-editing of machine translation: Evidence from eye movements, subjective ratings, and think-aloud protocols. [Doctoral Thesis, Newcastle University].
Nunes Vieira, L. (2018). Automation anxiety and translators. Translation Studies, 13, 1–21. <https://doi.org/10.1080/14781700.2018.1543613>. [Accessed: 20241212]
O’Brien, S. (2007). Eye-tracking and translation memory matches. Perspectives: Studies in Translatology, 14(3), 185–205. <https://doi.org/10.1080/09076760708669037>. [Accessed: 20241212]
O’Brien, S. (2009). Eye tracking in translation-process research: methodological challenges and solutions. En I. M. Mees, S. Göpferich y F. Alves (Eds.), Methodology, Technology and Innovation in Translation Process Research. A Tribute to Arnt Lykke Jakobsen (Copenhagen). Samfundslitteratur Press. <https://doi.org/10.1075/target.24.1.13tir>. [Accessed: 20241212]
O’Brien, S. (2011). Towards predicting post-editing productivity. Machine Translation, 25, 197–215. <https://doi.org/10.1007/s10590-011-9096-7>. [Accessed: 20241212]
O’Brien, S. (2017). Machine translation and cognition. In J. W. Schwieter & A. Ferreira (Eds.), The Handbook of Translation and Cognition (1st ed., pp. 311–331). <https://doi.org/10.1002/9781119241485.ch17>. [Accessed: 20241212]
Olalla-Soler, C. (2018). Using electronic information resources to solve cultural translation problems: Differences between students and professional translators. Journal of Documentation, 74(6), 1293–1317. <https://doi.org/10.1108/JD-02-2018-0033>. [Accessed: 20241212]
Popović, M., Arčan, M., & Lommel, A. (2016). Potential and limits of using post-edits as reference translations for MT evaluation. Proceedings of the 19th Annual Conference of the European Association for Machine Translation, EAMT 2016, 218–229.
R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. <https://www.R-project.org/>. [Accessed: 20241212]
Ragni, V., & Nunes Vieira, L. (2022). What has changed with neural machine translation? A critical review of human factors. Perspectives, 30(1), 137–158. <https://doi.org/10.1080/0907676X.2021.1889005>. [Accessed: 20241212]
Sakamoto, A. (2019). Why do many translators resist post-editing? A sociological analysis using Bourdieu’s concepts. The Journal of Specialised Translation, 31, 201–216.
Saldanha, G., & O’Brien, S. (2014). Research methodologies in translation studies. In Research Methodologies in Translation Studies. <https://doi.org/10.4324/9781315760100>. [Accessed: 20241212]
Sánchez-Gijón, P., Moorkens, J., & Way, A. (2019). Post-editing neural machine translation versus translation memory segments. Machine Translation, 33(1–2), 31–59. <https://doi.org/10.1007/s10590-019-09232-x>. [Accessed: 20241212]
Schmaltz, M. S., da Silva, I. A. L., Pagano, A. S., Alves, F., Leal, A. L., Wong, D. F., Chao, L. S., & Quaresma, P. (2016). Cohesive relations in text comprehension and production: An exploratory study comparing translation and post-editing. In M. Carl, S. Bangalore, & M. Schaeffer (Eds.), New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB (pp. 239–263). Springer Cham. <https://doi.org/10.1007/978-3-319-20358-4_11>. [Accessed: 20241212]
Sjørup, A. C. (2013). Cognitive effort in metaphor translation: An eye-tracking and key-logging study. Copenhagen Business School.
Stasimioti, M., & Sosoni, V. (2019). MT output and post-editing effort: Insights from a comparative analysis of SMT and NMT output for the English to Greek language pair and implications for the training of post-editors. In C. Szabó & R. Besznyák (Eds.), Teaching Specialised Translation and Interpreting in a Digital Age - Fit-For-Market Technologies, Schemes and Initiatives (pp. 151–175). Vernon Press.
Stasimioti, M., & Sosoni, V. (2021). Investigating post-editing: A mixed-methods study with experienced and novice translators in the English-Greek language pair. In Tra&Co Group (Ed.), Translation, interpreting, cognition: The way out of the box (pp. 79–104). Language Science Press. <https://doi.org/10.5281/zenodo.4545037>. [Accessed: 20241212]
Stasimioti, M., Sosoni, V., Mouratidis, D., & Kermanidis, K. (2020). Machine translation quality: A comparative evaluation of SMT, NMT and tailored-NMT outputs. Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, EAMT 2020, 441–450.
Tatsumi, M. (2009). Correlation between automatic evaluation metric scores, post-editing speed, and some other factors. Proceedings of MT Summit XII, 2001, 332–339. <http://www.mt-archive.info/MTS-2009-Tatsumi.pdf>. [Accessed: 20241212]
Tatsumi, M., & Roturier, J. (2010). Source text characteristics and technical and temporal post-editing effort: What is their relationship? Second Joint EM+/CNGL Workshop “Bringing MT to the User: Research on Integrating MT in the Translation Industry” JEC 2010, 43–51. Retrieved from http://www.mt-archive.info/JEC-2010-Tatsumi.pdf
Teixeira, C. S. C. (2014). The impact of metadata on translator performance: How translators work with translation memories and machine translation. [Doctoral Thesis, Universitat Rovira i Virgili]. <https://doi.org/10.13140/RG.2.1.2190.2887>. [Accessed: 20241212]
Teixeira, C. S. C., & O’Brien, S. (2017). Investigating the cognitive ergonomic aspects of translation tools in a workplace setting. Translation Spaces, 6(1), 79–103. <https://doi.org/10.1075/ts.6.1.05tei>. [Accessed: 20241212]
Temnikova, I. (2010). Cognitive evaluation approach for a controlled language post-editing experiment. Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010, January 2010, 3485–3490.
Toral, A., & Sánchez-Cartagena, V. M. (2017). A multifaceted evaluation of neural versus phrase-based machine translation for 9 language directions. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 1063–1073. <https://doi.org/10.48550/arXiv.1701.02901>. [Accessed: 20241212]
Toral, A., Wieling, M., & Way, A. (2018). Post-editing effort of a novel with statistical and neural machine translation. Frontiers in Digital Humanities, 5. <https://doi.org/10.3389/fdigh.2018.00009>. [Accessed: 20241212]
Wang, L., Lyu, C., Ji, T., Zhang, Z., Yu, D., Shi, S., & Tu, Z. (2023). Document-level machine translation with large language models. ArXiv, abs/2304.02210. <https://api.semanticscholar.org/CorpusID:257952312>. [Accessed: 20241212]
Witczak, O. (2021). Information searching in the post-editing and translation process [Doctoral Thesis, Adam Mickiewicz University]. <https://doi.org/10.13140/RG.2.2.15691.87847>. [Accessed: 20241212]
Publicades
Com citar
Descàrregues
Drets d'autor (c) 2024 ANA MARÍA ROJO LÓPEZ, María Inmaculada Vicente López, Kristian Tangsgaard Hvelplund
Aquesta obra està sota una llicència internacional Creative Commons Reconeixement 4.0.
