Writing on steroids? Accuracy of automatic corrective feedback in L2 competence development

Robert Martínez-Carrasco; Alicia Chabert

doi:10.5565/rev/jtl3.1142

¿Escriben dopados? Precisión de la retroalimentación correctiva automática en el desarrollo de la L2

Autores/as

Robert Martínez-Carrasco Universitat Jaume I https://orcid.org/0000-0002-2148-8637
Alicia Chabert Universitat Jaume I https://orcid.org/0000-0003-4599-8524

PDF (EN)

Resumen

El artículo profundiza en la retroalimentación correctiva automática como herramienta para el desarrollo de la L2. Si bien la retroalimentación de la producción escrita se considera instrumental en el aprendizaje de una lengua extranjera, proporcionar retroalimentación exhaustiva puede no ser siempre posible, de ahí que los correctores puedan convertirse, al promover el aprendizaje autodirigido, en una herramienta útil que apuntale el aprendizaje del alumnado. Partiendo de esa base, en el estudio que presentamos se utilizó la herramienta Grammarly para evaluar la producción escrita en inglés de estudiantes hispanohablantes (n=91; C1 MCER) y analizar la retroalimentación proporcionada, a saber, los tipos y frecuencia de errores y el nivel de precisión de la herramienta. Los resultados sugieren que, aunque los correctores gramaticales pueden ser útiles como herramienta complementaria, suelen estar desarrollados pensando en hablantes nativos, por lo que la retroalimentación no siempre satisface las necesidades de los estudiantes.

Palabras clave

Grammarly, Evaluación automática de la producción escrita, Inglés como lengua extranjera, Retroalimentación correctiva, Evaluación de la competencia escrita

Citas

Alakrash, H. M., & Razak, N. A. (2021). Technology-based language learning: Investigation of digital technology and digital literacy. Sustainability (Switzerland), 13(21). https://doi.org/10.3390/su132112304

Bachman, L. F. & Palmer, A. S. (2010). Language assessment in practice. Oxford University Press.

Bai, L., & Hu, G. (2017). In the face of fallible AWE feedback: how do students respond? Ed-ucational Psychology, 37(1), 67-81. https://doi.org/10.1080/01443410.2016.1223275

Bailey, D., & Lee, A. R. (2020). An Exploratory Study of Grammarly in the Language Learn-ing Context: An Analysis of Test-Based, Textbook-Based and Facebook Corpora. TESOL International Journal, 15(2), 4-27.

Bitchener, J., & Ferris, D. R. (2012). Written Corrective Feedback in Second Language Acqui-sition and Writing. Routledge.

Canale, M. (1983). From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and Communication (pp. 2-27). Routledge.

Canale, M. & Swain, M. (1980). Theoretical Bases of Communicative Approaches to Second Language Teaching and Testing. Applied Linguistics, 1(1), 1-47.

Cavaleri, M., & Dianati, S. (2016). You want me to check your grammar again? The useful-ness of an online grammar checker as perceived by students. Journal of Academic Language & Learning, 10(1), 223.

Celce-Murcia, M. (2007). Rethinking the role of communicative competence in language teaching. In E. Alcón Soler & M. P. Safont Jordà (Eds.), Intercultural language use and language learning (pp. 41-57). Springer.

Chabert, A. & Agost, R. (2020). Communicative language teaching: Is there a place for L1 in L2 learning? European Journal of Language Policy, 12(1), 55-83. https://doi.org/10.3828/ejlp.2020.4

Cotos, E. (2014). Genre-Based Automated Writing Evaluation for L2 Research Writing: From Design to Evaluation and Enhancement. Palgrave Macmillan UK. https://doi.org/10.1057/9781137333377_1

Dizon, G., & Gayed, J. M. (2021). Examining the Impact of Grammarly on The Quality of Mobile L2 Writing. JALT CALL Journal, 17(2), 74-92.

https://doi.org/10.29140/JALTCALL.V17N2.336

Dolores, P., Keselman, A., & Monopoli, M. (2003). The academic writing of community col-lege remedial students: Text and learner variables. Higher Education, 45, 19-42.

Dong, Y., & Shi, L. (2021). Using Grammarly to support students’ source-based writing prac-tices. Assessing Writing, 50(September), 100564. https://doi.org/10.1016/j.asw.2021.100564

Elliot, N., & Klobucar, A. (2013). Automated essay evaluation and the teaching of writing. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Evaluation: Cur-rent Applications and New Directions (pp. 16-35). Routledge.

Ferster, B., Hammond, T. C., Curby Alexander, R., & Lyman, H. (2012). Automated forma-tive assessment as a tool to scaffold student documentary writing. Journal of Interac-tive Learning Research, 23(1), 81-99.

Gain, A., Rao, M., & Bhat, K. S. (2019). Usage of Grammarly - online grammar and spelling checker tool at the health sciences library, Manipal Academy of Higher Education, Manipal: A Study. Library Philosophy and Practice, 2019.

Ghufron, M. A., & Rosyida, F. (2018). The Role of Grammarly in Assessing English as a For-eign Language (EFL) Writing. Lingua Cultura, 12(4), 395. https://doi.org/10.21512/lc.v12i4.4582

Graham, S., MacArthur, C. A., & Fitzgerald, J. (2018). Best practices in writing instruction (Third). Guilford Press.

Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. Journal of Technology, Learning, and Assessment, 8(6), 1-43.

Heil, C. R., Wu, J. S., Lee, J. J., & Schmidt, T. (2016). A review of mobile language learning applications: Trends, challenges, and opportunities. The EuroCALL Review, 24(2),

–50. https://doi.org/10.4995/eurocall.2016.6402

Hoover, B., Lytvyn, M., & Shevchenko, O. (2015). Systems and Methods for Advanced Grammar Checking (Patent No. US 9002700 B2). U.S. Patent and Trademark Office.

Huang, H. W., Li, Z., & Taylor, L. (2020). The Effectiveness of Using Grammarly to Improve Students’ Writing Skills. In ACM International Conference Proceeding Series, 2020 (pp. 122-127). https://doi.org/10.1145/3402569.3402594

John, P., & Woll, N. (2020). Using grammar checkers in an ESL context: An investigation of automatic corrective feedback. CALICO Journal, 37(2), 169-192. https://doi.org/10.1558/cj.36523

Khoshnevisan, B. (2019). The affordances and constraints of automatic writing evaluation (AWE) tools: A case for Grammarly. ARTESOL EFL Journal, 2(2), 12-25.

Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44(September 2019), 100450. https://doi.org/10.1016/j.asw.2020.100450

Krashen, S. D. (1982). Principles and Practice in Second Language Acquisition. Pergamon Press Inc. http://www.sdkrashen.com/content/books/principles_and_practice.pdf

Lastres-López, C., & Manalastas, G. (2017). Errors in L1 and L2 University Students’ Writing in English: Grammar, Spelling and Punctuation. RAEL: Revista Electrónica de Lingüística Aplicada, 16(2), 118-135

Lailika, H. I. (2019). Students’ Peceptions of The Use of Grammarly as an Online Grammar Checker in Thesis Writing. http://digilib.uinsby.ac.id/34607/

Leeman, J. (2007). Feedback in L2 learning: Responding to errors during practice. In R. DeKeyser (Ed.), Practice in a Second Language: Perspectives from Linguistics and Psychology (pp. 111-138). Cambridge University Press. https://doi.org/10.1017/cbo9780511667275.007

Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evalua-tion (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1-18. https://doi.org/10.1016/j.jslw.2014.10.004

Nova, M. (2018). Utilizing Grammarly in Evaluating Academic Writing: a Narrative Research on Efl Students’ Experience. Premise: Journal of English Education, 7(1), 80-97. https://doi.org/10.24127/pj.v7i1.1300

Nunan, D. (1999). Second language teaching & learning. Heinle & Heinle Publishers.

Page, E. B. (2003). Project Essay Grade: PEG. In M. Shermis & B. JC (Eds.), Automated essay scoring: A cross-disciplinary perspective. (pp. 43-54). Lawrence Erlbaum Associates Publishers.

Park, J. (2019). An AI-based English Grammar Checker vs. Human Raters in Evaluating EFL Learners’ Writing. Multimedia-Assisted Language Learning, 22(1), 112-131.

http://journal.kamall.or.kr/wp-content/uploads/2019/3/Park_22_1_04.pdfhttp://www.kamall.or.kr

Pratama, Y. D. (2020). The Investigation of Using Grammarly As Online Grammar Checker in the Process of Writing. English Ideas: Journal of English Language Education, 1(1), 46-54.

Ranalli, J. (2018). Automated written corrective feedback: how well can students make use of it? Computer Assisted Language Learning, 31(7), 653-674. https://doi.org/10.1080/09588221.2018.1428994

Ranalli, J., Link, S., & Chukharev-Hudilainen, E. (2017). Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and use-fulness of feedback as part of argument-based validation. Educational Psychology, 37(1), 8-25. https://doi.org/10.1080/01443410.2015.1136407

Richards, J. C., & Rodgers, T. (1999). Approaches and methods in language teaching. Cam-bridge University Press.

Sahu, S. (2020). Evaluating performance of different grammar checking tools. International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 2227-2233. https://doi.org/10.30534/ijatcse/2020/201922020

Savignon, S. J. (2002). Communicative Language Teaching: Linguistic Theory and Classroom Practice. In S. J. Savignon (Ed.), Interpreting Communicative Language Teaching: Contexts and Concerns in Teacher Education (pp.1-28). Yale University Press.

Schindler, L. A., Burkholder, G. J., Morad, O. A., & Marsh, C. (2017). Computer-based tech-nology and student engagement: a critical review of the literature. International Jour-nal of Educational Technology in Higher Education, 14(1).

https://doi.org/10.1186/s41239-017-0063-0

Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quali-ty of writing. Assessing Writing, 19, 51-65. https://doi.org/10.1016/j.asw.2013.11.007

Weigle, S. C. (2013). English as a Second Language Writing and Automated Essay Evaluation. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Evaluation (pp. 36-54). Routledge.

Wilson, J. (2017). Associated effects of automated essay evaluation software on growth in writing quality for students with and without disabilities. Reading and Writing, 30(4), 691-718. https://doi.org/10.1007/s11145-016-9695-z

Woodworth, J., & Barkaoui, K. (2020). Perspectives on Using Automated Writing Evaluation Systems to Provide Written Corrective Feedback in the ESL Classroom. TESL Canada Journal, 37(2), 234-247. https://doi.org/10.18806/tesl.v37i2.1340

Zinkevich, N. A., & Ledeneva, T. V. (2021). Using Grammarly to Enhance Students’ Academ-ic Writing Skills. Professional Discourse & Communication, 3(4), 51-63. https://doi.org/10.24833/2687-0126-2021-3-4-51-63

Biografía del autor/a

Robert Martínez-Carrasco, Universitat Jaume I

Holds a PhD in Applied Languages, Literature and Translation from Jaume I University (Spain). His research interests include translation education, critical pedagogy and the construction of gender, sex and identity. At Universitat Jaume I, he teaches courses in Catalan-English translation and English linguistics.

Alicia Chabert, Universitat Jaume I

Holds a PhD in Applied Languages, Literature and Translation from Jaume I University (Spain). Her research interests include the role of the mother tongue in foreign language learning and English as a Lingua Francia. At Universitat Jaume I, she teaches Spanish-English translation.