Writing on steroids? Accuracy of automatic corrective feedback in L2 competence development

Robert Martínez-Carrasco; Alicia Chabert

doi:10.5565/rev/jtl3.1142

Escriuen dopats? Precisió de la retroalimentació correctiva automàtica en el desenvolupament de la L2

Autors/ores

Robert Martínez-Carrasco Universitat Jaume I https://orcid.org/0000-0002-2148-8637
Alicia Chabert Universitat Jaume I https://orcid.org/0000-0003-4599-8524

PDF (EN)

Resum

L'article aprofundeix en la retroalimentació correctiva automàtica com a eina per al desenvolupament de la L2. Si bé la retroalimentació de la producció escrita es considera instrumental per a l'aprenentatge de la llengua estrangera, proporcionar-la pot no ser sempre possible, de manera que els correctors, en promoure l'aprenentatge autodirigit, poden convertir-se en una eina útil que apuntale l'aprenentatge de l'alumnat. Així doncs, l'estudi que presentem fa servir l'eina Grammarly per tal d’avaluar la producció escrita en anglés d'un grup d’estudiants hispanoparlants (n=91; C1 MECR) i analitzar-ne la retroalimentació, a saber, els tipus i freqüència d'errors i el nivell de precisió de l'eina. Els resultats suggereixen que, encara que els correctors gramaticals poden ser útils com a eina complementària, solen estar desenvolupats per a parlants natius, per la qual cosa la retroalimentació no sempre satisfà les necessitats dels estudiants.

Paraules clau

Grammarly, Avaluació automàtica de la producció escrita, Anglès com a llengua estrangera, Retroalimentació correctiva, Avaluació de la competència escrita

Referències

Alakrash, H. M., & Razak, N. A. (2021). Technology-based language learning: Investigation of digital technology and digital literacy. Sustainability (Switzerland), 13(21). https://doi.org/10.3390/su132112304

Bachman, L. F. & Palmer, A. S. (2010). Language assessment in practice. Oxford University Press.

Bai, L., & Hu, G. (2017). In the face of fallible AWE feedback: how do students respond? Ed-ucational Psychology, 37(1), 67-81. https://doi.org/10.1080/01443410.2016.1223275

Bailey, D., & Lee, A. R. (2020). An Exploratory Study of Grammarly in the Language Learn-ing Context: An Analysis of Test-Based, Textbook-Based and Facebook Corpora. TESOL International Journal, 15(2), 4-27.

Bitchener, J., & Ferris, D. R. (2012). Written Corrective Feedback in Second Language Acqui-sition and Writing. Routledge.

Canale, M. (1983). From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and Communication (pp. 2-27). Routledge.

Canale, M. & Swain, M. (1980). Theoretical Bases of Communicative Approaches to Second Language Teaching and Testing. Applied Linguistics, 1(1), 1-47.

Cavaleri, M., & Dianati, S. (2016). You want me to check your grammar again? The useful-ness of an online grammar checker as perceived by students. Journal of Academic Language & Learning, 10(1), 223.

Celce-Murcia, M. (2007). Rethinking the role of communicative competence in language teaching. In E. Alcón Soler & M. P. Safont Jordà (Eds.), Intercultural language use and language learning (pp. 41-57). Springer.

Chabert, A. & Agost, R. (2020). Communicative language teaching: Is there a place for L1 in L2 learning? European Journal of Language Policy, 12(1), 55-83. https://doi.org/10.3828/ejlp.2020.4

Cotos, E. (2014). Genre-Based Automated Writing Evaluation for L2 Research Writing: From Design to Evaluation and Enhancement. Palgrave Macmillan UK. https://doi.org/10.1057/9781137333377_1

Dizon, G., & Gayed, J. M. (2021). Examining the Impact of Grammarly on The Quality of Mobile L2 Writing. JALT CALL Journal, 17(2), 74-92.

https://doi.org/10.29140/JALTCALL.V17N2.336

Dolores, P., Keselman, A., & Monopoli, M. (2003). The academic writing of community col-lege remedial students: Text and learner variables. Higher Education, 45, 19-42.

Dong, Y., & Shi, L. (2021). Using Grammarly to support students’ source-based writing prac-tices. Assessing Writing, 50(September), 100564. https://doi.org/10.1016/j.asw.2021.100564

Elliot, N., & Klobucar, A. (2013). Automated essay evaluation and the teaching of writing. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Evaluation: Cur-rent Applications and New Directions (pp. 16-35). Routledge.

Ferster, B., Hammond, T. C., Curby Alexander, R., & Lyman, H. (2012). Automated forma-tive assessment as a tool to scaffold student documentary writing. Journal of Interac-tive Learning Research, 23(1), 81-99.

Gain, A., Rao, M., & Bhat, K. S. (2019). Usage of Grammarly - online grammar and spelling checker tool at the health sciences library, Manipal Academy of Higher Education, Manipal: A Study. Library Philosophy and Practice, 2019.

Ghufron, M. A., & Rosyida, F. (2018). The Role of Grammarly in Assessing English as a For-eign Language (EFL) Writing. Lingua Cultura, 12(4), 395. https://doi.org/10.21512/lc.v12i4.4582

Graham, S., MacArthur, C. A., & Fitzgerald, J. (2018). Best practices in writing instruction (Third). Guilford Press.

Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. Journal of Technology, Learning, and Assessment, 8(6), 1-43.

Heil, C. R., Wu, J. S., Lee, J. J., & Schmidt, T. (2016). A review of mobile language learning applications: Trends, challenges, and opportunities. The EuroCALL Review, 24(2),

–50. https://doi.org/10.4995/eurocall.2016.6402

Hoover, B., Lytvyn, M., & Shevchenko, O. (2015). Systems and Methods for Advanced Grammar Checking (Patent No. US 9002700 B2). U.S. Patent and Trademark Office.

Huang, H. W., Li, Z., & Taylor, L. (2020). The Effectiveness of Using Grammarly to Improve Students’ Writing Skills. In ACM International Conference Proceeding Series, 2020 (pp. 122-127). https://doi.org/10.1145/3402569.3402594

John, P., & Woll, N. (2020). Using grammar checkers in an ESL context: An investigation of automatic corrective feedback. CALICO Journal, 37(2), 169-192. https://doi.org/10.1558/cj.36523

Khoshnevisan, B. (2019). The affordances and constraints of automatic writing evaluation (AWE) tools: A case for Grammarly. ARTESOL EFL Journal, 2(2), 12-25.

Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44(September 2019), 100450. https://doi.org/10.1016/j.asw.2020.100450

Krashen, S. D. (1982). Principles and Practice in Second Language Acquisition. Pergamon Press Inc. http://www.sdkrashen.com/content/books/principles_and_practice.pdf

Lastres-López, C., & Manalastas, G. (2017). Errors in L1 and L2 University Students’ Writing in English: Grammar, Spelling and Punctuation. RAEL: Revista Electrónica de Lingüística Aplicada, 16(2), 118-135

Lailika, H. I. (2019). Students’ Peceptions of The Use of Grammarly as an Online Grammar Checker in Thesis Writing. http://digilib.uinsby.ac.id/34607/

Leeman, J. (2007). Feedback in L2 learning: Responding to errors during practice. In R. DeKeyser (Ed.), Practice in a Second Language: Perspectives from Linguistics and Psychology (pp. 111-138). Cambridge University Press. https://doi.org/10.1017/cbo9780511667275.007

Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evalua-tion (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1-18. https://doi.org/10.1016/j.jslw.2014.10.004

Nova, M. (2018). Utilizing Grammarly in Evaluating Academic Writing: a Narrative Research on Efl Students’ Experience. Premise: Journal of English Education, 7(1), 80-97. https://doi.org/10.24127/pj.v7i1.1300

Nunan, D. (1999). Second language teaching & learning. Heinle & Heinle Publishers.

Page, E. B. (2003). Project Essay Grade: PEG. In M. Shermis & B. JC (Eds.), Automated essay scoring: A cross-disciplinary perspective. (pp. 43-54). Lawrence Erlbaum Associates Publishers.

Park, J. (2019). An AI-based English Grammar Checker vs. Human Raters in Evaluating EFL Learners’ Writing. Multimedia-Assisted Language Learning, 22(1), 112-131.

http://journal.kamall.or.kr/wp-content/uploads/2019/3/Park_22_1_04.pdfhttp://www.kamall.or.kr

Pratama, Y. D. (2020). The Investigation of Using Grammarly As Online Grammar Checker in the Process of Writing. English Ideas: Journal of English Language Education, 1(1), 46-54.

Ranalli, J. (2018). Automated written corrective feedback: how well can students make use of it? Computer Assisted Language Learning, 31(7), 653-674. https://doi.org/10.1080/09588221.2018.1428994

Ranalli, J., Link, S., & Chukharev-Hudilainen, E. (2017). Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and use-fulness of feedback as part of argument-based validation. Educational Psychology, 37(1), 8-25. https://doi.org/10.1080/01443410.2015.1136407

Richards, J. C., & Rodgers, T. (1999). Approaches and methods in language teaching. Cam-bridge University Press.

Sahu, S. (2020). Evaluating performance of different grammar checking tools. International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 2227-2233. https://doi.org/10.30534/ijatcse/2020/201922020

Savignon, S. J. (2002). Communicative Language Teaching: Linguistic Theory and Classroom Practice. In S. J. Savignon (Ed.), Interpreting Communicative Language Teaching: Contexts and Concerns in Teacher Education (pp.1-28). Yale University Press.

Schindler, L. A., Burkholder, G. J., Morad, O. A., & Marsh, C. (2017). Computer-based tech-nology and student engagement: a critical review of the literature. International Jour-nal of Educational Technology in Higher Education, 14(1).

https://doi.org/10.1186/s41239-017-0063-0

Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quali-ty of writing. Assessing Writing, 19, 51-65. https://doi.org/10.1016/j.asw.2013.11.007

Weigle, S. C. (2013). English as a Second Language Writing and Automated Essay Evaluation. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Evaluation (pp. 36-54). Routledge.

Wilson, J. (2017). Associated effects of automated essay evaluation software on growth in writing quality for students with and without disabilities. Reading and Writing, 30(4), 691-718. https://doi.org/10.1007/s11145-016-9695-z

Woodworth, J., & Barkaoui, K. (2020). Perspectives on Using Automated Writing Evaluation Systems to Provide Written Corrective Feedback in the ESL Classroom. TESL Canada Journal, 37(2), 234-247. https://doi.org/10.18806/tesl.v37i2.1340

Zinkevich, N. A., & Ledeneva, T. V. (2021). Using Grammarly to Enhance Students’ Academ-ic Writing Skills. Professional Discourse & Communication, 3(4), 51-63. https://doi.org/10.24833/2687-0126-2021-3-4-51-63

Biografies de l'autor/a

Robert Martínez-Carrasco, Universitat Jaume I

Holds a PhD in Applied Languages, Literature and Translation from Jaume I University (Spain). His research interests include translation education, critical pedagogy and the construction of gender, sex and identity. At Universitat Jaume I, he teaches courses in Catalan-English translation and English linguistics.

Alicia Chabert, Universitat Jaume I

Holds a PhD in Applied Languages, Literature and Translation from Jaume I University (Spain). Her research interests include the role of the mother tongue in foreign language learning and English as a Lingua Francia. At Universitat Jaume I, she teaches Spanish-English translation.