Writing on steroids? Accuracy of automatic corrective feedback in L2 competence development
Abstract
The article examines the use of written corrective feedback by online grammar checkers as a tool for L2 competence development. While feedback on students’ writing is considered an instrumental part in their learning process, providing comprehensive feedback may not always be possible. Grammar checkers can therefore become an interesting tool to scaffold the students’ learning process as they promote self-directed learning. For the purposes of this study, a set of authentic EFL compositions in English (n=91; C1 CEFR) was subjected to a popular Automated Writing Evaluation (AWE) system, Grammarly, in order to analyse the feedback provided, namely the types and frequency of errors and the level of accuracy in error detection. The findings determine that, while grammar checkers may be useful as a complementary tool, they are developed with native speakers in mind, so the feedback does not always meet the EFL learners’ needs.
Keywords
Grammarly, AWE, EFL, Corrective feedback, Writing competence assessmentReferences
Alakrash, H. M., & Razak, N. A. (2021). Technology-based language learning: Investigation of digital technology and digital literacy. Sustainability (Switzerland), 13(21). https://doi.org/10.3390/su132112304
Bachman, L. F. & Palmer, A. S. (2010). Language assessment in practice. Oxford University Press.
Bai, L., & Hu, G. (2017). In the face of fallible AWE feedback: how do students respond? Ed-ucational Psychology, 37(1), 67-81. https://doi.org/10.1080/01443410.2016.1223275
Bailey, D., & Lee, A. R. (2020). An Exploratory Study of Grammarly in the Language Learn-ing Context: An Analysis of Test-Based, Textbook-Based and Facebook Corpora. TESOL International Journal, 15(2), 4-27.
Bitchener, J., & Ferris, D. R. (2012). Written Corrective Feedback in Second Language Acqui-sition and Writing. Routledge.
Canale, M. (1983). From communicative competence to communicative language pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and Communication (pp. 2-27). Routledge.
Canale, M. & Swain, M. (1980). Theoretical Bases of Communicative Approaches to Second Language Teaching and Testing. Applied Linguistics, 1(1), 1-47.
Cavaleri, M., & Dianati, S. (2016). You want me to check your grammar again? The useful-ness of an online grammar checker as perceived by students. Journal of Academic Language & Learning, 10(1), 223.
Celce-Murcia, M. (2007). Rethinking the role of communicative competence in language teaching. In E. Alcón Soler & M. P. Safont Jordà (Eds.), Intercultural language use and language learning (pp. 41-57). Springer.
Chabert, A. & Agost, R. (2020). Communicative language teaching: Is there a place for L1 in L2 learning? European Journal of Language Policy, 12(1), 55-83. https://doi.org/10.3828/ejlp.2020.4
Cotos, E. (2014). Genre-Based Automated Writing Evaluation for L2 Research Writing: From Design to Evaluation and Enhancement. Palgrave Macmillan UK. https://doi.org/10.1057/9781137333377_1
Dizon, G., & Gayed, J. M. (2021). Examining the Impact of Grammarly on The Quality of Mobile L2 Writing. JALT CALL Journal, 17(2), 74-92.
https://doi.org/10.29140/JALTCALL.V17N2.336
Dolores, P., Keselman, A., & Monopoli, M. (2003). The academic writing of community col-lege remedial students: Text and learner variables. Higher Education, 45, 19-42.
Dong, Y., & Shi, L. (2021). Using Grammarly to support students’ source-based writing prac-tices. Assessing Writing, 50(September), 100564. https://doi.org/10.1016/j.asw.2021.100564
Elliot, N., & Klobucar, A. (2013). Automated essay evaluation and the teaching of writing. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Evaluation: Cur-rent Applications and New Directions (pp. 16-35). Routledge.
Ferster, B., Hammond, T. C., Curby Alexander, R., & Lyman, H. (2012). Automated forma-tive assessment as a tool to scaffold student documentary writing. Journal of Interac-tive Learning Research, 23(1), 81-99.
Gain, A., Rao, M., & Bhat, K. S. (2019). Usage of Grammarly - online grammar and spelling checker tool at the health sciences library, Manipal Academy of Higher Education, Manipal: A Study. Library Philosophy and Practice, 2019.
Ghufron, M. A., & Rosyida, F. (2018). The Role of Grammarly in Assessing English as a For-eign Language (EFL) Writing. Lingua Cultura, 12(4), 395. https://doi.org/10.21512/lc.v12i4.4582
Graham, S., MacArthur, C. A., & Fitzgerald, J. (2018). Best practices in writing instruction (Third). Guilford Press.
Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. Journal of Technology, Learning, and Assessment, 8(6), 1-43.
Heil, C. R., Wu, J. S., Lee, J. J., & Schmidt, T. (2016). A review of mobile language learning applications: Trends, challenges, and opportunities. The EuroCALL Review, 24(2),
–50. https://doi.org/10.4995/eurocall.2016.6402
Hoover, B., Lytvyn, M., & Shevchenko, O. (2015). Systems and Methods for Advanced Grammar Checking (Patent No. US 9002700 B2). U.S. Patent and Trademark Office.
Huang, H. W., Li, Z., & Taylor, L. (2020). The Effectiveness of Using Grammarly to Improve Students’ Writing Skills. In ACM International Conference Proceeding Series, 2020 (pp. 122-127). https://doi.org/10.1145/3402569.3402594
John, P., & Woll, N. (2020). Using grammar checkers in an ESL context: An investigation of automatic corrective feedback. CALICO Journal, 37(2), 169-192. https://doi.org/10.1558/cj.36523
Khoshnevisan, B. (2019). The affordances and constraints of automatic writing evaluation (AWE) tools: A case for Grammarly. ARTESOL EFL Journal, 2(2), 12-25.
Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44(September 2019), 100450. https://doi.org/10.1016/j.asw.2020.100450
Krashen, S. D. (1982). Principles and Practice in Second Language Acquisition. Pergamon Press Inc. http://www.sdkrashen.com/content/books/principles_and_practice.pdf
Lastres-López, C., & Manalastas, G. (2017). Errors in L1 and L2 University Students’ Writing in English: Grammar, Spelling and Punctuation. RAEL: Revista Electrónica de Lingüística Aplicada, 16(2), 118-135
Lailika, H. I. (2019). Students’ Peceptions of The Use of Grammarly as an Online Grammar Checker in Thesis Writing. http://digilib.uinsby.ac.id/34607/
Leeman, J. (2007). Feedback in L2 learning: Responding to errors during practice. In R. DeKeyser (Ed.), Practice in a Second Language: Perspectives from Linguistics and Psychology (pp. 111-138). Cambridge University Press. https://doi.org/10.1017/cbo9780511667275.007
Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evalua-tion (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1-18. https://doi.org/10.1016/j.jslw.2014.10.004
Nova, M. (2018). Utilizing Grammarly in Evaluating Academic Writing: a Narrative Research on Efl Students’ Experience. Premise: Journal of English Education, 7(1), 80-97. https://doi.org/10.24127/pj.v7i1.1300
Nunan, D. (1999). Second language teaching & learning. Heinle & Heinle Publishers.
Page, E. B. (2003). Project Essay Grade: PEG. In M. Shermis & B. JC (Eds.), Automated essay scoring: A cross-disciplinary perspective. (pp. 43-54). Lawrence Erlbaum Associates Publishers.
Park, J. (2019). An AI-based English Grammar Checker vs. Human Raters in Evaluating EFL Learners’ Writing. Multimedia-Assisted Language Learning, 22(1), 112-131.
http://journal.kamall.or.kr/wp-content/uploads/2019/3/Park_22_1_04.pdfhttp://www.kamall.or.kr
Pratama, Y. D. (2020). The Investigation of Using Grammarly As Online Grammar Checker in the Process of Writing. English Ideas: Journal of English Language Education, 1(1), 46-54.
Ranalli, J. (2018). Automated written corrective feedback: how well can students make use of it? Computer Assisted Language Learning, 31(7), 653-674. https://doi.org/10.1080/09588221.2018.1428994
Ranalli, J., Link, S., & Chukharev-Hudilainen, E. (2017). Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and use-fulness of feedback as part of argument-based validation. Educational Psychology, 37(1), 8-25. https://doi.org/10.1080/01443410.2015.1136407
Richards, J. C., & Rodgers, T. (1999). Approaches and methods in language teaching. Cam-bridge University Press.
Sahu, S. (2020). Evaluating performance of different grammar checking tools. International Journal of Advanced Trends in Computer Science and Engineering, 9(2), 2227-2233. https://doi.org/10.30534/ijatcse/2020/201922020
Savignon, S. J. (2002). Communicative Language Teaching: Linguistic Theory and Classroom Practice. In S. J. Savignon (Ed.), Interpreting Communicative Language Teaching: Contexts and Concerns in Teacher Education (pp.1-28). Yale University Press.
Schindler, L. A., Burkholder, G. J., Morad, O. A., & Marsh, C. (2017). Computer-based tech-nology and student engagement: a critical review of the literature. International Jour-nal of Educational Technology in Higher Education, 14(1).
https://doi.org/10.1186/s41239-017-0063-0
Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quali-ty of writing. Assessing Writing, 19, 51-65. https://doi.org/10.1016/j.asw.2013.11.007
Weigle, S. C. (2013). English as a Second Language Writing and Automated Essay Evaluation. In M. D. Shermis & J. Burstein (Eds.), Handbook of Automated Essay Evaluation (pp. 36-54). Routledge.
Wilson, J. (2017). Associated effects of automated essay evaluation software on growth in writing quality for students with and without disabilities. Reading and Writing, 30(4), 691-718. https://doi.org/10.1007/s11145-016-9695-z
Woodworth, J., & Barkaoui, K. (2020). Perspectives on Using Automated Writing Evaluation Systems to Provide Written Corrective Feedback in the ESL Classroom. TESL Canada Journal, 37(2), 234-247. https://doi.org/10.18806/tesl.v37i2.1340
Zinkevich, N. A., & Ledeneva, T. V. (2021). Using Grammarly to Enhance Students’ Academ-ic Writing Skills. Professional Discourse & Communication, 3(4), 51-63. https://doi.org/10.24833/2687-0126-2021-3-4-51-63
Published
How to Cite
Downloads
Copyright (c) 2023 Robert Martínez-Carrasco, Alicia Chabert
This work is licensed under a Creative Commons Attribution 4.0 International License.
