LexVEN
A quick vocabulary test for proficiency in Venetan
Abstract
While there is growing awareness that knowledge of regional minority languages must be protected, assessing the vitality of such varieties is challenging due to the lack of assessment materials. Our contribution documents the creation and validation of LexVEN, a bimodal yes/no vocabulary task, with the purpose of assessing the proficiency of users of Venetan with receptive and/or productive skills. The materials will be freely available. The vocabulary items were validated in a survey with 161 speakers of Italian, of which 136 came from the Veneto area. We have relied on Classical Test Theory (CTT) and Item Response Theory (IRT) in validating test items, and we demonstrate correlations with self-reported measures of proficiency and language use. Given the absence of any other assessment tools of Venetan for cross-validation, we further collected speech samples from a subset of the Venetan participants with active proficiency whose proficiency was rated by native speakers of Venetan. We discuss the procedures within a validation framework, concluding that LexVEN is reliable as a proficiency assessment tool of Venetan.
Keywords
Venetan, regional minorities, vocabulary, proficiency test, validationReferences
Alderson, Charles. 2005. Diagnosing foreign language proficiency. Continuum.
Alderson, Charles, & Ari Huhta. 2005. The development of a suite of computer-based diagnosis tests based on the Common European Framework. Language Testing 22(3): 301–320. https://doi.org/10.1191/0265532205lt310oa
Amenta, Simona, Badan, Linda, & Marc Brysbaert. 2021. LexITA: A Quick and Reliable Assessment Tool for Italian L2 Receptive Vocabulary Size. Applied Linguistics 42(2): 292–314. https://doi.org/10.1093/applin/amaa020
Bachman, Lyle F. 2004. Statistical analyses for language assessment. Cambridge University Press.
Bates, Douglas, Maechler, Martin, Bolker, Ben, & Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67(1): 1–48. https://doi.org/10.18637/jss.v067.i01
Beeckmans, Renaud, Eyckmans, June, Janssens, Vera, Dufranne, Michel, & Hans Van de Velde. 2001. Examining the yes/no vocabulary test: Some methodological issues in theory and practice. Language Testing 18(3): 235–274. https://doi.org/10.1177/026553220101800301
Berruto, Gaetano. 2018. The languages and dialects of Italy. In W. Ayres-Bennett, J. Carruthers (eds), Manual of Romance Sociolinguistics, 494–525. Berlin; Boston: De Gruyter. https://doi.org/10.1515/9783110365955-019
Besler, Alexandra, Ferin, Maria, Venagli, Ilaria, & Tanja Kupisch. 2023. On the relation between attitudes and dialect maintenance (Sicilian and Venetan) in Italy. Ms, University of Konstanz.
Bonifacio, Emily. 2014. The Acquisition of the Copula in two Bilingual Italian-Venetan Children. MA thesis. Ca’ Foscari University, Venice.
Brandi, Luciana, & Patrizia Cordin. 1981. Dialetti e italiano: Un confronto sul parametro del soggetto nullo. Rivista di Grammatica Generativa 6: 33–87.
Brandi, Luciana, & Patrizia Cordin. 1989. Two Italian dialects and the null subject parameter. In O. Jaeggli, & K. J. Safir (eds), The null subject parameter 111–142. Dordrecht: Kluwer. https://doi.org/10.1007/978-94-009-2540-3_4
Brysbaert, Marc. 2013. LEXTALE_Fr. A fast, free, and efficient test to measure language proficiency in French. Psychologica Belgica 53: 23–37. https://doi.org/10.5334/pb-53-1-23
Cardinaletti, Anna. 2013. On the bilingual acquisition of Italian and venetan dialects: a focus on subject and object clitic pronouns. Paper Presented at the Generative Linguistics in the Old World (Glow) 36, April 2-6 2013, Lund University.
Chapelle, Carol, & Hye-won Lee. 2021. Understanding argument-based validity in language testing. In C. Chapelle, & E. Voss (eds), Validity argument in language testing, 19–44. Cambridge University Press. https://doi.org/10.1017/9781108669849.004
Cortelazzo, Michele. 1999. Itinerari dialettali veneti. Esedra.
Eberhard, David, Simons, Gary, & Charles Fennig (eds). 2022. Ethnologue: Languages of the World. Twenty-fifth edition. SIL International. http://www.ethnologue.com
Eyckmans, June. 2004. Measuring receptive vocabulary size. Reliability and validity of the Yes/No vocabulary test for French-speaking learners of Dutch. Utrecht . LOT.
Ferré, Pilar, & Marc Brysbaert. 2017. Can Lextale-Esp discriminate between groups of highly proficient Catalan–Spanish bilinguals with different language dominances?. Behavioral Research Methods 49(2): 717–723. https://doi.org/10.3758/s13428-016-0728-y
Gagarina, Natalia & Ute Bohnacker. 2022. Storytelling in bilingual children. How organization of narratives is (not) affected by linguistic skills and environmental factors. Linguistic Approaches to Bilingualism 12(4): 407–412. https://doi.org/10.1075/lab.00035.int
Garraffa, Maria, Beveridge, Madeleine, & Antonella Sorace. 2015. Linguistic and cognitive skills in Sardinian–Italian bilingual children. Frontiers in Psychology 6(1898). https://doi.org/10.3389/fpsyg.2015.01898
Garraffa, Maria, Obregon, Matteo, & Antonella Sorace. 2017. Linguistic and Cognitive Effects of Bilingualism with Regional Minority Languages: A Study of Sardinian–Italian Adult Speakers. Frontiers in Psychology 8(1907). https://doi.org/10.3389/fpsyg.2017.01907
Gyllstad, Henrik. 2007. Testing English collocations: Developing receptive tests for use with advanced Swedish learners. Lund University.
Gyllstad, Henrik. 2013. Looking at L2 vocabulary knowledge dimensions from an assessment perspective – Challenges and potential solutions. In C. Bardel, B. Laufer, & C. Lindqvist (eds), L2 Vocabulary acquisition, knowledge and use: New perspectives on assessment and corpus analysis, 11–28. Eurosla Monographs Series, 2. EUROSLA.
Gyllstad, Henrik, McLean, Stuart, & Jeffrey Stewart. 2021. Using Confidence Intervals to Determine Adequate Item Sample Sizes for Vocabulary Tests: An Essential but Overlooked Practice. Language Testing 38(4): 558–579. https://doi.org/10.1177/0265532220979562
Gyllstad, Henrik, Kupisch, Tanja & Anika Lloyd Smith. 2023. NSVT—Developing a North Sámi vocabulary test. Ms, University of Lund/University of Konstanz/UiT The Arctic University of Norway.
Harrington, Michael, & Michael Carey. 2009. The on-line Yes/No test as a proficiency tool. System 37(4): 614–626. https://doi.org/10.1016/j.system.2009.09.006
Hashimoto, Brett J. 2021. Is frequency enough?: The frequency model in vocabulary size testing. Language Assessment Quarterly 18(2): 171–187. https://doi.org/10.1080/15434303.2020.1860058
Huibregtse, Ineke, Admiraal, Wilfred, & Paul Meara. 2002. Scores on a yes–no vocabulary test: Correction for guessing and response style. Language Testing 19(3): 227–245. https://doi.org/10.1191/0265532202lt229oa
International Commission of European Citizens. 2022. Venetian is a language and not a dialect. https://icec.ngo/news/venetian-is-a-language-and-not-a-dialect
ISTAT. 2012. L’uso della lingua italiana, dei dialetti e di altre lingue in Italia, published online 27th October 2014, http://www.istat.it/archivio/136496
Izura, Cristina, Cuetos, Fernando, & Marc Brysbaert. 2014. Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica 35: 49–66.
Kane, Michael. 2012. Validating score interpretations and uses. Language Testing 29(1): 3–17. https://doi.org/10.1177/0265532211417210
Kane, Michael. 2013. Validating the interpretations and uses of test scores. Journal of Educational Measurement 50(1): 1–73. https://doi.org/10.1111/jedm.12000
Kupisch, Tanja, & Ewgenia Klaschik. 2017. Cross-lectal influence and gender marking in bilectal Venetan-Italian acquisition. In E. Blom, L. Cornips, & J. Schaeffer, Cross-Linguistic Influence in Bilingualism, 127–152. John Benjamins. https://doi.org/10.1075/sibil.52.07kup
Kupisch, Tanja, Arona, Sebastiano, Besler, Alexandra, Cruschina, Silvio, Ferin, Maria, Gyllstad, Henrik, & Ilaria Venagli. 2023. LexSIC: A quick vocabulary test for Sicilian. Isogloss. Open Journal of Romance Linguistics 9(1)/6: 1–24. https://doi.org/10.5565/rev/isogloss.302
Laufer, Batia, & Zahava Goldstein. 2004. Testing vocabulary knowledge: Testing Vocabulary Knowledge: Size, Strength, and Computer Adaptiveness. Language Learning 54: 399–436. https://doi.org/10.1111/j.0023-8333.2004.00260.x
Leiner, Dominik J. 2014. SoSci Survey (Version 2.4.00-i). [Computer software]
Lenth, Russel. 2023. emmeans: Estimated Marginal Means, aka Least-Squares Means. R package version 1.8.4-1, https://CRAN.R-project.org/package=emmeans
Leivada, Evelina, Papadopoulou, Elena, Kambanaros, Maria, & Kleanthes Grohman. 2017. The Influence of Bilectalism and Non-standardization on the Perception of Native Grammatical Variants. Frontiers in Psychology 8(205). https://doi.org/10.3389/fpsyg.2017.00205
Lemhöfer, Kristin, & Mirjam Broersma. 2012. Introducing LexTALE: A quick and valid Lexical Test for Advanced Learners of English. Behavioral Research Methods 44: 325–343. doi:10.3758/s13428-011-0146-0
Linacre, John Michael. 2002. What do infit and outfit, mean-square and standardized mean. Rasch Measurement Transactions 16(2): 878. www.rasch.org/rmt/rmt162f.htm
Linacre, John Michael 2021. ‘Winsteps® Rasch measurement computer program’ Version 5.1.6. Winsteps.com
Loporcaro, Michele. 2013. Profilo Linguistico dei dialetti italiani. Laterza. [2nd edition]
MacIntyre, Peter, Noels, Kimberley, & Richard Clément. 1997. Biases in self-ratings of second language proficiency: The role of language anxiety. Language Learning 47: 265–287. https://doi.org/10.1111/0023-8333.81997008
Mair, Patrick, Hatzinger, Reinhold, & Marco Maier. 2021. Extended Rasch modeling: The eRm package for the application of IRT models in R. Journal of Statistical Software 20: 1–20.
Marcato, Carla. 2004. Il Veneto. In M. Cortelazzo, C. Marcato, N. De Blasi, & G. P. Clivio (eds), I dialetti italiani: Storia, struttura, uso, 297–328. UTET.
Meara, Paul. 1996. The dimensions of lexical competence. In G. Brown, K. Malmkjaer, & J. Williams (eds), Performance and competence in second language acquisition, 35–53. Cambridge University Press.
Meara, Paul, & Barbara Buxton, B. 1987. An alternative to multiple choice vocabulary tests. Language Testing 4(2): 142–154. https://doi.org/10.1177/026553228700400202
Meara, Paul, & Imma Miralpeix. 2017. Tools for researching vocabulary. Multilingual Matters.
Mochida, Akira, & Michael Harrington. 2006. The Yes/No test as a measure of receptive vocabulary knowledge. Language Testing 23(1): 73–98. https://doi.org/10.1191/0265532206lt321oa
Moseley, Christopher. 2010. Atlas of the worlds’ languages in danger. Unesco Publishing.
Pellicer-Sánchez, Anna, & Norbert Schmitt. 2012. ‘Scoring Yes–No vocabulary tests: Reaction time vs. nonword approaches. Language Testing 29(4): 489–509. https://doi.org/10.1177/0265532212438053
Plonsky, Luke, & Frederick L Oswald. 2014. How big is “big”? Interpreting effect sizes in L2 research. Language Learning 64(4): 878–912. https://doi.org/10.1111/lang.12079
Poletto, Cecilia. 1993. La sintassi del soggetto nei dialetti italiani settentrionali. Unipress.
R Core Team. 2022. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org
Salmela, Rosa, Lehtonen, Minna, Garusi, Stefano, & Raymond Bertram. 2021. Lexize: A test to quickly assess vocabulary knowledge in Finnish. Scandinavian Journal of Psychology 62(6): 775–894. https://doi.org/10.1111/sjop.12768
Sanfelici, Emmanuela, & Maja Roch. 2021. The Native Speaker in Italian-Dialects Bilingualism: Insights from the acquisition of Vicentino by preschool children. Frontiers in Psychology 12. https://doi.org/10.3389/fpsyg.2021.717639
Santipolo, Matteo, & Silvestro Tucciarone. 2006. Dalla semidialettofonia di ritorno al bilinguismo consapevole: un’ipotesi di evoluzione sociolinguistica in Veneto. In I. Tempesta, & M. Maggio (eds), Lingue in contatto a scuola. Tra italiano, dialetto e italiano L2, 162–168. Milano, Franco Angeli.
Schmitt, Norbert. 2010. Researching vocabulary: A vocabulary research manual. Palgrave Macmillan.
Schmitt, Norbert, Nation, Paul, & Benjamin Kremmel. 2020. Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching 53(1): 109–120. https://doi.org/10.1017/S0261444819000326
Staehr, Lars. 2008. Vocabulary size and the skills of listening, reading and writing. Language Learning Journal 36(2): 139–152. https://doi.org/10.1080/09571730802389975
Stubbe, Raymond. 2012. Do pseudoword false alarm rates and overestimation rates in yes–no vocabulary tests change with Japanese university students’ ability levels? Language Testing 29(4): 471–488. https://doi.org/10.1177/0265532211433033
Uchihara, Takumi, & Jon Clenton. 2020. Investigating the role of vocabulary size in second language speaking ability. Language Teaching Research 24(4): 540–556. https://doi.org/10.1177/1362168818799371
Van der Jeught, Stefaan. 2016. The protection of linguistic minorities in Italy: A clean break with the past. Journal on Ethnopolitics and Minority Issues in Europe 15(3): 57-81.
Vangsnes, Øystein, Söderlung, Göran, & Morten Blekesaune. 2015. The effect of bidialetal literacy on school achievement. International Journal of Bilingual Education and Bilingualism 20(3): 346–361. https://doi.org/10.1080/13670050.2015.1051507
Zamboni, Alberto. 1988. Venezien/Veneto. In G. Holtus, M. Metzeltin, & C. Schmitt (eds), Lexikon der romanistischen Linguistik, Volume 4, 517–538. Tübingen: Niemeyer.
Zhang, Xian, Liu, Jianda, & Haiyang Ai. 2020. Pseudowords and guessing in the Yes/No format vocabulary Test. Language Testing 37(1): 6–30. https://doi.org/10.1177/0265532219862265
Published
How to Cite
Downloads
Copyright (c) 2023 Maria Ferin, Henrik Gyllstad, Ilaria Venagli, Angelica Zordan, Tanja Kupisch
This work is licensed under a Creative Commons Attribution 4.0 International License.
