Incidental vocabulary recognition effects of subtitled, captioned and reverse subtitled audiovisual input

Jana van der Kolk; Sara Feijoo

doi:10.4995/rlyla.2024.18056

Authors

Jana van der Kolk Universitat de Barcelona
Sara Feijoo Universitat de Barcelona https://orcid.org/0000-0002-2679-7367

DOI:

https://doi.org/10.4995/rlyla.2024.18056

Keywords:

EFL, subtitles, captions, reverse subtitles, multimodal input

Abstract

This study compares effects of brief exposure to L1 subtitled, L2 captioned, and reverse subtitled audiovisual input on three aspects of vocabulary learning: meaning, form, and pronunciation of target language words. A within-subjects design was used, in which three video clips in the different viewing modes were shown to ten L1 Spanish participants who underwent a pre-test, post-test, and delayed post-test of English terms that each occurred in just one of the videos. The tests measured recognition of meaning, form, and pronunciation through translation into the L1, dictation, and reading the target words aloud with native speaker ratings, respectively. The findings show statistically significant gains in producing the accurate written form of vocabulary in reverse subtitled and L2 captioned video clips, and mixed results for the other variables, including statistically significant gains in pronunciation of vocabulary with L1 subtitles.

Downloads

Download data is not yet available.

References

Araujo, L., & Costa, P. (2013). The European Survey on Language Competences: School-Internal and External Factors in Language Learning. JRC Scientific and Technical Report (EU 26078-2013). Luxemburg: Publications Office of the European Union.

Aydin Yildiz, T. (2017). The Effect of Videos with Subtitles on Vocabulary Learning of EFL Learners. International Journal of Humanities and Social Science 7(9), 125-130.

Bird, S.A., & Williams, J.N. (2002). The Effect of Bimodal Input on Implicit and Explicit Memory: An Investigation into the Benefits of Within-Language Subtitling. Applied Psycholinguistics, 23(4), 509-533. https://doi.org/10.1017/S0142716402004022

Bisson, M.J., van Heuven, W.J.B., Conklin, K., & Tunney, R.J. (2015). The role of verbal and pictorial information in multimodal incidental acquisition of foreign language vocabulary. Quarterly Journal of Experimental Psychology, 68(7), 1306-1326. https://doi.org/10.1080/17470218.2014.979211

Bright, K.S., Crane, D., & Kauffman, M. (Producers). (1995). Friends [Television series]. New York: NBC Studios.

Caruana, S. (2021). An Overview of Audiovisual Input as a Means for Foreign Language Acquisition in Different Contexts. Language and Speech. https://doi.org/10.1177/0023830920985897

Compleat Web VP. Retrieved March 2, 2021, from https://www.lextutor.ca/vp/comp/

Corpus of Contemporary American English. Retrieved March 2, 2021, from https://www.english-corpora.org/

Danan, M. (1992). Reversed Subtitling and Dual Coding Theory: New directions for foreign language instruction. Language Learning, 42(4), 497- 527. https://doi.org/10.1111/j.1467-1770.1992.tb01042.x

d'Ydewalle, G. (2002). The Case of Subtitling: Foreign-Language Acquisition by Watching Subtitled Television Programs. Journal of Foreign Language Education and Research, 12, 59-77.

Fazilatfar, A.M., Shahin, G., & Samavarchi, L. (2011). The effect of standard and reversed subtitling versus no subtitling mode on L2 vocabulary learning. The Journal of Teaching Language Skills, 3(1), 43-64.

Fievez, I., Montero Perez, M., Cornillie, F., & Desmet, P. (2020). Vocabulary Learning through Viewing Captioned or Subtitled Videos and the Role of Learner-and Word-related Factors. CALICO Journal, 37(3), 21-41. https://doi.org/10.1558/cj.39370

Garnier, M. (2014). Intentional vocabulary learning from watching DVDs with subtitles: A case study of an "average" learner of French. International Journal of Research Studies in Language Learning, 3(1), 21-32. https://doi.org/10.5861/ijrsll.2013.521

Gorjian, B. (2014). The Effect of Movie Subtitling on Incidental Vocabulary Learning among EFL Learners. International Journal of Asian Social Science, 4(9), 1013-1026.

Gass, S., Winke, P., Isbell, D., & Ahn, J. (2019). How captions help people learn languages: A working-memory, eye-tracking study. Language Learning & Technology, 23(2), 84-104.

Hsieh, Y. (2020). Effects of Video Captioning on EFL Vocabulary Learning and Listening Comprehension. Computer Assisted Language Learning, 33(5-6), 567-589. https://doi.org/10.1080/09588221.2019.1577898

Jaber, R.A., & Dollar, Y.K. (2019). A Meta-analysis of the Effect of Bimodal Subtitling on Vocabulary Learning among Adult EFL Learners. Focus on ELT Journal, 1(1), 43-52. https://doi.org/10.14744/felt.2019.00001

Jelani, N.A. M., & Boers, F. (2018). Examining incidental vocabulary acquisition from captioned video. ITL - International Journal of Applied Linguistics, 169(1), 169-190. https://doi.org/10.1075/itl.00011.jel

Koolstra, C.M., Peeters, A.L. & Spinhof, H. (2002). The Prons and Cons of Dubbing and Subtitling. European Journal of Communication, 17, 325-351. https://doi.org/10.1177/0267323102017003694

Kuppens, A.H. (2010). Incidental Foreign Language Acquisition from Media Exposure. Learning, Media and Technology, 35(1), 65-85. https://doi.org/10.1080/17439880903561876

Liao, S., Kruger, J., & Doherty, S. (2020). The impact of monolingual and bilingual subtitles on visual attention, cognitive load, and comprehension. The Journal of Specialised Translation, 33, 70-98.

Mardani, M., & Najmabadi, A. (2016). The Effect of Movies with Different Types of Subtitles on Incidental English Vocabulary Learning of Iranian High School Students. Iranian Journal of Research in English Language Teaching, 3(2), 46-57.

Markham, P.L., Peter, L.A., & McCarthy, T.J. (2001). The Effects of Native Language vs. Target Language Captions on Foreign Language Students' DVD Video Comprehension. Foreign Language Annals, 34(5), 439-445. https://doi.org/10.1111/j.1944-9720.2001.tb02083.x

Markham, P., & Peter, L. (2003). The Influence of English Language and Spanish Language Captions on Foreign Language Listening/Reading Comprehension. Journal of Educational Technology Systems, 31(3), 331-341. https://doi.org/10.2190/BHUH-420B-FE23-ALA0

Matielo, R., D'Ely, R.C. S.F., & Baretta, L. (2015). The Effects of Interlingual and Intralingual Subtitles on Second Language Learning/Acquisition: A State-of-the-Art Review. Trabalhos Em Linguística Aplicada, 54, 161-182. https://doi.org/10.1590/0103-18134456147091

Matielo, R., Oliveira, R.P. de, & Baretta, L. (2018). Subtitling, Working Memory, and L2 Learning: A Correlational Study. Revista Brasileira de Linguística Aplicada, 18(3). https://doi.org/10.1590/1984-6398201812773

Mayer, R. (2020). Multimedia Learning (3rd ed.). Cambridge University Press. https://doi.org/10.1017/9781108894333.003

Mitterer, H., & McQueen, J.M. (2009). Foreign Subtitles Help but Native-Language Subtitles Harm Foreign Speech Perception. PLoS ONE, 4(11), 4-8. https://doi.org/10.1371/journal.pone.0007785

Montero Perez, M., Peters, E., & Desmet, P. (2015). Enhancing Vocabulary Learning Through Captioned Video: An Eye-Tracking Study. Modern Language Journal, 99(2), 308-328. https://doi.org/10.1111/modl.12215

Montero Perez, M., Van Den Noortgate, W., & Desmet, P. (2013). Captioned Video for L2 Listening and Vocabulary Learning: A Meta-analysis. System, 41(3), 720-739. https://doi.org/10.1016/j.system.2013.07.013

Muñoz, C. (2017). The Role of Age and Proficiency in Subtitle Reading: an eye-tracking study. System, 67, 77-86. https://doi.org/10.1016/j.system.2017.04.015

Nasab, M.S. B., & Motlagh, S.F. P. (2017). Vocabulary Learning Promotion through English Subtitled Cartoons. Communication and Linguistics Studies, 3(1-1), 1-7.

Pattemore, A., & Muñoz, C. (2020). Learning L2 Constructions from Captioned Audio-visual Exposure: The effect of learner-related factors. System, 93, 1-13. https://doi.org/10.1016/j.system.2020.102303

Pattemore, A., Suárez, M.M., & Muñoz, C. (2020). Exploring L2 TV mode preferences and perceptions of learning. In CALL for widening participation: short papers from EUROCALL 2020, 272-278. Research-publishing.net. https://doi.org/10.14705/rpnet.2020.48.1200

Pemberton, L., S. Fallahkhair & J. Masthoff (2005). Learner-Centred Development of a Mobile and iTV Language Learning Support System. Educational Technology & Society, 8(4), 52-63.

Peters, E., Heynen, E., & Puimège, E. (2016). Learning Vocabulary through Audiovisual Input: The differential effect of L1 subtitles and captions. System, 63, 134-148. https://doi.org/10.1016/j.system.2016.10.002

Puimège, E., & Peters, E. (2019). Learning L2 vocabulary from audiovisual input: an exploratory study into incidental learning of single words and formulaic sequences. The Language Learning Journal, 47(4). https://doi.org/10.1080/09571736.2019.1638630

Pujadas, G., & Muñoz, C. (2019). Extensive Viewing of Captioned and Subtitled TV Series: A study of L2 vocabulary learning by adolescents. Language Learning Journal, 47(4), 479-496. https://doi.org/10.1080/09571736.2019.1616806

Pujadas, G., & Muñoz, C. (2020). Examining Adolescent EFL Learners' TV Viewing Comprehension through Captions and Subtitles. Studies in Second Language Acquisition, 42(3), 551-575. https://doi.org/10.1017/S0272263120000042

Rodgers, M.P. H., & Webb, S. (2017). The effects of captions on EFL learners' comprehension of English-language television programs. CALICO Journal, 34(1), 20-38. https://doi.org/10.1558/cj.29522

Sadiku, A. (2018). The Role of Subtitled Movies on Students' Vocabulary Development. International Journal of Sciences: Basic and Applied Research, 42(1), 212-221.

Sockett, G. (2014). The Online Informal Learning of English. Basingstoke: Palgrave Macmillan. https://doi.org/10.1057/9781137414885

Sydorenko, T. (2010). Modality of input and vocabulary acquisition. Language Learning & Technology, 14(2), 50-73.

Teng, F. (2019). Incidental vocabulary learning for primary school students: the effects of L2 caption type and word exposure frequency. The Australian Educational Researcher, 46(1), 113-136. https://doi.org/10.1007/s13384-018-0279-6

Vanderplank, R. (2016). 'Effects of' and 'Effects with' Captions: How exactly does watching a TV programme with same-language subtitles make a difference to language learners? Language Teaching, 49(2), 235-250. https://doi.org/10.1017/S0261444813000207

Vanderplank, R. (2019). 'Gist Watching Can Only Take You So Far': Attitudes, strategies and changes in behaviour in watching films with captions. Language Learning Journal, 47(4), 407-423. https://doi.org/10.1080/09571736.2019.1610033

Vulchanova, M., Aurstad, L.M. G., Kvitnes, I.E. N., & Eshuis, H. (2015). As naturalistic as it gets: subtitles in the English classroom in Norway. Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.01510

Wang, Y. (2019). Effects of L1/L2 Captioned TV Programs on Students' Vocabulary Learning and Comprehension. CALICO Journal, 36(3), 204-224. https://doi.org/10.1558/cj.36268

Webb, S., & Rodgers, M.P.H. (2009a). The Lexical Coverage of Movies. Applied Linguistics, 30(3), 407-427. https://doi.org/10.1093/applin/amp010

Webb, S., & Rodgers, M.P.H. (2009b). Vocabulary Demands of Television Programs. Language Learning, 59(2), 335-366. https://doi.org/10.1111/j.1467-9922.2009.00509.x

Wisniewska, N., & Mora, J.C. (2020). Can Captioned Video Benefit Second Language Pronunciation? Studies in Second Language Acquisition, 42(3). https://doi.org/10.1017/S0272263120000029

Winke, P., Gass, S., & Sydorenko, T. (2010). The Effects of Captioning Videos Used for Foreign Language Listening Activities. Language Learning & Technology, 14(1), 65-86.

Yang, J. (2018). Use of L1 and L2 Subtitles for Incidental Vocabulary Learning in a Second Language: Focusing on Word Frequency. STEM Journal, 19, 127-147. https://doi.org/10.16875/stem.2018.19.3.127