Could a multimodal dictionary serve as a learning tool? An examination of the impact of technologically enhanced visual glosses on L2 text comprehension

Takeshi Sato

https://orcid.org/0000-0003-4797-0234

Japan

Tokyo University of Agriculture and Technology

Takeshi Sato is an Associate Professor at the Division of Language and Culture Studies, Tokyo University of Agriculture and Technology, Japan. His academic interests include L2 vocabulary acquisition, computer & mobile-assisted language learning, and English as a Lingua Franca pedagogy.

Submitted: 04/08/2016

|

Accepted: 11/29/2016

|

Published: 09/30/2016

DOI: https://doi.org/10.4995/eurocall.2016.5236
Funding Data

Downloads

Keywords:

animated image, explicit learning, multimodal glosses, online dictionary, prepositions, second vocabulary acquisition

Supporting agencies:

This research was not funded

Abstract:

This study examines the efficacy of a multimodal online bilingual dictionary based on cognitive linguistics in order to explore the advantages and limitations of explicit multimodal L2 vocabulary learning. Previous studies have examined the efficacy of the verbal and visual representation of words while reading L2 texts, concluding that it facilitates incidental word retention. This study explores other potentials of multimodal L2 vocabulary learning: explicit learning with a multimodal dictionary could enhance not only word retention, but also text comprehension; the dictionary could serve not only as a reference tool, but also as a learning tool; and technology-enhanced visual glosses could facilitate deeper text comprehension. To verify these claims, this study investigates the multimodal representations’ effects on Japanese students learning L2 locative prepositions by developing two online dictionaries, one with static pictures and one with animations. The findings show the advantage of such dictionaries in explicit learning; however, no significant differences are found between the two types of visual glosses, either in the vocabulary or in the listening tests. This study confirms the effectiveness of multimodal L2 materials, but also emphasizes the need for further research into making the technologically enhanced materials more effective.
Show more Show less

References:

Al-Seghayer, K. (2001). The effect of multimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning and Technology, 5(1), 202-232.

Boers, F. (2004). Expanding learners' vocabulary through metaphor awareness: What expansion, what learners, what vocabulary? In M. Achard and S. Niemeier (Eds.), Cognitive Linguistics, Second Language Acquisition, and Foreign Language Teaching (pp. 211-232). Berlin: Mouton de Gruyter. https://doi.org/10.1515/9783110199857.211

Boers, F., & Lindstromberg, S. (2008). How cognitive linguistics can foster effective vocabulary teaching. In F. Boers, & S. Lindstromberg (Eds.), Cognitive linguistic approaches to teaching vocabulary and phraseology (pp.1-64). Berlin: Mouton de Gruyter. https://doi.org/10.1515/9783110199161.1.1

Boers, F. (2013). Cognitive linguistic approaches to teaching vocabulary: Assessment and integration. Language Teaching, 46, 208-24. https://doi.org/10.1017/S0261444811000450

Chapelle, C. A. (1998). Multimedia CALL: Lessons to be learned from research on instructed SLA. Language Learning & Technology, 2(1), 22-34.

Cho, K. (2010). Fostering the acquisition of English prepositions by Japanese learners with networks and prototypes. In S. de Knop, F. Boers, & T. De Rycker (Eds.), Fostering language teaching efficiency through cognitive linguistics (pp. 259–276). Berlin, New York: Mouton de Gruyter. https://doi.org/10.1515/9783110245837.257

Chun, D. L., & Plass, J. L. (1996). Effects of multimedia annotations on vocabulary acquisition. Modern Language Journal, 80(2), 183-198 .https://doi.org/10.1111/j.1540-4781.1996.tb01159.x

Dewell, R. B. (1994). Over again: Image-schema transformations in semantic analysis. Cognitive Linguistics, 5, 351-380. https://doi.org/10.1515/cogl.1994.5.4.351

Ellis, N. C. (1995). The psychology of foreign language vocabulary acquisition: Implication for CALL. Computer Assisted Language Learning 8(2-3), 103-128. https://doi.org/10.1080/0958822940080202

Groot, P. (2000). Computer assisted second language vocabulary acquisition. Language Learning and Technology 4(1), 60-81.

Johnson, M. (1987). Body in the mind. Chicago: University of Chicago Press.

Keane, J.A., Nkwenti-Azeh, B., and Singleton, J.A. (1997). Design considerations for a CALL conceptual dictionary. Computer Assisted Language Learning. 10(4), 363-374. https://doi.org/10.1080/0958822970100406

Lakoff, G. (1987). Woman, fire and dangerous thing. Chicago: University of Chicago Press. https://doi.org/10.7208/chicago/9780226471013.001.0001

Langacker, R, W. (1987). Foundations of cognitive grammar, Volume I, theoretical prerequisites. Stanford, California: Stanford University Press.

Lin, C. & Tseng, Y. (2012). Videos and animations for vocabulary learning: A study on difficult words. TOJET: The Turkish Online Journal of Educational Technology, 11(4), 346-355.

Lindstromberg, S. (1996). English prepositions explained. Amsterdam: Benjamins.

Lindstromberg, S. (2001). Preposition entries in UK monolingual learner's dictionaries: Problems and possible solutions. Applied Linguistics, 2 (1), 79-103. https://doi.org/10.1093/applin/22.1.79

Littlemore, J. (2009). Applying cognitive linguistics to second language learning and teaching. Basingstoke: Palgrave Macmillan. https://doi.org/10.1057/9780230245259

Lomicka, L. L. (1998). "To gloss or not to gloss": An investigation of reading comprehension online. Language Learning & Technology, 1, 41-50.

Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86(3), 389-401. https://doi.org/10.1037/0022-0663.86.3.389

Ngu, B, H. & Rethinasamy, S. (2006), Evaluating a CALL software on the learning of English prepositions. Computer & Education, 47, 41-55. https://doi.org/10.1016/j.compedu.2004.08.016

Morimoto, S., & Loewen, S. (2007). A comparison of the effects of image-schema-based instruction and translation-based instruction on the acquisition of L2 polysemous words. Language Teaching Research, 11(3), 347–372. https://doi.org/10.1177/1362168807081181

Pachler, N. (2001). Electronic reference tools for foreign language learners, teachers and users: Offline vocabulary look-up programs. Language Learning Journal, 24, 24-29. https://doi.org/10.1080/09571730185200181

Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart, and Winston.

Sato, T., & Suzuki, A. (2010). Do multimedia-oriented visual glosses really facilitate EFL vocabulary learning?: A comparison of planar images with three- dimensional images. Asian EFL Journal, 12(4), 160-172.

Sato, T., & Suzuki A. (2011). Verifying Multimedia Gloss: Image schema and polysemous vocabulary in English. Proceedings of the EuroCALL 2011 Conference, 285-293.

Sato, T., Lai Y., & Burden, T. (2014). Examining the impact of individual differences of information processing styles in technology-enhanced second vocabulary learning. Proceedings of CLaSIC 2014, 432-440.

Sundberg, P.A. (1998) Animation in CALL: Learning to think in the fourth dimension. In CALICO'98 Symposium (7/1998), San Diego, California. Retrieved 24th September 2016 from http://lrs.ed.uiuc.edu/students/p-sundb/homepage/animCALL.html.

Tanaka, S. (1990). Ninchi imiron [Cognitive Semantics]. Tokyo: San'yusha.

Tanaka, S., Takeda, S., & Kawade, S. (Eds.) (2003). E-gate English-Japanese dictionary. Tokyo: Benesse Corporation.

Yasuda, S. (2010). Learning phrasal verbs through conceptual metaphors: A case of Japanese EFL learners. TESOL Quarterly, 44(2), 250–273. https://doi.org/10.5054/tq.2010.219945

Yeh, Y., & Wang, C. (2003). Effects of multimedia vocabulary annotations and learning styles on vocabulary learning. CALICO Journal, 21(1), 131-144.

Yoshii, M., & Fraitz, J. (2002). Second language incidental vocabulary retention: The effect of text and picture annotation types. CALICO Journal, 20(1), 33-58.

Zwaan, R, A, & Radvansky, G, A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123(2), 162-185. https://doi.org/10.1037/0033-2909.123.2.162

Show more Show less