July 2023, Volume 26, Issue 3

Special Issue on "Contextualized Multimodal Language Learning"

Guest Editor(s): Meei-Ling Liaw and Hsin-I Chen

Download Table of Content in PDF

Download Complete Issue in PDF


Meei-Ling Liaw and Hsin-I Chen

Meei-Ling Liaw

National Taichung University of Education, Taichung, Taiwan // meeilingliaw@gmail.com

Hsin-I Chen

National Taipei University of Technology, Taipei, Taiwan // hichen@mail.ntut.edu.tw


We live in an era of digitally accessible multimodality for various purposes and practices. Researchers and educators agree that multimodal literacies are essential by human beings to communicate, work, and thrive in the global world of the 21st century (Gee, 2003; Jewitt & Kress, 2003; New London Group, 1996). Along with this need, teachers need to be aware of “multimodal possibilities” (Lotherington & Jenson, 2011, p. 227) and their ramifications for teaching and learning. In second/foreign language education, multimodality has become even more central than ever. The interconnectedness among learning contexts, digital tools and materials, and learners is dynamic, multi-faceted, and, more importantly, awaits further exploration so language teachers and learners can transform the understandings into effective pedagogical practices. In this special issue, we present seven research efforts contributing to moving toward this goal. Under the overarching theme of contextualized multimodal language learning, the studies tackle issues in theoretical perspectives, methodological choices, educational contexts, and applications of innovative technological tools. Collectively, the studies revealed positive pedagogical values for language teachers of different educational contexts to enhance the learning experiences of different age groups by creatively taking advantage of multiple modes of knowing and meaning-making.


Multimodal, Contextualized language learning, Digital tools, Language teaching

Cite as:Liaw, M.-L., & Chen, H.-I. (2023). Guest Editorial: Contextualized Multimodal Language Learning. Educational Technology & Society, 26(3), 1-4.
Published November 15, 2022

Special Issue Articles

Pei-Lin Liu

National Chia-Yi University, Taiwan // peilin@mail.ncyu.edu.tw

Chiu-Jung Chen

National Chia-Yi University, Taiwan // chenc@mail.ncyu.edu.tw


This study aimed to examine the effects of an AI-based object detection translation (AI-based ODT) application (app) on EFL students’ vocabulary learning. We developed a system that utilized strategies to facilitate learners’ vocabulary learning. The app applied dual code theory to present the objects in picture, word, and pronunciation formats. Seventy-two elementary school students were divided into lower-ability and higher-ability groups according to their English proficiency, and were then randomly assigned to the control and experimental conditions based on their ability. The learners’ learning performance in the control and experimental conditions was compared using a pre-test–post-test design. Through two-way ANOVA analysis, we observed that in the experimental group the higher-ability students benefited more from the AI-based ODT app technology than did the lower-ability students. This significant difference could be taken as evidence of the positive effect of the AI-based ODT app technology, particularly for higher-ability students.


Artificial intelligence, Dual code theory, English as a foreign language (EFL), Object detection translation, Vocabulary learning

Cite as:Liu, P.-L., & Chen, C.-J. (2023). Using an AI-Based Object Detection Translation Application for English Vocabulary Learning. Educational Technology & Society, 26(3), 5-20.
Published November 20, 2022

Shiyan Jiang

North Carolina State University, United States // sjiang24@ncsu.edu


The field of STEM education calls for a nuanced understanding of participation as participation measured by attendance provides limited information about student learning. This multiple case study contributes to a nuanced understanding of youth’s participation trajectories in a multimodal composition project. In the project, fifth to eighth grade students worked in small groups to create multimodal science fiction stories in which they needed to propose creative solutions to issues related to climate change. In this study, I adopted two theoretical perspectives, disciplinary identity development and community of practice, to analyze participation trajectories with multiple sources of data. This study shows that STEM practices mediated by multiple modes can not only offer students flexibility in moving across forms of participation, but also open space for them to demonstrate their expertise as knowledge producers. Furthermore, this study suggests that the following strategies could be effective for broadening participation in STEM practices: supporting the development of reflective understanding of connections between disciplines through digital literacies, providing exposure in composing with multiple modes, focusing on building a close relation between self and digital artifacts, and offering flexibility in moving across interactional spaces. These insights shed light on broadening participation in other multimodal learning settings.


Multimodal composition, Participation trajectory, Integrated STEM, Digital literacy, Broaden participation

Cite as:Jiang, S. (2023). Investigating Adolescents’ Participation Trajectories in a Collaborative Multimodal Composing Learning Environment. Educational Technology & Society, 26(3), 21-36.
Published November 15, 2022

Michael Yi-Chao Jiang, Morris Siu-Yung Jong, Wilfred Wing-Fat Lau, Ching-Sing Chai and Na Wu

Michael Yi-Chao Jiang

School of Foreign Languages, Shenzhen Technology University, China // mjiang@cuhk.edu.hk

Morris Siu-Yung Jong

Department of Curriculum and Instruction, The Chinese University of Hong Kong, Hong Kong SAR // Centre for Learning Sciences and Technologies, The Chinese University of Hong Kong, Hong Kong SAR // mjong@cuhk.edu.hk

Wilfred Wing-Fat Lau

Department of Curriculum and Instruction, The Chinese University of Hong Kong, Hong Kong SAR // wwflau@cuhk.edu.hk

Ching-Sing Chai

Department of Curriculum and Instruction, The Chinese University of Hong Kong, Hong Kong SAR // cschai@cuhk.edu.hk

Na Wu

College of International Business, Shenyang Normal University, China // wunacib@synu.edu.cn


This study examined the effects of using automatic speech recognition (ASR) technology on Chinese students’ willingness to communicate (WTC) in oral English and the development trajectories of their interactional features in a flipped EFL context. One hundred sixty undergraduates from a Chinese university participated in the 14-week quasi-experiment. Both groups were taught in a flipped fashion. The treatment group was required to use the ASR technology for oral practice in their pre-class self-learning, while the control group conducted their self-learning without the ASR technology. The results found that the ASR-based oral practice led to a significant between-group difference in students’ WTC with teacher and class and WTC with non-Chinese, showing that the ASR technology may contribute to improving the Chinese students’ WTC in oral English. Conversely, except for the between-group effect on negotiation for meaning, there was no significant difference between the two groups on the other measures of interactional features. Moreover, none of the interactional features of the students in the treatment group changed significantly over time, indicating a limited role of the ASR technology on Chinese students’ interactional features. Discussions were conducted regarding the contradictory effects of the ASR technology on WTC and peer interaction.


Automatic speech recognition, English as a foreign language, Interactional features, Willingness to communicate

Cite as:Jiang, M. Y. C., Jong, M. S. Y., Lau, W. W. F., Chai, C. S., & Wu, N. (2023). Effects of Automatic Speech Recognition Technology on EFL Learners’ Willingness to Communicate and Interactional Features. Educational Technology & Society, 26(3), 37-52.
Published November 15, 2022

Lucas Kohnke

The Education University of Hong Kong, China // lmakohnke@eduhk.hk

Dennis Foung

The University of British Columbia, Canada // dennis.foung@ubc.ca

Julia Chen

The Hong Kong Polytechnic University, China // julia.chen@polyu.edu.hk


This study investigated how English learners complete multimodal formative quizzes. Participants included 17,950 students enrolled in a mandatory English for Academic Purposes course at a university in Hong Kong. We retrieved data from Blackboard, a learning management system, and conducted a two-step cluster analysis to examine student self-regulated learning (SRL) profiles with the quizzes. We first identified five clusters of learners with distinctively different self-regulated learning patterns. Then, we performed a multivariate analysis of variance (MANOVA) to further explore their differences in SRL, in terms of start day, days started before deadline, differences in scores between first and last attempt, and scores in language learning activities. Our findings echoed those of previous studies on the relationship between self-regulated learning and academic success. This research enables us to better understand the needs of EAP students in Hong Kong.


Cluster analysis, English for Academic Purposes, Multimodal, Formative, Quizzes

Cite as:Kohnke, L., Foung, D., & Chen, J. (2023). Cluster Analysis of Hong Kong Students’ Self-Regulated Learning in Contextualized Multimodal Language Learning. Educational Technology & Society, 26(3), 53-68.
Published November 20, 2022

Muntaha Muntaha

School of Education, Curtin University, Perth, Australia // muntaha.abdulqodir@postgrad.curtin.edu.au

Julian Chen

School of Education, Curtin University, Perth, Australia // julian.chen@curtin.edu.au

Toni Dobinson

School of Education, Curtin University, Perth, Australia // T.Dobinson@Curtin.edu.au


Employing multimodal computer-mediated communication (CMC) for online language learning and teaching has gained momentum worldwide due to the emergence of various digital modes, such as text, image, audio, and video, for online communication. This pilot study aimed to explore students’ learning experiences with multimodal CMC tasks through Instagram. Thirty first-year students at an Indonesian university completed seven CMC tasks, consisting of information gap, reasoning gap, and opinion gap tasks, through three Instagram communication channels: text chat, voice chat, and video chat. Pre- and post-study surveys, journal reflections, and interviews were analyzed using a mixed methods approach. Findings revealed that students overall positively perceived their experiences with tasks delivered through Instagram video, audio, and text chats. They also reported that paralinguistic features afforded by the multimodal Instagram channels—such as emojis, GIFs, images in text chat, intonation in voice chat, and gestures in video chat—facilitated effective communication. However, challenges such as poor internet connections, lack of consciousness of student agency when interacting in video chats, and high anxiety at the beginning of task implementation were also documented during student task performance. The findings suggest that the use of multimodal CMC channels affords greater accessibility and provides multimodal affordances for language learners to communicate using rich semiotic resources. They can strategically draw upon their digital literacy skills to convey messages during meaningful task interaction. Nevertheless, language instructors should consider the availability of internet infrastructure and students’ language proficiency prior to utilizing multimodal CMC channels as language learning tools.


Multimodality, CMC, Social networking sites, TBLT, Instagram

Cite as:Muntaha, M., Chen, J., & Dobinson, T. (2023). Exploring Students’ Experiences of Using Multimodal CMC Tasks for English Communication: A Case with Instagram. Educational Technology & Society, 26(3), 69-83.
Published November 15, 2022

Yueh-hui Vanessa Chiang

National Chengchi University, Taiwan // vanessa.chiang@gmail.com

Ya-Wen Cheng

Asia University, Taiwan // yawencheng@asia.edu.tw

Nian-Shing Chen

National Taiwan Normal University, Taiwan // nianshing@gmail.com


Understanding the obstacles and causes students faced while learning with new technologies is the key to inform effective instructional designs. To achieve this aim, this study conducted a qualitative video analysis on language learners’ observable behaviors when they took part in learning activities supported by the technology of robots and IoT-based tangible objects. Insightful findings and instructional implications emerge from the attempt to explore learners’ learning process in terms of the obstacles learners encountered and the causes of the obstacles. Based on the findings and implications, eight instructional guidelines are proposed for teachers/instructional designers to design effective language learning activities with robots and IoT-based tangible objects. This study contributes to the literature on enhancing learning and teaching by integrating educational robots and IoT-based tangible objects in the field of robot assisted language learning (RALL).


Contextualized multimodal language learning, Robot-assisted language learning, IoT-based tangible objects, Qualitative video analysis, Instructional design guidelines

Cite as:Chiang, Y. V., Cheng, Y.-W., & Chen, N.-S. (2023). Improving Language Learning Activity Design through Identifying Learning Difficulties in a Platform Using Educational Robots and IoT-based Tangible Objects. Educational Technology & Society, 26(3), 84-100.
Published November 15, 2022

Rae Ping Lin

Department of English Language, Literature, & Linguistics, Providence University, Taiwan // raelin@gm.pu.edu.tw

Wen-Chi Vivian Wu

Department of Foreign Languages and Literature, Asia University, Taiwan // Department of Medical Research, China Medical University Hospital, China Medial University, Taiwan // vivwu123@asia.edu.tw


This qualitative study aims to explore how the production of language learning materials using spherical video-based virtual reality (SVVR) affords pre-service teachers multiliteracy development while also attempting to discover their perceptions toward adopting this emerging technology for future language teaching. Data from multiple sources was collected from pre-service English teachers enrolled in a TESOL graduate program in Taiwan, including video-recordings of the participants’ presentations on their final SVVR projects and their self-generated VR teaching materials/artifacts, with two one-to-one semi-structured interviews further analyzed based on thematic analysis. The major findings demonstrate that through conducting the SVVR project, participants learned: (1) to compose multimodal lessons; (2) to concretize intangible contexts for learning; and (3) to use space as a mode for teaching and learning. The authors presented insights into affordance of SVVR material production for their multiliteracy development of engaging pre-service English teachers, as well as their perceptions with regard to this SVVR hands-on experience. Moreover, the authors offer recommendations for putting such experience into practice.


SVVR, Multimodality, Pre-service English teachers

Cite as:Lin, R. P., & Wu, W.-C. V. (2023). Exploring Multiliteracy of Pre-Service Language Teachers through Spherical Video-Based Virtual Reality. Educational Technology & Society, 26(3), 101-114.
Published November 15, 2022

Full Length Articles

Ying-Tien Wu

Graduate Institute of Network Learning Technology, National Central University, Taiwan // ytwu@cl.ncu.edu.tw

Li-Jen Wang

Language Teaching and Research Center, National Yang Ming Chiao Tung University, Taiwan // tommywang@nycu.edu.tw


Previous research has revealed that university students have multiple learning difficulties in argumentative essay writing (AEW). To address this issue, Knowledge building (KB) pedagogy that aims to create holistic learning environments highlighting idea-refinement, learner agency, and collaborative discourse could be promising. Therefore, this study designed and implemented two KB-based holistic AEW instructions integrating KB pedagogy and explicit instruction on argumentative essay structure and writing conventions. A quasi-experimental design explored the effects of the two holistic KB-based AEW instructions on university EFL students’ AEW learning. Two classes of university EFL students were assigned to two instruction groups: The Constant agency enhancement (AE) Instruction group (n = 34) and the Progressive opportunistic collaboration (OC) Instruction group (n = 32). The treatments were two different KB-based holistic AEW instructions for 16 weeks. The participant’s perception of learning environments was assessed before and after the instructions to examine if the learning environments created by the two instructions were aligned with KB pedagogy. To investigate the effects of the two instructions on students’ AEW performance, the students’ argumentative essays were evaluated before, in the middle, and after the instruction. It was found that the two KB-based holistic AEW instructions did align with KB pedagogy but provided university EFL students with distinct and unique learning contexts and opportunities. More importantly, this study also revealed that, compared with the Constant AE instruction, the Progressive OC instruction significantly benefited students more in their argumentative essay writing performance in both the structure and the quality of essays (p < .05).


Knowledge building, Knowledge building pedagogy, Argumentative essay writing, University students, EFL

Cite as:Wu, Y.-T., & Wang, L.-J. (2023). Advancing University EFL Students’ Argumentative Essay Writing Performance through Knowledge-Building-based Holistic Instruction. Educational Technology & Society, 26(3), 115-128.
Submitted April 13, 2022; Revised September 23, 2022; Accepted October 3, 2022; Published November 20, 2022

Rob Elliott

Indiana University Purdue University Indianapolis, USA // elliott@iupui.edu


Higher education has become dependent on the use of digital materials, which may include texts, audiovisual content, and software applications. Because students in higher education are largely responsible for providing the computing devices they are required to use to interact with their digital course materials, instructors and instructional designers are often unaware of the personal computing ecosystems in use by their students. This study describes a large-scale survey of student ownership and use of computing devices at a large public university in the midwestern United States. The results demonstrate that students generally have access to devices that allow them to engage with their digital course materials, but age and demographic factors correlated with socioeconomic status appear to impact the type and quality of devices owned. The study also shows that students have access to a variety of device types and that most students perform their computing tasks on a single screen. Understanding the personal computing ecosystems of students will allow instructors and instructional designers to develop course materials that are accessible to students on the devices in use and can inform the decision-making process when an institution considers adoption of new learning technologies. This data can also be used as a foundation for future studies that examine the influence of a student’s technology access and ownership on their academic outcomes.


Computer-supported collaborative learning, Distance learning/education, Mobile computing, Technology enhanced language learning

Cite as:Elliott, R. (2023). The Demographics of Student Device Ownership: An Examination of the Personal Computing Ecosystems of Students in Higher Education. Educational Technology & Society, 26(3), 129-140.
Submitted May 17, 2022; Revised August 19, 2022; Accepted October 1, 2022; Published November 20, 2022

Students’ Learning Styles and Preferences in a Gamification-enhanced Partially Flipped Classroom: A Q-Methodology Study

Liwen Chen

Implementing Unplugged CS and Use-Modify-Create to Develop Student Computational Thinking Skills: – A Nationwide Implementation in Colombia

Camilo Vieira, Ricardo L. Gómez, Margarita Gómez, Michael Canu and Mauricio Duque

Starting from Volume 17 Issue 4, all published articles of the journal of Educational Technology & Society are available under Creative Commons CC-BY-ND-NC 3.0 license.