Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-10T17:14:08.732Z Has data issue: false hasContentIssue false
  • English
  • Français

What makes a good-quality language MOOC? An empirical study of criteria to evaluate the quality of online language courses from learners’ perspectives

Published online by Cambridge University Press:  31 March 2021

Rong Luo Open the ORCID record for Rong Luo [Opens in a new window]  and
Zixuan Ye Open the ORCID record for Zixuan Ye [Opens in a new window]
Rong Luo
Affiliation:
Hangzhou Normal University, Mainland, China (luoronghznu@163.com)
Zixuan Ye
Affiliation:
Hangzhou Normal University, Mainland, China (hshalyy@163.com)
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper focuses on quality assurance in language massive open online courses (LMOOCs). It is a qualitative study that adopts the grounded theory method and analyses evaluative comments on the quality of LMOOCs from learners’ perspectives. With the data collected from 1,000 evaluations from English as a second language (ESL) learners on China’s biggest MOOC platform “iCourse”, this study examines what has influenced learners’ perceptions of LMOOCs and identifies the specific quality criteria of five types of them, including ESL courses for speaking, reading, writing, cultural studies, and integrated skills. The results of the study will lay a foundation for the establishment of a quality criteria framework for LMOOCs and provide insights into design principles for effective online language courses tailored to the diverse needs of a massive number of language learners.

Keywords

language MOOCs (LMOOCs)quality assuranceevaluationlearnersgrounded theory method
Type
Research Article
Information
ReCALL , Volume 33 , Issue 2 , May 2021 , pp. 177 - 192
Check for updates
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of European Association for Computer Assisted Language Learning

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adamopoulos, P. (2013) What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. Proceedings of the Thirty Fourth International Conference on Information Systems: Reshaping society through information systems design. Milan, Italy, 15–18 December.Google Scholar
Aloizou, V., Villagrá Sobrino, S. L., Martínez Monés, A., Asensio-Pérez, J. I. & García-Sastre, S. (2019) Quality assurance methods assessing instructional design in MOOCs that implement active learning pedagogies: An evaluative case study. Proceedings of EMOOCs 2019: Work in progress papers of the research, experience and business tracks. Naples, Italy, 20–22 May.Google Scholar
Bai, B., Chen, L. & Swithenby, S. J. (2014) Critical factors of quality assurance in open university—A case study of UK Open University. China’s Open Education Research, 1: 29–34.Google Scholar
Bárcena, E. & Martín-Monje, E. (2014) Introduction: Language MOOCs: An emerging field. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, 1–15. https://doi.org/10.2478/9783110420067.1 Google Scholar
Beaven, T., Codreanu, T. & Creuzé, A. (2014) Motivation in a language MOOC: Issues for course designers. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, 48–66. https://doi.org/10.2478/9783110420067.4 Google Scholar
Castrillo, M. D. (2014) Language teaching in MOOCs: the integral role of the instructor. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, Language MOOCs, 67–88. http://doi.org/10.2478/9783110420067.5 Google Scholar
Charmaz, K. (2006) Constructing grounded theory: A practical guide through qualitative analysis. London: SAGE.Google Scholar
Charmaz, K. (2007) Constructionism and the grounded theory method. In Holstein, J. A. & Gubrium, J. F. (eds.), Handbook of constructionist research. New York: The Guilford Press, 397–412.Google Scholar
Conole, G. (2016) MOOCs as disruptive technologies: Strategies for enhancing the learner experience and quality of MOOCs. RED: Revista de Educación a Distancia, 50(2): 1–18. https://doi.org/10.6018/red/50/2 Google Scholar
Creelman, A., Ehlers, U.-D. & Ossiannilsson, E. (2014) Perspectives on MOOC quality: An account of the EFQUEL MOOC Quality Project. INNOQUAL: The International Journal for Innovation and Quality in Learning, 2(3): 78–87.Google Scholar
Cross, J. S., Keerativoranan, N., Carlon, M. K. J., Tan, Y. H., Rakhimberdina, Z. & Mori, H. (2019) Improving MOOC quality using learning analytics and tools. In 2019 IEEE Learning With MOOCS (LWMOOCS). Milwaukee: IEEE, 174–179. http://doi.org/10.1109/LWMOOCS47620.2019.8939617 CrossRefGoogle Scholar
Downes, S. (2014) The quality of massive open online courses. In Khan, B. H. & Ally, M. (eds.), International handbook of e-learning, volume 1: Theoretical perspectives and research. New York: Routledge, 65–78.Google Scholar
Dyomin, V., Mozhaeva, G., Babanskaya, O. & Zakharova, U. (2017) MOOC quality evaluation system presentation: Tomsk State University experience. In Kloos, C. D., Jermann, P., Pérez-Sanagustín, M., Seaton, D. T. & White, S. (eds.), Digital education: Out to the world and back to the campus: 5th European MOOCs Stakeholders Summit, EMOOCs 2017, Madrid, Spain, May 22–26, 2017: Proceedings. Cham: Springer, 197–202.CrossRefGoogle Scholar
Ehlers, U.-D., Ossiannilsson, E. & Creelman, A. (2013) Week 1: MOOCs and Quality – Where are we – where do we go from here …? http://mooc.efquel.org/first-post-of-the-series Google Scholar
Finger, G. & Capan, L. (2014) MOOCs and quality issues: A student perspective. ACEC 2014: Australian Computers in Education Conference: Now it’s personal: Innovating education. Adelaide, Australia, 30 September–3 October.Google Scholar
Gamage, D., Fernando, S. & Perera, I. (2015) Factors leading to an effective MOOC from participants perspective. In 2015 8th International Conference on Ubi-Media Computing (UMEDIA). Piscataway: IEEE, 230–235. https://doi.org/10.1109/UMEDIA.2015.7297460 CrossRefGoogle Scholar
Glaser, B. G. & Strauss, A. L. (1967) The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.Google Scholar
Huang, L., Pei, X. & Zhu, X. (2017) An empirical study on influencing factors of MOOCs course quality. Modern Distance Education Research, 5: 78–86.Google Scholar
Kernohan, D. (2014) Talking about quality: Massive misalignment: The challenges of designing and accrediting MOOCs. A response to Benjamin Brabon’s paper. Gloucester: The Quality Assurance Agency for Higher Education, 6–7.Google Scholar
Khalil, M., Taraghi, B. & Ebner, M. (2016) Engaging learning analytics in MOOCS: The good, the bad, and the ugly. In Carmo, M. (ed.), END 2016: International Conference on Education and New Developments. Lisbon: World Institute for Advanced Research and Science, 3–7.Google Scholar
Kinash, S. (2013) MOOCing about MOOCs. Education Technology Solutions, 57: 56–58.Google Scholar
Lowenthal, P. R. & Hodges, C. B. (2015) In search of quality: Using quality matters to analyze the quality of massive, open, online courses (MOOCs). The International Review of Research in Open and Distributed Learning, 16(5): 83–101. https://doi.org/10.19173/irrodl.v16i5.2348 CrossRefGoogle Scholar
Lu, S., Ja, F. & Wu, Y. W. (2016) A survey of the learning of Chinese college English reading course. Journal of Xi’an International Studies University, 9: 72–75.Google Scholar
Luo, R. (2017) Self-directed learning modes in the context of MOOCs. Foreign Language World, 6: 29–36.Google Scholar
Ma, R. (2018) The study of evaluation system of xMOOCs. China’s Information Technology Education Journal, 8: 60–65.Google Scholar
Manning, C., Morrison, B. R. & McIlroy, T. (2014) MOOCs in language education and professional teacher development: Possibilities and potential. SiSAL Journal, 5(3): 294–308. https://doi.org/10.37237/050308 CrossRefGoogle Scholar
Margaryan, A., Manuela, B. & Littlejohn, A. (2015) Instructional quality of massive open online courses (MOOCs). Computers & Education, 80: 77–83. http://doi/org/10.1016/j.compedu.2014.08.005 CrossRefGoogle Scholar
Martín-Monje, E. & Bárcena, E. (eds.) (2014) Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open. https://doi.org/10.2478/9783110420067 CrossRefGoogle Scholar
Merrill, M. D. (2002) First principles of instruction. Educational Technology Research and Development, 50(3): 43–59. http://doi.org/10.1007/bf02505024 CrossRefGoogle Scholar
Merrill, M. D. (2009) First principles of instruction. In Reigeluth, C. M. & Carr, A. (eds.), Instructional design theories and models: Building a common knowledge base (Vol. III). New York: Routledge Publishers.Google Scholar
Merrill, M. D. (2013) First principles of instruction: Identifying and designing effective, efficient, and engaging instruction. San Francisco: Pfeiffer.Google Scholar
Morrison, D. (2016) MOOCs desperately seeking quality. https://onlinelearninginsights.wordpress.com/2016/02/26/moocs-desperately-seeking-quality/ Google Scholar
Perifanou, M. (2014a) How to design and evaluate a massive open online course (MOOC) for language learning. Proceedings of the 10th International Scientific Conference on eLearning and Software for Education. Bucharest, Romania, 24–25 April.Google Scholar
Perifanou, M. & Economides, A. (2014) MOOCs for foreign language learning: An effort to explore and evaluate the first practices. INTED2014 Proceedings. Valencia, Span, 10–12 March.Google Scholar
Perifanou, M. A. (2014b) PLEs & MOOCs in language learning context: A challenging connection. PLE Conference 2014. Tallinn, Estonia, 16–18 July.Google Scholar
Poce, A., Amenduni, F., Re, M. R. & De Medio, C. (2019) Establishing a MOOC quality assurance framework: A case study. Open Praxis, 11(4): 451–460. https://doi.org/10.5944/openpraxis.11.4.1019 CrossRefGoogle Scholar
Pomerol, J.-C., Epelboin, Y. & Thoury, C. (2015) MOOCs: Design, use and business models. London: ISTE. http://doi.org/10.1002/9781119081364.ch3 Google Scholar
Qian, K. & Bax, S. (eds.) (2017) Beyond the language classroom: Researching MOOCs and other innovations. Dublin: Research-publishing.net. http://doi.org/10.14705/rpnet.2017.mooc2016.9781908416537 Google Scholar
Read, T. (2014) The architectonics of language MOOCs. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, 91–105. https://doi.org/10.2478/9783110422504.1 Google Scholar
Reichertz, J. (2007) Abduction: The logic of discovery of grounded theory. In Bryant, A. & Charmaz, K. (eds.), The SAGE handbook of grounded theory. London: SAGE, 214–228.Google Scholar
Richards, J. C. & Rodgers, T. S. (2001) Approaches and methods in language teaching (2nd ed.). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511667305.021 CrossRefGoogle Scholar
Shukor, N. A. & Abdullah, Z. (2019) Using learning analytics to improve MOOC instructional design. International Journal of Emerging Technologies in Learning, 14(24): 6–17. https://doi.org/10.3991/ijet.v14i24.12185 CrossRefGoogle Scholar
Sokolik, M. (2014) What constitutes an effective language MOOC? In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, 16–32. https://doi.org/10.2478/9783110422504.1 Google Scholar
Stickler, U. & Hampel, R. (2019) Qualitative research in online language learning: What can it do? International Journal of Computer-Assisted Language Learning and Teaching, 9(3): 14–28. https://doi.org/10.4018/IJCALLT.2019070102 CrossRefGoogle Scholar
Stracke, C. M., Tan, E., Teixeira, A. M., Pinto, M., Vassiliadis, B., Kameas, A. & Sgouropoulou, C. (2018) Gap between MOOC designers’ and MOOC learners’ perspectives on interaction and experiences in MOOCs: Findings from the Global MOOC Quality Survey. 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT). Mumbai, India, 9–13 July. https:doi.org/10.1109/ICALT.2018.00007 Google Scholar
Stracke, C. M. (2017) The quality of MOOCs: How to improve the design of open education and online courses for learners? In Zaphiris, P. & Ioannou, A. (eds.), Learning and collaboration technologies. Novel learning ecosystems: 4th International Conference, LCT 2017, held as part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings, Part I. Cham: Springer International, 285–293. https://doi.org/10.1007/978-3-319-58509-3_23 CrossRefGoogle Scholar
Teixeira, A. M. & Mota, J. (2014) A proposal for the methodological design of collaborative language MOOCs. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, 33–47. https://doi.org/10.2478/9783110420067.3 Google Scholar
The New Media Consortium (2013) NMC Horizon Project short list: Higher education edition. http://www.nmc.org/pdf/2013-horizon-higher-ed-shortlist.pdf Google Scholar
Tong, X. & Jia, X. (2017) The establishment of quality criteria framework of MOOCs. China’s Distance Education Journal, 5: 63–71.Google Scholar
Uvalić-Trumbić, S. & Daniel, J. (2013) Making sense of MOOCs: The evolution of online learning in higher education. In Hernández-Leo, D., Ley, T., Klamma, R. & Harrer, A. (eds), Scaling up learning for sustained impact: 8th European Conference, on Technology Enhanced Learning, EC-TEL 2013, Paphos, Cyprus, September 17–21, 2013: Proceedings. Berlin: Springer, 1–4. https://doi.org/10.1007/978-3-642-40814-4_1 Google Scholar
Wang, H. (2014) Reform in the teaching of college English writing in the big data era. Modern Distance Education Research, 3: 66–72, 86. https://doi.org/10.3969/j.issn.1009-5195.2014.03.008 Google Scholar
Wang, R., Zhao, C. & Wan, L. (2017) The establishment of quality criteria framework for open online courses based on grounded theory. China’s Distance Education Journal, 11: 70–76. https://doi.org/10.13541/j.cnki.chinade.20171120.002 Google Scholar
Watson, W. R., Watson, S. L. & Janakiraman, S. (2017) Instructional quality of massive open online courses: A review of attitudinal change MOOCs. International Journal of Learning Technology, 12(3): 219–240. https://doi.org/10.1504/IJLT.2017.088406 CrossRefGoogle Scholar
Yousef, A. M. F., Chatti, M. A., Schroeder, U. & Wosnitza, M. (2014) What drives a successful MOOC? An empirical examination of criteria to assure design quality of MOOCs. In 14th IEEE International Conference on Advanced Learning Technologies (ICALT 2014), Athens, Greece, 7–9 July 2014. Los Alamitos: Conference Publishing Services/IEEE Computer Society, 44–48. https://doi.org/10.1109/ICALT.2014.23 CrossRefGoogle Scholar
Yuan, L. & Powell, S. (2013) MOOCs and open education: Implications for higher education. https://doi.org/10.13140/2.1.5072.8320 Google Scholar
Supplementary material: File

Luo and Ye supplementary material

Luo and Ye supplementary material 1

Download Luo and Ye supplementary material(File)
File 490 Bytes
Supplementary material: File

Luo and Ye supplementary material

Luo and Ye supplementary material 2

Download Luo and Ye supplementary material(File)
File 18.4 KB