Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-27T08:34:01.518Z Has data issue: false hasContentIssue false
  • English
  • Français

Innovative assessment of aviation English in a virtual world: Windows into cognitive and metacognitive strategies

Published online by Cambridge University Press:  24 January 2018

Moonyoung Park Open the ORCID record for Moonyoung Park [Opens in a new window]
Moonyoung Park*
Affiliation:
Chinese University of Hong Kong, Hong Kong SAR, China (email: moonyoungpark@cuhk.edu.hk)
Get access Rights & Permissions [Opens in a new window]

Abstract

Aviation English proficiency is a core competency in the global air traffic controller profession. There is, however, growing concern about the current ineffective paper-based assessment methods and the severe lack of interactive online testing for such a critical profession, one that should be ideally assessed in an authentic task and situation (Alderson, 2010; Douglas, 2013). The tests, which lack validity and authenticity, cannot capture how knowledge of aviation English communication and strategic competence are used through valid means and, therefore, inevitably fail to predict how test takers actually perform in the target language use situations (Douglas, 2000). In the present study, the researcher examines the potential use of verbal report data produced by test takers interacting in a virtual testing environment. The research seeks to answer two specific questions: (a) What types of strategies are used in the virtual aviation English task performance? and (b) How can the assessed strategies be interpreted in relation to test takers’ performance? The analysis of the test takers’ verbal reports from stimulated recalls indicates that various cognitive, metacognitive, and communication strategies were used while performing the Virtual Interactive Tasks for Aviation English Assessment, and that there is a positive relationship between the total number of cognitive and metacognitive strategies adopted and the test scores. The findings suggest that the use of an immersive interface and simulated tasks in a virtual world could provide language learners with more authentic opportunities to perform the target tasks and promote strategic, as well as linguistic, competence.

Keywords

aviation Englishlanguage testingSecond Lifecognitive strategymetacognitive strategycommunication strategy
Type
Regular papers
Information
ReCALL , Volume 30 , Special Issue 2: Interactions for language learning in and around virtual worlds , May 2018 , pp. 196 - 213
Check for updates
Copyright
Copyright © European Association for Computer Assisted Language Learning 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abdallah, M. M. S. and Mansour, M. M. (2015) Virtual task-based situated language-learning with Second Life: Developing EFL pragmatic writing and technological self-efficacy. Arab World English . Journal, 2: 150–182.Google Scholar
Alderson, J. C. (2006) Final report on a survey of aviation English tests. Lancaster: Lancaster University and the European Organisation for the Safety of Air Navigation (Eurocontrol).Google Scholar
Alderson, J. C. (2009) Air safety, language assessment policy, and policy implementation: The case of aviation English. Annual Review of Applied Linguistics, 29: 168–187. https://doi.org/10.1017/S0267190509090138 Google Scholar
Alderson, J. C. (2010) A survey of aviation English tests. Language Testing, 27(1): 51–72. https://doi.org/10.1177/0265532209347196 CrossRefGoogle Scholar
Alderson, J. C., Clapham, C. and Wall, D. (1995) Language test construction and evaluation. Cambridge: Cambridge University Press.Google Scholar
Bachman, L. F. and Palmer, A. S. (1996) Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.Google Scholar
Bachman, L. F. and Palmer, A. S. (2010) Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: Oxford University Press.Google Scholar
Berns, A., Gonzalez-Pardo, A. and Camacho, D. (2013) Game-like language learning in 3-D virtual environments. Computers & Education, 60(1): 210–220. https://doi.org/10.1016/j.compedu.2012.07.001 CrossRefGoogle Scholar
Brusso, R. C., Wisher, R. A., Paddock, A. and Hatfield, J. (2014) Best practices and provisional guidelines for integrating mobile, virtual, and videogame-based training and assessments. Fairfax: ICF International. https://doi.org/10.21236/ADA596660 Google Scholar
Can, T. and Simsek, I. (2015) The use of 3D virtual learning environments in training foreign language pre-service teachers. Turkish Online Journal of Distance Education, 16(4): 114–124.Google Scholar
Canto, S., Jauregi, K. and van den Bergh, H. (2013) Integrating cross-cultural interaction through video-communication and virtual worlds in foreign language teaching programs: Is there an added value? ReCALL, 25(1): 105–121. https://doi.org/10.1017/S0958344012000274 CrossRefGoogle Scholar
Chamot, A. U. and O’Malley, J. M. (1987) The cognitive academic language learning approach: A bridge to the mainstream. TESOL Quarterly, 21(2): 227–249. https://doi.org/10.2307/3586733 Google Scholar
Chapelle, C. A. (1998) Construct definition and validity inquiry in SLA research. In Bachman, L. F. & Cohen, A. D. (eds.), Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press, 32–70.Google Scholar
Chapelle, C. A. (2005) Interactionist SLA theory in CALL research. In Egbert, J. L. & Petrie, G. M. (eds.), CALL research perspectives. Mahwah, NJ: Lawrence Erlbaum, 53–64.Google Scholar
Chuang, M.-T., Chang, B. and Chen, H.-C. (2014) Investigating reported social and affective strategy use by EFL learners in virtual and real learning environments. International Journal of Applied Linguistics & English Literature, 3(2): 27–33. https://doi.org/10.7575/aiac.ijalel.v.3n.2p.27 Google Scholar
Clarke-Midura, J. and Dede, C. (2010) Assessment, technology, and change. Journal of Research on Technology in Education, 42(3): 309–328. https://doi.org/10.1080/15391523.2010.10782553 CrossRefGoogle Scholar
Corbin, J. and Strauss, A. (2008) Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks: Sage. https://doi.org/10.4135/9781452230153 Google Scholar
Creswell, J. W. and Plano Clark, V. L. (2007) Designing and conducting mixed methods research. Thousand Oaks: Sage.Google Scholar
Crystal, D. (2003) English as a global language (2nd ed.). New York: Cambridge University Press.Google Scholar
Cushing, S. (1994) Fatal words: Communication clashes and aircraft crashes. Chicago: University of Chicago Press.Google Scholar
da Silva, K. T. F. (2012) Preparing language teachers to teach in virtual worlds: Analyzing their content, technological, and pedagogical needs. Iowa State University, unpublished PhD.Google Scholar
Dörnyei, Z. and Scott, M. L. (1997) Communication strategies in a second language: Definitions and taxonomies. Language Learning, 47(1): 173–210. https://doi.org/10.1111/0023-8333.51997005 Google Scholar
Douglas, D. (2000) Assessing languages for specific purposes. Cambridge: Cambridge University Press.Google Scholar
Douglas, D. (2001) Language for specific purposes assessment criteria: Where do they come from? Language Testing, 18(2): 171–185. https://doi.org/10.1191/026553201678777068 CrossRefGoogle Scholar
Douglas, D. (2013) ESP and assessment. In Paltridge, B. & Starfield, S. (eds.), The handbook of English for specific purposes. Malden: John Wiley & Sons, 367–383.Google Scholar
Hammick, J. K. and Lee, M. J. (2014) Do shy people feel less communication apprehension online? The effects of virtual reality on the relationship between personality characteristics and communication outcomes. Computers in Human Behavior, 33: 302–310. https://doi.org/10.1016/j.chb.2013.01.046 CrossRefGoogle Scholar
Holliday, L. (2007) Theory and research: Interaction and awareness. In Egbert, J. & Hanson-Smith, E. (eds.), CALL environments: Research, practice, and critical issues (2nd ed.). Alexandria, VA: Teachers of English to Speakers of Other Languages, 163–170.Google Scholar
International Civil Aviation Organization (ICAO) (2004) Manual on the implementation of ICAO language proficiency requirements (ICAO Doc 9835). Chicago: Author.Google Scholar
Khalsa, D. K., Maloney-Krichmar, D. and Peyton, J. K. (2007) Theory and research: Interaction via computers. In Egbert, J. & Hanson-Smith, E. (eds.), CALL environments: Research, practice, and critical (2nd ed.). Alexandria, VA: Teachers of English to Speakers of Other Languages, 19–31.Google Scholar
Lai, E. R. (2011) Metacognition: A literature review: Research report. Upper Saddle River: Pearson Education.Google Scholar
Lin, T.-J., Wang, S.-Y., Grant, S., Chien, C.-L. and Lan, Y.-J. (2014) Task-based teaching approaches of Chinese as a foreign language in Second Life through teachers’ perspectives. Procedia Technology, 13: 16–22. https://doi.org/10.1016/j.protcy.2014.02.004 Google Scholar
Long, M. H. (2005) Methodological issues in learner needs analysis. In Long, M. H. (ed.), Second language needs analysis. Cambridge: Cambridge University Press, 19–76. https://doi.org/10.1017/CBO9780511667299.002 Google Scholar
Melchor-Couto, S. (2017) Foreign language anxiety levels in Second Life oral interaction. ReCALL, 29(1): 99–119. https://doi.org/10.1017/S0958344016000185 Google Scholar
Mell, J. (1992) Study of verbal communication between pilot and air traffic controller in standard and non-standard situations. Ecole Nationale de l’Aviation Civile, unpublished PhD.Google Scholar
Messick, S. (1989) Validity. In Linn, R. L. (ed.), Educational measurement (3rd ed.). New York: Macmillan, 13–103.Google Scholar
Oliver, M. and Carr, D. (2009) Learning in virtual worlds: Using communities of practice to explain how people learn from play. British Journal of Educational Technology, 40(3): 444–457. https://doi.org/10.1111/j.1467-8535.2009.00948.x Google Scholar
Paltridge, B. and Phakiti, A. (2015) Research methods in applied linguistics: A practical resource. London: Bloomsbury.Google Scholar
Peterson, M. (2012) EFL learner collaborative interaction in Second Life. ReCALL, 24(1): 20–39. https://doi.org/10.1017/S0958344011000279 Google Scholar
Purpura, J. E. (1999) Learner strategy use and performance on language tests: A structural equation modeling approach. Cambridge: Cambridge University Press.Google Scholar
Sadler, R. (2011) Virtual Worlds for language learning: From theory to practice. Bern: Peter Lang.Google Scholar
Sadler, R. (2012) Virtual worlds: An overview and pedagogical examination. Bellaterra Journal of Teaching & Learning Language and Literature, 5(1): 1–22.CrossRefGoogle Scholar
Schwienhorst, K. (2002) Why virtual, why environments? Implementing virtual reality concepts in computer-assisted language learning. Simulation & Gaming, 33(2): 196–209. https://doi.org/10.1177/1046878102332008 Google Scholar
Stevenson, D. K. (1985) Authenticity, validity and a tea party. Language Testing, 2(1): 41–47. https://doi.org/10.1177/026553228500200105 Google Scholar
Swier, R. (2014) Tasks for easily modifiable virtual environments. The JALT CALL Journal, 10(3): 203–219.Google Scholar
Tajima, A. (2004) Fatal miscommunication: English in aviation safety. World Englishes, 23(3): 451–470. https://doi.org/10.1111/j.0883-2919.2004.00368.x Google Scholar
Tseng, J.-J., Tsai, Y.-H. and Chao, R.-C. (2013) Enhancing L2 interaction in avatar-based virtual worlds: Student teachers’ perceptions. Australasian Journal of Educational Technology, 29(3): 357–371. https://doi.org/10.14742/ajet.283 Google Scholar
Van Moere, A., Suzuki, M., Downey, R. and Cheng, J. (2009) Implementing ICAO language proficiency requirements in the Versant Aviation English Test. Australian Review of Applied Linguistics, 32(3): 27.1–27.17.Google Scholar
Verhagen, T., Feldberg, F., van den Hooff, B., Meents, S. and Merikivi, J. (2012) Understanding users’ motivations to engage in virtual worlds: A multipurpose model and empirical testing. Computers in Human Behavior, 28(2): 484–495. https://doi.org/10.1016/j.chb.2011.10.020 CrossRefGoogle Scholar
Vuopala, E., Hyvönen, P. and Järvelä, S. (2016) Interaction forms in successful collaborative learning in virtual learning environments. Active Learning in Higher Education, 17(1): 25–38. https://doi.org/10.1177/1469787415616730 Google Scholar
Wang, C. X., Calandra, B., Hibbard, S. T. and Lefaiver, M. L. M. (2012) Learning effects of an experimental EFL program in Second Life. Educational Technology Research and Development, 60(5): 943–961. https://doi.org/10.1007/s11423-012-9259-0 Google Scholar
Zokić, M., Boras, D. and Lazić, N. (2012) Say again. International Journal of Education and Information Technologies, 6(4): 363–370.Google Scholar