This chapter reports on one aspect of the argument-based validation research conducted to evaluate the interpretation and use of scores from the Oral English Certification Test (OECT). The test of English speaking ability for prospective international teaching assistants (ITAs) was updated by introducing a new a web-based rating system, called Rater-Platform (R-PLAT). R-PLAT was intended to improve the efficiency of the rating process, but research was needed to investigate its effects on all aspects of the interpretation/use argument (Kane, 2013). The study investigated the warrant underlying the evaluation inference: the observed performance on the OECT recorded via R-PLAT provides observed scores and observed performance descriptors reflective of targeted speaking ability. The assumption in need of support was that the quality of rating conditions created by R-PLAT was sufficient for gathering accurate scores. Backing was found through analysis of raters’ perceptions towards and their use of R-PLAT collected through questionnaires and interviews. This chapter concludes with the validity argument showing how evidence collected from this study supported the assumptions underlying the evaluation inference. It suggests future research needed to build the complete validity argument for the OECT with R-PLAT, and potential use of a web-based rating system for other speaking tests.

Research examining how language testing organizations measure fluency, design their fluency rating scales and develop their fluency descriptors is limited; the complexities involved in human rating of fluency make it difficult to assess fluency in any kind of multidimensional way. This chapter highlights the importance of assessing fluency objectively, accurately and consistently, in order to increase validity; also, we argue that developing a research-evidenced approach to assessing fluency in L2 tests of speaking is needed, particularly in taking a more dynamic, task-based approach into consideration. After examining the existing fluency descriptors and rating scales in some international tests of speaking, the chapter reports recent research investigating fluency across different levels of proficiency. It will also discuss in what ways a broader theoretical perspective to assessment of fluency, e.g. using conversation analysis techniques, should be considered when validating assessment of fluency. Effects of raters, rating scales and rating descriptors on judgements of fluency will also be discussed, and merits and limitations of automated assessment of fluency will be evaluated, relating the implications of important developments for practice in language testing and for future research.