6 - Validity of criterion-referenced tests

Summary

Introduction

Traditionally, validity has been defined as “the degree to which a test measures what it claims, or purports, to be measuring” (Brown, 1996, p. 231). For example, if a criterion-referenced test is designed to measure the objectives of a specific course, the validity of the test could be defended by showing that the test is indeed measuring those objectives. Naturally, making such arguments from a number of perspectives will be more convincing to you and other users of the tests. The perspectives that have traditionally been most useful for studying the validity of criterion-referenced tests are content validity and construct validity. Each of these validity strategies will be defined and discussed in turn in the sections that follow. In addition, ample examples will be provided to help illustrate how these concepts are applied in practice and how they can be combined.

Recently, expanded views of what validity ought to include have begun to surface in the educational measurement literature. Chief among the proponents of such expanded views are Messick and Cronbach. Messick (1988) discusses the evidential and consequential bases of test interpretation and use, and Cronbach (1988) covers the functional, political, economic, and explanatory perspectives on test validity. We feel that the considerations suggested by Messick and Cronbach may be especially important for criterion-referenced tests. This is particularly true to the extent that validity is viewed as being concerned with the inferences that are made on the basis of test scores, rather than being any intrinsic quality of the particular test form itself.