3 - Criterion-referenced test items

Summary

Introduction

In his review of Roid and Haladyna (1982), Nitko (1984) notes that:

Although techniques for creating test items are traditionally covered in measurement textbooks, much of what novice item writers read in these textbooks is “clinical lore”: Elder item-writers pass down to novices lists of rules and suggestions which they and their item-writing forefathers have learned through the process of applied art, empirical study and practical experience,

(p. 201)

Without accepting the mantle of “elder item-writers”, we would venture to propose that a set of general rules, which can be modified to help in writing good tests, has already been suggested for the area of discourse interpretation. These rules, called Grice's Maxims for the Cooperative Principle of Discourse (Grice, 1975), are related to quantity, quality, relation, and manner of general discourse, but they can also be used to cover the cooperative nature of test writing. The maxims can be summarized as follows:

1. Make your contribution as informative as is required but not more informative than required.

2. Be relevant.

3. Avoid ambiguity and obscurity of expression.

4. Be orderly.

Therefore, in writing test items and instructions: do not explain too much; do not use trick questions; provide only the information necessary; avoid ambiguity; and be orderly in test presentation. Remember that the examinees will assume that you are presenting them with all, and only, the relevant information.