12 - Applying coupon-collecting theory to computer-aided assessments

Summary

Abstract

Computer-based tests with randomly generated questions allow a large number of different tests to be generated. Given a fixed number of alternatives for each question, the number of tests that need to be generated before all possible questions have appeared is surprisingly low.

AMS subject classification (MSC2010) 60G70, 60K99

Introduction

The use of computer-based tests in which questions are randomly generated in some way provides a means whereby a large number of different tests can be generated; many universities currently use such tests as part of the student assessment process. In this paper we present findings that illustrate that, although the number of different possible tests is high and grows very rapidly as the number of alternatives for each question increases, the average number of tests that need to be generated before all possible questions have appeared at least once is surprisingly low. We presented preliminary findings along these lines in Cornish et al. (2006).

A computer-based test consists of q questions, each (independently) selected at random from a separate bank of a alternatives. Let N_q be the number of tests one needs to generate in order to see all the aq questions in the q question banks at least once. We are interested in how, for fixed a, the random variable N_q grows with the number of questions q in the test.