Hostname: page-component-5f745c7db-hj587 Total loading time: 0 Render date: 2025-01-06T07:21:18.982Z Has data issue: true hasContentIssue false
  • English
  • Français

A Compensatory Approach to Optimal Selection with Mastery Scores

Published online by Cambridge University Press:  01 January 2025

Wim J. van der Linden  and
Hans J. Vos
Wim J. van der Linden*
Affiliation:
University of Twente
Hans J. Vos
Affiliation:
University of Twente
*
Requests for reprints should be sent to Wire J. van der Linden, Department of Educational Measurement and Data Analysis, University of Twente, PO Box 217, 7500 AE Enschede, THE NETHERLANDS.
Get access Rights & Permissions [Opens in a new window]

Abstract

A Bayesian approach for simultaneous optimization of test-based decisions is presented using the example of a selection decision for a treatment followed by a mastery decision. A distinction is made between weak and strong rules where, as opposed to strong rules, weak rules use prior test scores as collateral data. Conditions for monotonicity of optimal weak and strong rules are presented. It is shown that under mild conditions on the test score distributions and utility functions, weak rules are always compensatory by nature.

Keywords

decision theorymastery testingmonotone Bayes rulesselection decisions
Type
Original Paper
Information
Psychometrika , Volume 61 , Issue 1 , March 1996 , pp. 155 - 172
Copyright
© 1996 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors are indebted to Wilbert Kallenberg for his valuable comments and to Jan Gulmans for providing the data for the empirical example. The names of the authors are alphabetical; they are equally responsible for the contents of this paper.

References

Brouwer, U., Vijn, P. (1979). Bayesiaanse schatter voor de correlatiecoëfficiënt in “restriction of range” (geval 1) [Bayesian estimators for the correlation coefficient with restriction of range (Case 1)]. Tijdschrift voor Onderwijsresearch, 4, 281–290.Google Scholar
Chuang, D. T., Chen, J. J., Novick, M. R. (1981). Theory and practice for the use of cut-scores for personnel decisions. Journal of Educational Statistics, 6, 129–152.CrossRefGoogle Scholar
Cronbach, L. J., Gleser, G. C. (1965). Psychological tests and personnel decisions 2nd ed.,, Urbana, IL: University of Illinois Press.Google Scholar
Davis, C. E., Hickman, J., Novick, M. R. (1973). A primer on decision analysis for individually prescribed instruction, Iowa City, IA: American Testing College Program.Google Scholar
de Gruijter, D. N. M., Hambleton, R. K. (1984). On problems encountered using decision theory to set cutoff scores. Applied Psychological Measurement, 8, 1–8.CrossRefGoogle Scholar
de Gruijter, D. N. M., Hambleton, R. K. (1984). Reply to van der Linden's “Thoughts on the use of decision theory to set cutoff scores”. Applied Psychological Measurement, 8, 19–20.Google Scholar
Ferguson, T. S. (1967). Mathematical statistics: A decision theoretic approach, New York: Academic Press.Google Scholar
Gross, A. L., Su, W. H. (1975). Defining a “fair” or “unbiased” selection model: A question of utilities. Journal of Applied Psychology, 60, 345–351.CrossRefGoogle Scholar
Hambleton, R. K., Novick, M. R. (1973). Toward an integration of theory and method for criterion-referenced tests. Journal of Educational Measurement, 10, 159–170.CrossRefGoogle Scholar
Huynh, H. (1976). Statistical considerations of mastery scores. Psychometrika, 41, 65–79.CrossRefGoogle Scholar
Huynh, H. (1977). Two simple classes of mastery scores based on the beta-binomial model. Psychometrika, 42, 601–608.CrossRefGoogle Scholar
Huynh, H. (1980). A nonrandomized minimax solution for passing scores in the binomial error model. Psychometrika, 45, 167–182.CrossRefGoogle Scholar
Huynh, H. (1982). A Bayesian procedure for mastery decisions based on multivariate normal test data. Psychometrika, 47, 309–319.CrossRefGoogle Scholar
Huynh, H., Perney, J. C. (1979). Determination of mastery scores when instructional units are hierarchically related. Educational and Psychological Measurement, 39, 317–323.CrossRefGoogle Scholar
Keeney, R. L., Raiffa, H. (1976). Decisions with multiple objectives: Preferences and value tradeoffs, New York: Wiley.Google Scholar
Kendall, M., Stuart, M. A. (1979). The advanced theory of statistics. Volume 2: Inference and relationship, London: Charles Griffin & Co..Google Scholar
Lehmann, E. L. (1986). Testing statistical hypotheses 2nd ed.,, New York: John Wiley and Sons.CrossRefGoogle Scholar
Lindgren, B. W. (1976). Statistical theory 3rd ed.,, New York: Macmillan.Google Scholar
Lord, F. M., Novick, M. R. (1986). Statistical theories of mental test scores, Reading, MA: Addison-Wesley.Google Scholar
Mellenbergh, G. J., van der Linden, W. J. (1981). The linear utility model for optimal selection. Psychometrika, 46, 283–293.CrossRefGoogle Scholar
Novick, M. R., Lindley, D. V. (1979). Fixed-state assessment of utility functions. Journal of the American Statistical Association, 74, 306–311.Google Scholar
Novick, M. R., Petersen, N. S. (1976). Towards equalizing educational and employment opportunity. Journal of Educational Psychology, 13, 77–88.Google Scholar
Roe, R. A. (1979). The correction for restriction of range and the difference between intended and actual selection. Educational and Psychological Measurement, 39, 551–560.CrossRefGoogle Scholar
Sawyer, R. (1993, April). Decision theoretic models for validating course placement systems. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.Google Scholar
Taylor, H. C., Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection discussion and tables. Journal of Applied Psychology, 23, 565–578.CrossRefGoogle Scholar
van der Gaag, N. (1990). Empirische utiliteiten voor psychometrische beslissingen (Empirical utilities for psychometric decisions), The Netherlands: University of Amsterdam.Google Scholar
van der Linden, W. J. (1981). Using aptitude measurements for the optimal assignment of subjects to treatments with and without mastery scores. Psychometrika, 46, 257–274.CrossRefGoogle Scholar
van der Linden, W. J. (1984). Some thoughts on the use of decision theory to set cutoff scores: Comment on de Gruijter and Hambleton. Applied Psychological Measurement, 8, 9–18.CrossRefGoogle Scholar
van der Linden, W. J. (1987). The use of test scores for classification decisions with threshold utility. Journal of Educational Statistics, 12, 62–75.CrossRefGoogle Scholar
van der Linden, W. J. (1990). Applications of decision theory to test-based decision making. In Hambleton, R. K., Zaal, J. N. (Eds.), New developments in testing: Theory and applications (pp. 129–155). Boston: Kluwer.Google Scholar
van der Linden, W. J. (1995). Some decision theory for course placement. Manuscript submitted for publication.Google Scholar
van der Linden, W. J., Mellenbergh, G. J. (1977). Optimal cutting scores using a linear loss function. Applied Psychological Measurement, 1, 593–599.CrossRefGoogle Scholar
Vos, H. J. (1990). Simultaneous optimization of decisions using a linear utility function. Journal of Educational Statistics, 15, 309–340.CrossRefGoogle Scholar
Vos, H. J. (1991). Simultaneous optimization of the aptitude treatment interaction decision problem with mastery scores. In Wilson, M. (Eds.), Objective measurement: Theory into practice (pp. 313–331). Norwood, NJ: Ablex.Google Scholar
Vos, H. J. et al. (1993). Simultaneous optimization of selection-placement decisions followed by a mastery decision. In Maarse, F. J. et al. (Eds.), Computers in psychology: Tools for experimental and applied psychology (pp. 235–244). Lisse, The Netherlands: Swets & Zeitlinger.Google Scholar
Vos, H. J. (1994). Simultaneous optimization of test-based decisions in education, The Netherlands: University of Twente.Google Scholar
Vrijhof, B. J., Mellenbergh, G. J., van den Brink, W. P. (1983). Assessing and studying utility functions in psychometric decision theory. Applied Psychological Measurement, 7, 341–357.CrossRefGoogle Scholar