Hostname: page-component-5f745c7db-q8b2h Total loading time: 0 Render date: 2025-01-06T21:45:22.001Z Has data issue: true hasContentIssue false
  • English
  • Français

Ordinal Consistency and Ordinal True Scores

Published online by Cambridge University Press:  01 January 2025

Norman Cliff
Norman Cliff*
Affiliation:
University of Southern California
*
Requests for reprints should be sent to Norman Cliff, Department of Psychology, MC 1061, University of Southern California, Los Angeles, CA 90089-1061.
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.

Keywords

test theoryordinal regressionlocal independencereliability
Type
Original Paper
Information
Psychometrika , Volume 54 , Issue 1 , March 1989 , pp. 75 - 91
Copyright
Copyright © 1989 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Part of this work was done while the author was a Visiting Fellow at Macquarrie University. The paper has benefitted from discussions with Professors Thomas J. Reynolds and Roderick P. McDonald and from the comments of several anonymous reviewers.

References

Andersen, E. B. (1973). Conditional inference for multiple choice questionnaires. British Journal of Mathematical and Statistical Psychology, 26, 42–54.CrossRefGoogle Scholar
Bartholomew, D. J. (1984). The foundations of factor analysis. Biometrika, 71, 221–232.CrossRefGoogle Scholar
Bartholomew, D. J. (1985). Foundations of factor analysis: Some practical implications. British Journal of Mathematical and Statistical Psychology, 38, 127–137.CrossRefGoogle Scholar
Birnbaum, A. (1968). Some latent trait models and their uses in inferring an examinee's ability. In Lord, F. M., Novick, M. R. (Eds.), Statistical theories of mental test scores (pp. 397–472). Reading, MA: Addison, Wesley.Google Scholar
Bock, R. D., Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Cliff, N. (1977). A theory of consistency of ordering generalizable to tailored testing. Psychometrika, 42, 373–393.CrossRefGoogle Scholar
Cliff, N. (1979). Test theory without true scores?. Psychometrika, 44, 373–393.CrossRefGoogle Scholar
Douglas, G. A., Wright, B. D. (1968). The two category model for objective measurment, Chicago: University of Chicago.Google Scholar
Gans, L. P., Robertson, C. A. (1981). Distributions of Goodman and Kruskal's gamma and Spearman's rho in 2 × 2 tables for small and moderate sample sizes. Journal of the American Statistical Association, 76, 942–946.Google Scholar
Goodman, L., Kruskal, W. B. (1955). Measures of association for crossclassifications. Journal of the American Statistical Association, 49, 732–764.Google Scholar
Green, B. F. (1976). Comments on computers in testing. In Clark, C. L. (Eds.), Proceedings of the First Conference on Computerized Adaptive Testing, Washington, DC: U.S. Government Printing Office.Google Scholar
Guttman, L. (1950). The basis for scalogram analysis. In Stouffer, S. A. (Eds.), Measurement and prediction, Princeton, NJ: Princeton University.Google Scholar
Guttman, L. (1971). Measurement as structural theory. Psychometrika, 36, 329–347.CrossRefGoogle Scholar
Hubert, L. J. (1985). Combinatorial data analysis: Association and partial association. Psychometrika, 59, 449–467.CrossRefGoogle Scholar
Humphreys, L. G. (1956). The normal curve and the attenuation paradox in test theory. Psychological Bulletin, 53, 472–476.CrossRefGoogle ScholarPubMed
Keats, J. A. (1951). A statistical theory of mental test scores, Melbourne: Australian Council for Educational Research.Google Scholar
Kendall, M. G. (1970). Rank correlation methods 3rd ed.,, New York: Hafner.Google Scholar
Krantz, D. H., Luce, R. D., Suppes, P., Tversky, A. (1971). Foundations of measurement, New York: Academic Press.Google Scholar
Lazarsfeld, P. F. (1950). The logical and mathematical foundation of latent structural analysis. In Stouffer, S. A. (Eds.), Measurement and prediction (pp. 362–412). Princeton, NJ: Princeton University.Google Scholar
Lazarsfeld, P. F. (1950). The interpretation and computation of some latent structures. In Stouffer, S. A. (Eds.), Measurement and prediction (pp. 413–473). Princeton, NJ: Princeton University.Google Scholar
Lazarsfeld, P. F., Henry, N. W. (1968). Latent structure analysis, New York: Houghton Mifflin.Google Scholar
Levine, M. V. (1970). Transformations that render curves parallel. Journal of Mathematical Psychology, 7, 410–443.CrossRefGoogle Scholar
Loevinger, J. A. (1947). A systematic approach to the construction and evaluation of tests of ability. Psychological Monographs, 61, (4, Whole No. 285).CrossRefGoogle Scholar
Loevinger, J. A. (1948). A technique of homogeneous test evaluation compared with some aspects of “scale analysis”. Psychological Bulletin, 45, 507–529.CrossRefGoogle Scholar
Lord, F. M., Novick, M. R. (1968). Statistical theories of mental test scores, Reading, MA: Addison, Wesley.Google Scholar
Luce, R. D., Tukey, J. W. (1964). Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of Mathematical Psychology, 1, 1–27.CrossRefGoogle Scholar
Mokken, R. J. (1971). A theory and procedure of scale analysis, Hawthorne, NY: Mouton & Co.CrossRefGoogle Scholar
Mokken, R. J., Lewis, C. (1982). A nonparametric approach to the analysis of dichotomous item responses. Applied Psychological Measurement, 6, 417–430.CrossRefGoogle Scholar
Mokken, R. J., Lewis, C., Sÿtsma, K. (1986). Rejoinder to: “The Mokken Scale: A critical discussion.”. Applied Psychological Measurement, 10, 229–285.CrossRefGoogle Scholar
Novick, M. R., Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1–4.CrossRefGoogle ScholarPubMed
Puri, M. L., Sen, P. K. (1985). Nonparametric methods in general linear models, New York: Wiley.Google Scholar
Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests, Chicago: University of Chicago Press (Reprint, with a Foreword and Afterword by Benjamin D. Wright)Google Scholar
Reynolds, T. J., Suttrick, K. H. (1986). Assessing the correspondence of a vector to a symmetric matrix using ordinal regression. Psychometrika, 51, 101–112.CrossRefGoogle Scholar
Roskam, E. E., Jansen, P. G. W. (1984). A new derivation of the Rasch model. In Degreef, E., van Buggenhaut, J. (Eds.), Trends in mathematical psychology (pp. 293–307). Amsterdam: Elsevier Science Publishers.CrossRefGoogle Scholar
Roskam, E. E., van den Wollenberg, A. L., Jansen, P. G. W. (1986). The Mokken Scale: A critical discussion. Applied Psychological Measurement, 10, 267–277.CrossRefGoogle Scholar
Schulman, R. S. (1976). Correlation and prediction in ordinal test theory. Psychometrika, 41, 329–340.CrossRefGoogle Scholar
Schulman, R. S., Haden, R. L. (1975). A test theory for ordinal measurements. Psychometrika, 40, 455–472.CrossRefGoogle Scholar
Shepard, R. N. (1962). The analysis of proximities: Multidimensional scaling with an unknown distance function. Psychometrika, 27, 219–246.CrossRefGoogle Scholar
Smith, R. B. (1974). Continuities in ordinal path analysis. Social forces, 53, 229–246.CrossRefGoogle Scholar
Somers, R. H. (1962). A new asymmetric measure of association for ordinal variables. American Sociological Review, 27, 799–811.CrossRefGoogle Scholar