Hostname: page-component-745bb68f8f-s22k5 Total loading time: 0 Render date: 2025-01-08T16:45:22.177Z Has data issue: false hasContentIssue false
  • English
  • Français

Psychometric Engineering as Art

Published online by Cambridge University Press:  01 January 2025

David Thissen
David Thissen*
Affiliation:
L. L. Thurstone Psychometric Laboratory, University of North Carolina at Chapel Hill
*
Requests for reprints should be sent to David Thissen, L.L. Thurstone Psychometric Laboratory, CB #3270, Davie Hall, University of North Carolina, Chapel Hill NC 27599-3270. E-Mail: dthissen@email.unc.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

The Psychometric Society is “devoted to the development of Psychology as a quantitative rational science”. Engineering is often set in contradistinction with science; art is sometimes considered different from science. Why, then, juxtapose the words in the title:psychometric, engineering, and art? Because an important aspect of quantitative psychology is problem-solving, and engineering solves problems. And an essential aspect of a good solution is beauty—hence, art. In overview and with examples, this presentation describes activities that are quantitative psychology as engineering and art—that is, as design. Extended illustrations involve systems for scoring tests in realistic contexts. Allusions are made to other examples that extend the conception of quantitative psychology as engineering and art across a wider range of psychometric activities.

Keywords

psychometricsquantitative psychologydesign
Type
Articles
Information
Psychometrika , Volume 66 , Issue 4 , December 2001 , pp. 473 - 485
Copyright
Copyright © 2001 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This article is based on the Presidential Address David Thissen gave at the 66th Annual Meeting of the Psychometric Society held in King of Prussia, Pennsylvania on June 24, 2001. The address was also given on July 16 at the 2001 International Meeting of the Psychometric Society held in Osaka, Japan.—Editor

Thanks to R. Darrell Bock, Paul De Boeck, Lyle V. Jones, Cynthia Null, Lynne Steinberg, and Howard Wainer for constructive comments on early drafts of this manuscript. And thanks to Val Williams, Mary Pommerich, Lee Chen, Kathleen Rosa, Lauren Nelson, Maria Orlando, Kimberly Swygert, Lori McLeod, Bryce Reeve, Fabian Camacho, David Flora, Viji Sathy, Michael Edwards, and Jack Vevea for their many contributions to some of the research that illustrates this commentary. Of course, any flaws in the argument or its presentation remain the author's.

References

Allen, N.L., Carlson, J.E., & Zelenak, C.A. (1999). The NAEP 1996 technical report. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.Google Scholar
Baker, F.B., & Harwell, M.R. (2001). Computing elementary symmetric functions and their derivatives: A didactic. Applied Psychological Measurement, 20(2), 169–192.CrossRefGoogle Scholar
Barr, A.H. (1946). Picasso: Fifty years of his art. New York, NY: The Museum of Modern Art.Google Scholar
Berkson, J. (1944). Application of the logistic function to bio-assay. Journal of the American Statistical Association, 39, 357–375.Google Scholar
Berkson, J. (1953). A statistically precise and relatively simple method of estimating the bio-assay with quantal response, based on the logistic function. Journal of the American Statistical Association, 48, 565–599.Google Scholar
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In Lord, F.M., Novick, M.R. (Eds.), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of the EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Bock, R.D., & Lieberman, M. (1970). Fitting a response model forn dichotomously scored items. Psychometrika, 35, 179–197.CrossRefGoogle Scholar
Bock, R.D., & Mislevy, R.J. (1981). An item response curve model for matrix-sampling data: The California grade-three assessment. New Directions for Testing and Measurement, 10, 65–90.Google Scholar
Bock, R.D., & Mislevy, R.J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431–444.CrossRefGoogle Scholar
Box, G.E.P. (1979). Some problems of statistics and everday life. Journal of the American Statistical Association, 74, 1–4.CrossRefGoogle Scholar
Brooks, F.P. (2001). The computer scientist as toolsmith II. Communications of the ACM, 39, 61–68.CrossRefGoogle Scholar
Brooks, F.P. (in press). The design of design. Communications of the ACM.Google Scholar
Chen, W.H. (1995). Estimation of item parameters for the three-parameter logistic model using the marginal likelihood of summed scores. Unpublished doctoral dissertation, The University of North Carolina at Chapel Hill.Google Scholar
Chen, W.H., & Thissen, D. (1999). Estimation of item parameters for the three-parameter logistic model using the marginal likelihood of summed scores. British Journal of Mathematical and Statistical Psychology, 52, 19–37.CrossRefGoogle Scholar
Cronbach, L.J., Gleser, G.C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY: John Wiley & Sons.Google Scholar
Finney, D.J. (1952). Probit analysis: A statistical treatment of the sigmoid response curve. London: Cambridge University Press.Google Scholar
Fischer, G.H. (1974). Einführung in die Theorie psychologischer Tests [Introduction to the theory of psychological tests], Bern: Huber.Google Scholar
Fischer, G.H., & Allerup, P. (1968). Rechentchnische Fragen zu Raschs eindimensionalem Model [An inquiry into computational techniques for the Rasch model]. In Fischer, G.H. (Eds.), Psychologische Testtheorie (pp. 269–280). Bern: Huber.Google Scholar
Goldstein, A. (2001). Making another big score. Time, 157, 66–67.Google Scholar
Henriques, D.B., & Steinberg, J. (2001, May 20). Errors plague testing industry. The New York Times, pp. A1, A22–A23.Google Scholar
Jones, L.V. (1998). L.L. Thurstone's vision of psychology as a quantitative rational science. In Kimble, G.A., & Wertheimer, M. (Eds.), Portraits of pioneers in psychology, Vol III (pp. 84–102). Washington, DC: American Psychological Association.Google Scholar
Kelley, T.L. (1927). The interpretation of educational measurements. New York, NY: World Book.Google Scholar
Kelley, T.L. (1947). Fundamentals of statistics. Cambridge: Harvard University Press.Google Scholar
Lazarsfeld, P.F. (1950). The logical and mathematical foundation of latent structure analysis. In Stouffer, S.A., Guttman, L., Suchman, E.A., Lazarsfeld, P.F., Star, S.A., & Clausen, J.A. (Eds.), Measurement and prediction (pp. 362–412). New York, NY: John Wiley & Sons.Google Scholar
Laidlaw, D.H., Fleischer, K.W., & Barr, A.H. (1995, September). Bayesian mixture classification of MRI data for geometric modeling and visualization. Poster presented at the First International Workshop on Statistical Mixture Modeling, Aussois, France. (Retrieved from the Worldwide Web: http://www.gg.caltech.edu/~dhl/aussois/paper.html)Google Scholar
Lewis, B. (2001). IS survival guide. Infoworld, 21, 96–96.Google Scholar
Lewis, B. (2001). IS survival guide. Infoworld, 23, 42–42.Google Scholar
Lindley, D.V., & Smith, A.F.M. (1972). Bayes estimates for the linear model. Journal of the Royal Statistical Society, Series B, 34, 1–41.CrossRefGoogle Scholar
Liou, M. (1994). More on the computation of higher-order derivatives of the elementary symmetric functions in the Rasch model. Applied Psychological Measurement, 18, 53–62.CrossRefGoogle Scholar
Lord, F.M. (1953). The relation of test score to the trait underlying the test. Educational and Psychological Measurement, 13, 517–548.CrossRefGoogle Scholar
Lord, F.M., & Novick, M. (1968). Statistical theories of mental test scores. Reading, MA: Addison Wesley.Google Scholar
Lord, F.M., & Wingersky, M.S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings”. Applied Psychological Measurement, 8, 453–461.CrossRefGoogle Scholar
Mislevy, R.M., Johnson, E.G., & Muraki, E. (1992). Scaling procedures in NAEP. Journal of Educational Statistics, 17, 131–154.Google Scholar
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.CrossRefGoogle Scholar
Muraki, E. (1997). A generalized partial credit model. In van der Linden, W., & Hambleton, R.K. (Eds.), Handbook of modern item response theory (pp. 153–164). New York, NY: Springer.CrossRefGoogle Scholar
Novick, M.R. (1980). Statistics as psychometrics. Psychometrika, 45, 411–424.CrossRefGoogle Scholar
Orlando, M. (1997). Item fit in the context of item response theory. Unpublished doctoral dissertation, The University of North Carolina at Chapel Hill.Google Scholar
Orlando, M., & Thissen, D. (2000). New item fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.CrossRefGoogle Scholar
Picasso, P. (1923). Picasso speaks—A statement by the artist. The Arts, 3, 315–326.Google Scholar
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Denmarks Paedagogiske Institut.Google Scholar
Raz, J., Turetsky, B.I., & Dickerson, L.W. (2001). Inference for a random wavelet packet model of single-channel event-related potentials. Journal of the American Statistical Association, 96, 409–420.CrossRefGoogle Scholar
Robbins, H. (1952). Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Soceity, 58, 527–535.CrossRefGoogle Scholar
Rosa, K., Swygert, K., Nelson, L., & Thissen, D. (2001). Item response theory applied to combinations of multiple-choice and constructed-response items—scale scores for patterns of summed scores. In Thissen, D., & Wainer, H. (Eds.), Test scoring (pp. 253–292). Mahwah, NJ: Lawrence Erlbaum & Associates.Google Scholar
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, No. 17.CrossRefGoogle Scholar
Samejima, F. (1997). Graded response model. In van der Linden, W., & Hambleton, R.K. (Eds.), Handbook of modern item response theory (pp. 85–100). New York, NY: Springer.CrossRefGoogle Scholar
Thissen, D., Nelson, L., Rosa, K., & McLeod, L.D. (2001). Item response theory for items scored in more than two categories. In Thissen, D., & Wainer, H. (Eds.), Test scoring (pp. 141–186). Mahwah, NJ: Lawrence Erlbaum & Associates.CrossRefGoogle Scholar
Thissen, D., Nelson, L., & Swygert, K. (2001). Item response theory applied to combinations of multiple-choice and constructed-response items—Approximation methods for scale scores. In Thissen, D., & Wainer, H. (Eds.), Test scoring (pp. 293–341). Mahwah, NJ: Lawrence Erlbaum & Associates.CrossRefGoogle Scholar
Thissen, D., & Orlando, M. (2001). Item response theory for items scored in two categories. In Thissen, D., Wainer, H. (Eds.), Test scoring (pp. 73–140). Mahwah, NJ: Lawrence Erlbaum & Associates.CrossRefGoogle Scholar
Thissen, D., Pommerich, M., Billeaud, K., & Williams, V.S.L. (1995). Item response theory for scores on tests including polytomous items with ordered responses. Applied Psychological Measurement, 19, 39–49.CrossRefGoogle Scholar
Thissen, D., & Wainer, H. (2001). Test scoring. Mahwah, NJ: Lawrence Erlbaum & Associates.CrossRefGoogle Scholar
Thurstone, L.L. (1925). A method of scaling psychological and educational tests. Journal of Educational Psychology, 16, 433–449.CrossRefGoogle Scholar
Thurstone, L.L. (1927). The law of comparative judgment. Psychological Review, 34, 278–286.CrossRefGoogle Scholar
Thurstone, L.L. (1937). Psychology as a quantitative rational science. Science, 85, 227–232.CrossRefGoogle ScholarPubMed
Thurstone, L.L. (1938). Primary mental abilities. Chicago, IL: University of Chicago Press.Google Scholar
Tukey, J.W. (1961). Data analysis and behavioral science or learning to bear the quantitative man's burden by shunning badmandments. Unpublished manuscript. (Reprinted in The collected works of John W. Tukey, Vol III, Philosophy and principles of data analysis: 1949–1964, pp. 187–389 by L.V. Jones (Ed.), 1986, Monterey, CA: Wadsworth & Brooks-Cole)Google Scholar
Tukey, J.W. (1962). The future of data analysis. Annals of Mathematical Statistics, 33, 1–67.CrossRefGoogle Scholar
Verhelst, N.D., & Veldhuijzen, N.H. (1991). A new algorithm for computing elementary symmetric functions and their first and second derivatives. Arnhem, The Netherlands: Netherlands Central Bureau of Statistics.Google Scholar
Wainer, H., Vevea, J.L., Camacho, F., Reeve, B., Rosa, K., Nelson, L., Swygert, K., & Thissen, D. (2001). Augmented scores—“borrowing strength” to compute scores based on small numbers of items. In Thissen, D., & Wainer, H. (Eds.), Test scoring (pp. 343–387). Mahwah, NJ: Lawrence Erlbaum & Associates.Google Scholar
Williams, V.S.L., Pommerich, M., & Thissen, D. (1998). A comparison of developmental scales based on Thurstone methods and item response theory. Journal of Educational Measurement, 35, 93–107.CrossRefGoogle Scholar
Yen, W.M. (1984). Obtaining maximum likelihood trait estimates from number-correct scores for the three-parameter logistic model. Journal of Educational Measurement, 21, 93–111.CrossRefGoogle Scholar