Hostname: page-component-cd9895bd7-q99xh Total loading time: 0 Render date: 2024-12-28T18:53:44.707Z Has data issue: false hasContentIssue false
  • English
  • Français

75 Years After Likert: Thurstone Was Right!

Published online by Cambridge University Press:  07 January 2015

Fritz Drasgow ,
Oleksandr S. Chernyshenko  and
Stephen Stark
Fritz Drasgow*
Affiliation:
University of Illinois at Urbana-Champaign
Oleksandr S. Chernyshenko
Affiliation:
Nanyang Technological University
Stephen Stark
Affiliation:
University of South Florida
*
E-mail: fdrasgow@uiuc.edu, Address: Department of Psychology, University of Illinois, 603 E. Daniel Street, Champaign, IL 61820
Get access Rights & Permissions [Opens in a new window]

Abstract

For over three-quarters of a century researchers and practitioners have analyzed rating scale data using methods that assume a dominance response process wherein an individual high on the trait assessed is assumed to answer positively with high probability. This approach derives from Likert's famous 1932 approach to the development and analysis of rating scales. In this paper, we argue that Likert scaling and related methods are misguided. Instead, we propose that methods that have evolved from Thurstone (1927, 1928, 1929) scaling provide a better representation of the choice process underlying rating scale judgments. These methods hypothesize an ideal point response process where the probability of endorsement is assumed to be directly related to the proximity of the statement to the individual's standing on the assessed trait. We review some research showing the superiority of ideal point methods for personality assessment and then describe several settings in which ideal point methods should provide tangible improvements over traditional approaches to assessment.

Type
Focal Article
Information
Industrial and Organizational Psychology , Volume 3 , Issue 4 , December 2010 , pp. 465 - 476
Copyright
Copyright © Society for Industrial and Organizational Psychology 2010 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Department of Psychology, University of Illinois at Urbana-Champaign

References

Ashton, M. C., & Lee, K. (2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11, 150–166. Google Scholar
Borman, W. C., Buck, D. E., Hanson, M. A., Motowidlo, S. J., Stark, S., & Drasgow, F. (2001). An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86, 965–973. Google Scholar
Chernyshenko, O. S., Stark, S., Chan, K.-Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523–562. Google Scholar
Chernyshenko, O. S., Stark, S., Drasgow, F., & Roberts, B. W. (2007). Constructing personality scales under the assumption of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19, 88–106. Google Scholar
Chernyshenko, O. S., Stark, S., Prewett, M. S., Gray, A. A., Stilson, F. R., & Tuttle, M. D. (2009). Normative scoring of multidimensional pairwise preference personality scales using IRT: Empirical comparisons with other formats. Human Performance, 22, 1–23. Google Scholar
Chernyshenko, O. S., Stark, S., & Williams, A. (2009). Latent trait theory approach to measuring person-organization fit: Conceptual rationale and empirical evaluation. International Journal of Testing, 9, 358–380. Google Scholar
Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18, 267–307. Google Scholar
Conn, S., & Rieke, M. L. (Eds.) (1994). The 16PF fifth edition technical manual. Champaign, IL: Institute for Personality and Ability Testing. Google Scholar
Coombs, C. H. (1964). A theory of data. New York: Wiley. Google Scholar
Davison, M. L. (1977). On a metric, unidimensional unfolding model for attitudinal and developmental data. Psychometrika, 42, 523–548. Google Scholar
Drasgow, F., Chernyshenko, O. S., & Stark, S. (2009). Test theory and personality measurement. In Butcher, J. N. (Ed.), Oxford handbook of personality assessment (pp. 59–80). New York: Oxford University Press. Google Scholar
Drasgow, F., Chernyshenko, O. S., & Stark, S. (2010). Tailored Adaptive Personality Assessment System (TAPAS). Urbana, IL: Authors. Google Scholar
Drasgow, F., Levine, M. V., Tsien, S., Williams, B., & Mead, A. D. (1995). Fitting polytomous item response theory models to multiple-choice tests. Applied Psychological Measurement, 19, 143–165. Google Scholar
Edwards, A. L. (1954). Personal preference schedule. New York: Psychological Corporation. Google Scholar
Edwards, J. R. (1994). The study of congruence in organizational behavior research: Critique and a proposed alternative. Organizational Behavior and Human Decision Processes, 58, 51–100 (erratum, 58, 323–325).Google Scholar
Hough, L. M., Eaton, N. L., Dunnette, M. D., Kamp, J. D., & McCloy, R. A. (1990). Criterion-related validities of personality constructs and the effect of response distortion on those validities [Monograph]. Journal of Applied Psychology, 75, 581–595. Google Scholar
Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced-choice offer a solution? Human Performance, 13, 371–388. Google Scholar
Levine, M. V. (1984). An introduction to multilinear formula score theory (Personnel and Training Research Programs, Office of Naval Research, Measurement Series No. 84-4). Arlington, VA: Personnel and Training Research Programs. Google Scholar
Lewin, K. (1951). Field theory in social science; selected theoretical papers. D. Cartwright (Ed.), New York: Harper & Row. Google Scholar
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22, 1–55. Google Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum. Google Scholar
McCloy, R. A., Heggestad, E. D., & Reeve, C. L. (2005). A silk purse from the sow's ear: Retrieving normative information from multidimensional forced-choice items. Organizational Research Methods, 8, 222–248. Google Scholar
Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531–551. Google Scholar
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill. Google Scholar
Reise, S. P., & Waller, N. G. (1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 45–58. Google Scholar
Roberts, J. S. (2001). GGUM2000: Estimation of parameters in the generalized graded unfolding model. Applied Psychological Measurement, 25, 38. Google Scholar
Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3–32. Google Scholar
Rounds, J. B., Henly, G. A., Dawis, R. V., Lofquist, L. H., & Weiss, D. J. (1981). Manual for the Minnesota Importance Questionnaire. Minneapolis, MN: University of Minnesota, Vocational Psychology Research. Google Scholar
Stark, S. (2002). A new IRT approach to test construction and scoring designed to reduce the effects of faking in personality assessment. Unpublished doctoral dissertation, University of Illinois at Urbana–Champaign.Google Scholar
Stark, S., & Chernyshenko, O. S. (2007, June). Adaptive testing with the multi-unidimensional pairwise preference (MUPP) model. Paper presented at the 2007 Graduate Management Admissions Council conference on Computerized Adaptive Testing. Minneapolis, MN.Google Scholar
Stark, S., Chernyshenko, O. S., Chan, K.-Y., Lee, W. C., & Drasgow, F. (2001). Effects of the testing situation on item responding: Cause for concern. Journal of Applied Psychology, 86, 943–953. Google Scholar
Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29, 184–203. Google Scholar
Stark, S., Chernyshenko, O. S., Drasgow, F., & Williams, B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring? Journal of Applied Psychology, 91, 25–39. Google Scholar
Stark, S., & Drasgow, F. (2002). An EM approach to parameter estimation for the Zinnes and Griggs paired comparison ideal point IRT model. Applied Psychological Measurement, 26, 208–227. Google Scholar
Tay, L., Drasgow, F., Rounds, J., & Williams, B. A. (2009). Fitting measurement models to vocational interest data: Are dominance models ideal? Journal of Applied Psychology, 94, 1287–1304. Google Scholar
Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34, 273–286. Google Scholar
Thurstone, L. L. (1928). Attitudes can be measured. The American Journal of Sociology, 33, 529–554. Google Scholar
Thurstone, L. L. (1929). Theory of attitude measurement. Psychological Review, 36, 222–241. Google Scholar
Vasilopoulos, N. L., Cucina, J. M., Dyomina, N. V., Morewitz, C. L., & Reilly, R. R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19, 175–199. Google Scholar
White, L. A., & Young, M. C. (1998, August). Development and validation of the Assessment of Individual Motivation (AIM). Paper presented at the 106th Annual Convention of the American Psychological Association, San Francisco, CA.Google Scholar
Zinnes, J. L., & Griggs, R. A. (1974). Probabilistic, multidimensional unfolding analysis. Psychometrika, 39, 327–350. Google Scholar