Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-01-13T17:25:47.247Z Has data issue: false hasContentIssue false
This chapter is part of a book that is no longer available to purchase from Cambridge Core

9 - Methods for Studying the Structure of Expertise: Psychometric Approaches

from PART III - METHODS FOR STUDYING THE STRUCTURE OF EXPERTISE

By
Phillip L. Ackerman  and
Margaret E. Beier
Edited by
K. Anders Ericsson ,
Neil Charness ,
Paul J. Feltovich  and
Robert R. Hoffman
Phillip L. Ackerman
Affiliation:
School of Psychology, Georgia Institute of Technology
Margaret E. Beier
Affiliation:
Department of Psychology, Rice University
K. Anders Ericsson
Affiliation:
Florida State University
Neil Charness
Affiliation:
Florida State University
Paul J. Feltovich
Affiliation:
University of West Florida
Robert R. Hoffman
Affiliation:
University of West Florida
Get access

Summary

“Psychometrics” refers to the scientific discipline that combines psychological inquiry with quantitative measurement. Though psychometric theory and practice pertain to all aspects of measurement, in the current context, psychometric approaches to expertise pertain to the measurement and prediction of individual differences and group differences (e.g., by gender, age) and, in particular, high levels of proficiency including expertise and expert performance. The scientific study of expertise involves several important psychometric considerations, such as reliability and validity of measurements, both at the level of predictors (e.g., in terms of developing aptitude measures that can predict which individuals will develop expert levels of performance), and at the level of criteria (the performance measures themselves). We will discuss these basic aspects of psychometric theory first, and then we will provide an illustration of psychometric studies that focus on the prediction of expert performance in the context of tasks that involve the development and expression of perceptual-motor skills, and tasks that involve predominantly cognitive/intellectual expertise. Finally, we will discuss challenges for future investigations.

Before we start, some psychological terms need to be defined. The first terms are “traits” and “states.” Traits refer to relatively broad and stable dispositions. Traits can be physical (e.g., visual acuity, strength) or psychological (e.g., personality, interests, intelligence). In contrast to traits, states represent temporary characteristics (e.g., sleepy, alert, angry). The second set of terms to be defined are “interindividual differences” and “intraindividual differences.”

Type
Chapter
Information
The Cambridge Handbook of Expertise and Expert Performance , pp. 147 - 166
Publisher: Cambridge University Press
Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and information processing perspectives. Psychological Bulletin, 102, 3–27.CrossRefGoogle Scholar
Ackerman, P. L. (1988). Determinants of individual differences during skill acquisition: Cognitive abilities and information processing. Journal of Experimental Psychology: General, 117, 288–318.CrossRefGoogle Scholar
Ackerman, P. L. (1990). A correlational analysis of skill specificity: Learning, abilities, and individual differences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 883–901.Google Scholar
Ackerman, P. L. (1992). Predicting individual differences in complex skill acquisition: Dynamics of ability determinants. Journal of Applied Psychology, 77, 598–614.CrossRefGoogle ScholarPubMed
Ackerman, P. L. (1996). A theory of adult intellectual development: Process, personality, interests, and knowledge. Intelligence, 22, 229–259.CrossRefGoogle Scholar
Ackerman, P. L. (2000). Domain-specific knowledge as the “dark matter” of adult intelligence: gf/gc, personality and interest correlates. Journal of Gerontology: Psychological Sciences, 55B, P69–P84.CrossRefGoogle Scholar
Ackerman, P. L., Bowen, K. R., Beier, M. B., & Kanfer, R. (2001). Determinants of individual differences and gender differences in knowledge. Journal of Educational Psychology, 93, 797–825.CrossRefGoogle Scholar
Ackerman, P. L., & Cianciolo, A. T. (2000). Cognitive, perceptual speed, and psychomotor determinants of individual differences during skill acquisition. Journal of Experimental Psychology: Applied, 6, 259–290.Google ScholarPubMed
Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121, 219–245.CrossRefGoogle ScholarPubMed
Ackerman, P. L., & Rolfhus, E. L. (1999). The locus of adult intelligence: Knowledge, abilities, and non-ability traits. Psychology and Aging, 14, 314–330.CrossRefGoogle Scholar
Ackerman, P. L., & Kanfer, R. (1993). Integrating laboratory and field study for improving selection: Development of a battery for predicting air traffic controller success. Journal of Applied Psychology, 78, 413–432.CrossRefGoogle Scholar
Ackerman, P. L., Kanfer, R., & Goff, M. (1995). Cognitive and noncognitive determinants and consequences of complex skill acquisition. Journal of Experimental Psychology: Applied, 1, 270–304.Google Scholar
Ackerman, P. L., & Woltz, D. J. (1994). Determinants of learning and performance in an associative memory/substitution task: Task constraints, individual differences, and volition. Journal of Educational Psychology, 86, 487–515.CrossRefGoogle Scholar
Adams, J. A. (1987). Historical review and ap-praisal of research on the learning, retention, and transfer of human motor skills. Psychological Bulletin, 101, 41–74.CrossRefGoogle Scholar
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215.CrossRefGoogle Scholar
Barrett, G. V., Alexander, R. A., & Doverspike, D. (1992). The implications for personnel selection of apparent declines in predictive validities over time: A critique of Hulin, Henry, and Noon. Personnel Psychology, 45, 601–617.CrossRefGoogle Scholar
Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job per-formance: A meta-analysis. Personnel Psychology, 1991 (44), 1–26.CrossRefGoogle Scholar
Binet, A., & Simon, Th. (1973). The development of intelligence in children. Trans. by Elizabeth Kite. New York: Arno Press.Google Scholar
Brunswik, E. (1952). The conceptual framework of psychology. Chicago: University of Chicago Press.Google Scholar
Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In Sternberg, R. J. (Ed.), Advances in the psychology of human intelligence (Vol. 1, pp. 7–76). Hillsdale, NJ: Erlbaum.Google Scholar
Cronbach, L. J., & Furby, L. (1970). How we should measure “change” – or should we? Psychological Bulletin, 74, 68–80.CrossRefGoogle Scholar
Dawis, R. V., & Lofquist, L. H. (1984). A psychological theory of work adjustment: An individual differences model and its applications. Minneapolis, MN: University of Minnesota Press.Google Scholar
Feltz, D. L. (1982). Path analysis of the causal elements in Bandura's theory of self-efficacy and anxiety-based model of avoidance behavior. Journal of Personality and Social Psychology, 42(4), 764–781.CrossRefGoogle Scholar
Ferguson, L. W. (1952). A look across the years 1920 to 1950. In Thurstone, L. L. (Ed.), Applications of psychology (pp. 1–17). New York: Harper & Brothers.Google Scholar
Fleishman, E. A., & Hempel, W. E. Jr. (1955). The relation between abilities and improvement with practice in a visual discrimination reaction task. Journal of Experimental Psychology, 49, 301–312.CrossRefGoogle Scholar
Guilford, J. P., Christensen, P. R., Bond, N. A. Jr., & Sutton, M. A. (1954). A factor analysis study of human interests. Psychological Monographs, 68, (4, Whole No. 375).CrossRefGoogle Scholar
Guttman, L. (1954). A new approach to factor analysis: The radex. In Lazarsfeld, P. F. (Ed.), Mathematical thinking in the social sciences (pp. 258–348). Glencoe, Illinois, The Free Press.Google Scholar
Hoffman, R. R. (1987, Summer). The problem of extracting the knowledge of experts from the perspective of experimental psychology. The AI Magazine, 8, 53–67.Google Scholar
Holland, J. L. (1959). A theory of vocational choice. Journal of Counseling Psychology, 6, 35–45.CrossRefGoogle Scholar
Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd Edition). Odessa, FL: Psychological Assessment Resources.Google Scholar
Honzik, M. P., MacFarlane, J. W., & Allen, L. (1948). The stability of mental test performance between two and eighteen years. Journal of Experimental Education, 17, 309–324.CrossRefGoogle Scholar
Humphreys, L. G. (1960). Investigations of the simplex. Psychometrika, 25, 313–323.CrossRefGoogle Scholar
Hunt, E. (1995). Will we be smart enough?: A cognitive analysis of the coming workforce. New York: Russell Sage Foundation.Google Scholar
Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.Google Scholar
Jones, M. B. (1962). Practice as a process of simplification. Psychological Review, 69, 274–294.CrossRefGoogle ScholarPubMed
Kanfer, R. (1987). Task-specific motivation: An integrative approach to issues of measurement, mechanisms, processes, and determinants. Journal of Social and Clinical Psychology, 5, 251–278.CrossRefGoogle Scholar
Kanfer, R., & Ackerman, P. L. (1989). Moti-vation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology – Monograph, 74, 657–690.CrossRefGoogle Scholar
Kant, I. (1790/1987). Critique of judgment. Trans. by Werner S. Pluhar. Indianapolis, IN: Hackett Publishing Co. Google Scholar
Lohman, D. F. (1999). Minding our p's and q's: On finding relationships between learning and intelligence. In Ackerman, P. L., Kyllonen, P. C., & Roberts, R. D. (Eds.), Learning and individual differences: Process, trait, and content determinants (pp. 55–76). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
McClelland, D. C., & Boyatzis, R. E. (1982). Leadership motive pattern and long-term success in management. Journal of Applied Psychology, 67, 737–743.CrossRefGoogle Scholar
McNemar, Q. (1940). A critical examination of the University of Iowa studies of environmental influences upon the IQ. Psychological Bulletin, 37, 63–92.CrossRefGoogle Scholar
Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–216.CrossRefGoogle ScholarPubMed
Murray, H. A. et al. (1938). Explorations in personality: A clinical and experimental study of fifty men of college age. New York: Oxford University Press.Google Scholar
Roe, A. (1956). The psychology of occupations. New York: Wiley & Sons.CrossRefGoogle Scholar
Simonton, D. K. (1994). Greatness: Who makes history and why. New York: Guilford Press.Google Scholar
Snow, R. E. (1989). Aptitude-treatment interaction as a framework for research on individual differences in learning. In Ackerman, P. L., Sternberg, R. J., & Glaser, R. (Eds.), Learning and individual differences: Advances in theory and research (pp. 13–59). New York: W. H. Freeman.Google Scholar
Spangler, W. D. (1992). Validity of question-naire and TAT measures of need of achieve-ment: Two meta-analyses. Psychological Bulletin, 112(1), 140–154.CrossRefGoogle Scholar
Stanovich, K. E. (1986). Matthew effects in reading. Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360–406.CrossRefGoogle Scholar
Super, D. E. (1940). Avocational interest patterns: A study in the psychology of avocations. Stanford, CA: Stanford University Press.Google Scholar
Terman, L. M. (1926). Genetic studies of genius: Mental and physical traits of a thousand gifted children. Stanford, CA: Stanford University Press.Google Scholar
Thorndike, E. L. (1908). The effect of practice in the case of a purely intellectual function. American Journal of Psychology, 19, 374 –384.CrossRefGoogle Scholar
Thorndike, R. L. (1949). Personnel selection. New York: John Wiley & Sons.Google Scholar
Warr, P. (1994). Age and employment. In Triandis, H. C., Dunnette, M. D., et al. (Eds), Handbook of industrial and organizational psychology, Vol. 4 (pp. 485–550). Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Watson, J. D. (2001). The double helix: A personal account of the discovery of the structure of DNA. New York: Simon & Schuster.Google Scholar
Willingham, W. W. (1974). Predicting success in graduate education. Science, 183, 273–278.CrossRefGoogle ScholarPubMed
Wittmann, W. W., & Süß, H.-M. (1999). Investigating the paths between working memory, intelligence, knowledge, and complex problem-solving performances via Brunswik symmetry. In Ackerman, P. L., Kyllonen, P. C., & Roberts, R. D. (Eds.), Learning and individual differences: Process, trait, and content determinants (pp. 77–108). Washington, DC: American Psychological Association.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×