Hostname: page-component-78c5997874-4rdpn Total loading time: 0 Render date: 2024-11-10T16:37:22.894Z Has data issue: false hasContentIssue false
  • English
  • Français

What can experimental studies of bias tell us about real-world group disparities?

Published online by Cambridge University Press:  08 January 2021

Joseph Cesario Open the ORCID record for Joseph Cesario [Opens in a new window]
Joseph Cesario*
Affiliation:
Department of Psychology, Michigan State University, East Lansing, MI 48824, USA. cesario@msu.edu www.cesariolab.com
Get access Rights & Permissions [Opens in a new window]

Abstract

This article questions the widespread use of experimental social psychology to understand real-world group disparities. Standard experimental practice is to design studies in which participants make judgments of targets who vary only on the social categories to which they belong. This is typically done under simplified decision landscapes and with untrained decision-makers. For example, to understand racial disparities in police shootings, researchers show pictures of armed and unarmed Black and White men to undergraduates and have them press “shoot” and “don't shoot” buttons. Having demonstrated categorical bias under these conditions, researchers then use such findings to claim that real-world disparities are also due to decision-maker bias. I describe three flaws inherent in this approach, flaws which undermine any direct contribution of experimental studies to explaining group disparities. First, the decision landscapes used in experimental studies lack crucial components present in actual decisions (missing information flaw). Second, categorical effects in experimental studies are not interpreted in light of other effects on outcomes, including behavioral differences across groups (missing forces flaw). Third, there is no systematic testing of whether the contingencies required to produce experimental effects are present in real-world decisions (missing contingencies flaw). I apply this analysis to three research topics to illustrate the scope of the problem. I discuss how this research tradition has skewed our understanding of the human mind within and beyond the discipline and how results from experimental studies of bias are generally misunderstood. I conclude by arguing that the current research tradition should be abandoned.

Keywords

Discriminationdisparate outcomesimplicit biasracial biasschool disciplineshooter biassocial psychologystereotyping
Type
Target Article
Information
Behavioral and Brain Sciences , Volume 45 , 2022 , e66
Check for updates
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

ACT. (2017). The Condition of College & Career Readiness 2017 (Tech. Rep.).Google Scholar
Akinola, M. N. (2009). Deadly decisions: An examination of racial bias in the decision to shoot under threat. Dissertation, Harvard University.Google Scholar
Andreoni, J., Kuhn, M. A., List, J. A., Samek, A., Sokal, K., & Sprenger, C. (2019). Toward an understanding of the development of time preferences: Evidence from field experiments. Journal of Public Economics, 177, 104039.CrossRefGoogle ScholarPubMed
Arcuri, L., Castelli, L., Galdi, S., Zogmaister, C., & Amadori, A. (2008). Predicting the vote: Implicit attitudes as predictors of the future behavior of decided and undecided voters. Political Psychology, 29, 369–387.CrossRefGoogle Scholar
Arkes, H. R., & Tetlock, P. E. (2004). Attributions of implicit prejudice, or “would Jesse Jackson ‘fail’ the Implicit Association Test?”. Psychological Inquiry, 15, 257–278.CrossRefGoogle Scholar
Banks, R. R., Eberhardt, J. L., & Ross, L. (2006). Discrimination and implicit bias in a racially unequal society. California Law Review, 94, 1169–1190.CrossRefGoogle Scholar
Barden, J., Maddux, W. W., Petty, R. E., & Brewer, M. B. (2004). Contextual moderation of racial bias: The impact of social roles on controlled and automatically activated attitudes. Journal of Personality and Social Psychology, 87, 5–22.CrossRefGoogle ScholarPubMed
Bargh, J. A. (1989). Conditional automaticity: Varieties of automatic influence in social perception and cognition. In Uleman, J. S. & Bargh, J. A. (Eds.), Unintended thought (pp. 3–51). Guilford.Google Scholar
Bargh, J. A. (1999). The cognitive monster: The case against the controllability of automatic stereotype effects. In Chaiken, S. & Trope, Y. (Eds.), Dual process theories in social psychology (pp. 361–382). Guilford.Google Scholar
Barnes, J., Jorgensen, C., Beaver, K. M., Boutwell, B. B., & Wright, J. P. (2015). Arrest prevalence in a national sample of adults: The role of sex and race/ethnicity. American Journal of Criminal Justice, 40, 457–465.CrossRefGoogle Scholar
Beaver, K. M., DeLisi, M., Wright, J. P., Boutwell, B. B., Barnes, J. C., & Vaughn, M. G. (2013). No evidence of racial discrimination in criminal justice processing: Results from the national longitudinal study of adolescent health. Personality and Individual Differences, 55, 29–34.CrossRefGoogle Scholar
Benbow, C. P., & Stanley, J. C. (1980). Sex differences in mathematical ability: Fact or artifact? Science (New York, N.Y.), 210, 1262–1264.CrossRefGoogle ScholarPubMed
Benbow, C. P., Lubinski, D., Shea, D. L., & Eftekhari-Sanjani, H. (2000). Sex differences in mathematical reasoning ability at age 13: Their status 20 years later. Psychological Science, 11, 474–480.CrossRefGoogle ScholarPubMed
Bertrand, M., & Mullainathan, S. (2004). Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American Economic Review, 94, 991–1013.CrossRefGoogle Scholar
Blair, I. V. (2002). The malleability of automatic stereotypes and prejudice. Personality and Social Psychology Review, 6, 242–261.CrossRefGoogle Scholar
Blanton, H., & Jaccard, J. (2008). Unconscious racism: A concept in pursuit of a measure. Annual Review of Sociology, 34, 277–297.CrossRefGoogle Scholar
Blanton, H., Jaccard, J., Christie, C., & Gonzales, P. M. (2007). Plausible assumptions, questionable assumptions and post hoc rationalizations: Will the real IAT, please stand up? Journal of Experimental Social Psychology, 43, 399–409.CrossRefGoogle Scholar
Blanton, H., Jaccard, J., Gonzales, P. M., & Christie, C. (2006). Decoding the implicit association test: Implications for criterion prediction. Journal of Experimental Social Psychology, 42, 192–212.CrossRefGoogle Scholar
Blanton, H., Jaccard, J., Klick, J., Mellers, B., Mitchell, G., & Tetlock, P. E. (2009). Strong claims and weak evidence: Reassessing the predictive validity of the IAT. Journal of Applied Psychology, 94, 567–582.CrossRefGoogle ScholarPubMed
Blanton, H., Jaccard, J., Strauts, E., Mitchell, G., & Tetlock, P. E. (2015). Toward a meaningful metric of implicit prejudice. Journal of Applied Psychology, 100, 1468–1481.CrossRefGoogle Scholar
Byrnes, J. P., Miller, D. C., & Schafer, W. D. (1999). Gender differences in risk taking: A meta-analysis. Psychological Bulletin, 125, 367–383.CrossRefGoogle Scholar
Carey, B., & Goode, E. (2016). Police try to lower bias, but under pressure, it isn't so easy. New York Times. Retrieved from http://www.nytimes.com/2016/07/12/science/bias-reduction-programs.html.Google Scholar
Ceci, S. J., & Williams, W. M. (2010). Sex differences in math-intensive fields. Current Directions in Psychological Science, 19, 275–279.CrossRefGoogle ScholarPubMed
Ceci, S. J., & Williams, W. M. (2011). Understanding current causes of women's underrepresentation in science. Proceedings of the National Academy of Sciences, 108, 3157–3162.CrossRefGoogle ScholarPubMed
Ceci, S. J., Williams, W. M., & Barnett, S. M. (2009). Women's underrepresentation in science: Sociocultural and biological considerations. Psychological Bulletin, 135, 218–261.CrossRefGoogle ScholarPubMed
Cesario, J. (2021). On selective emphasis, broad agreement, and future directions: Reply to Ross, Winterhalder, & McElreath. Retrieved from psyarxiv.com/2p5eg.Google Scholar
Cesario, J., & Carrillo, A. (in press). Racial bias in police officer deadly force decisions: What has social cognition learned? In Carlston, D. E., Johnson, K., & Hugenberg, K. (Eds.), The Oxford handbook of social cognition (2nd ed.). Oxford University Press.Google Scholar
Cesario, J., Johnson, D. J., & Terrill, W. (2019). Is there evidence of racial disparity in police use of deadly force? Analyses of officer-involved fatal shootings in 2015–2016. Social Psychological and Personality Science, 10, 586–595.CrossRefGoogle Scholar
Cheng, S. D. (2020). Careers Versus Children: How Childcare Affects the Academic Tenure-Track Gender Gap. Retrieved from https://scholar.harvard.edu/files/sdcheng/files/sdcheng_kids_jmpv7.pdf.Google Scholar
Corneille, O., & Hütter, M. (2020). Implicit? What do you mean? A comprehensive review of the delusive implicitness construct in attitude research. Personality and Social Psychology Review, 24, 212–232.CrossRefGoogle Scholar
Correll, J., Hudson, S. M., Guillermo, S., & Ma, D. S. (2014). The police officer's dilemma: A decade of research on racial bias in the decision to shoot. Social and Personality Psychology Compass, 8, 201–213.CrossRefGoogle Scholar
Correll, J., Park, B., Judd, C. M., & Wittenbrink, B. (2002). The police officer's dilemma: Using ethnicity to disambiguate potentially threatening individuals. Journal of Personality and Social Psychology, 83, 1314–1329.CrossRefGoogle ScholarPubMed
Correll, J., Park, B., Judd, C. M., Wittenbrink, B., Sadler, M. S., & Keesee, T. (2007). Across the thin blue line: Police officers and racial bias in the decision to shoot. Journal of Personality and Social Psychology, 92, 1006–1023.CrossRefGoogle Scholar
Correll, J., Wittenbrink, B., Park, B., Judd, C. M., & Goyle, A. (2011). Dangerous enough: Moderating racial bias with contextual threat cues. Journal of Experimental Social Psychology, 47, 184–189.CrossRefGoogle ScholarPubMed
Cortés, P., & Pan, J. (2020). Children and the remaining gender gaps in the labor market. NBER Working Paper 27980.CrossRefGoogle Scholar
Cox, W. T. L., & Devine, P. G. (2016). Experimental research on shooter bias: Ready (or relevant) for application in the courtroom? Journal of Applied Research in Memory and Cognition, 5, 236–238.CrossRefGoogle Scholar
Cox, W. T. L., Devine, P. G., Plant, E. A., & Schwartz, L. L. (2014). Toward a comprehensive understanding of officers’ shooting decisions: No simple answers to this complex problem. Basic and Applied Social Psychology, 36, 356–364.CrossRefGoogle Scholar
Cvencek, D., Greenwald, A. G., & Meltzoff, A. N. (2011a). Measuring implicit attitudes of 4-year-olds: The preschool implicit association test. Journal of Experimental Child Psychology, 109, 187–200.CrossRefGoogle Scholar
Cvencek, D., Meltzoff, A. N., & Greenwald, A. G. (2011b). Math–gender stereotypes in elementary school children. Child Development, 82, 766–779.CrossRefGoogle Scholar
Darley, J. M., & Gross, P. H. (1983). A hypothesis-confirming bias in labeling effects. Journal of Personality and Social Psychology, 44, 20–33.CrossRefGoogle Scholar
Dasgupta, N., & Stout, J. G. (2012). Contemporary discrimination in the lab and field: Benefits and obstacles of full-cycle social psychology. Journal of Social Issues, 68, 399–412.CrossRefGoogle Scholar
Davis, E., Whyde, A., & Langton, L. (2018). Contacts between police and the public, 2015. Bureau of Justice Statistics. U.S. Department of Justice.Google Scholar
DeNavas-Walt, C., Proctor, B., & Smith, J. (2013). Income, poverty, and health insurance coverage in the United States: 2012. U.S. Department of Commerce.Google Scholar
Devine, P. G. (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56, 5–18.CrossRefGoogle Scholar
Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004). The role of representative design in an ecological approach to cognition. Psychological Bulletin, 130, 959–988.CrossRefGoogle Scholar
Diekman, A. B., Steinberg, M., Brown, E. R., Belanger, A. L., & Clark, E. K. (2017). A goal congruity model of role entry, engagement, and exit: Understanding communal goal processes in STEM gender gaps. Personality and Social Psychology Review, 21, 142–175.CrossRefGoogle ScholarPubMed
Dovidio, J. F. (2001). On the nature of contemporary prejudice: The third wave. Journal of Social Issues, 57, 829–849.CrossRefGoogle Scholar
Dovidio, J. F., & Gaertner, S. L. (2000). Aversive racism and selection decisions: 1989 and 1999. Psychological Science, 11, 315–319.CrossRefGoogle ScholarPubMed
Dovidio, J. F., Penner, L. A., Albrecht, T. L., Norton, W. E., Gaertner, S. L., & Shelton, J. N. (2008). Disparities and distrust: The implications of psychological processes for understanding racial disparities in health and health care. Social Science & Medicine, 67, 478–486.CrossRefGoogle ScholarPubMed
Dreifus, C. (2015). Perceptions of race at a glance. New York Times. Retrieved from http://www.nytimes.com/2015/01/06/science/a-macarthur-grant-winner-tries-to-unearth-biases-to-aid-criminal-justice.html.Google Scholar
Duncan, B. L. (1976). Differential social perception and attribution of intergroup violence: Testing the lower limits of stereotyping of blacks. Journal of Personality and Social Psychology, 34, 590–598.CrossRefGoogle Scholar
Eagly, A. H., & Johannesen-Schmidt, M. C. (2001). The leadership styles of women and men. Journal of Social Issues, 57, 781–797.CrossRefGoogle Scholar
Eagly, A. H., & Johnson, B. T. (1990). Gender and leadership style: A meta-analysis. Psychological Bulletin, 108, 233–256.CrossRefGoogle Scholar
Eagly, A. H., Nater, C., Miller, D. I., Kaufmann, M., & Sczesny, S. (2020). Gender stereotypes have changed: A cross-temporal meta-analysis of U.S. public opinion polls from 1946 to 2018. American Psychologist, 75, 301–315.CrossRefGoogle ScholarPubMed
Eagly, A. H., Wood, W., & Diekman, A. H. (2000). Social role theory of sex differences and similarities: A current appraisal. In Eckes, T. & Trautner, H. M. (Eds.), The developmental social psychology of gender (pp. 123–174). Erlbaum.Google Scholar
Fazio, R. H. (1990). Multiple processes by which attitudes guide behavior: The MODE model as an integrative framework. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 23, pp. 75–109). Elsevier.Google Scholar
Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995). Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline? Journal of Personality and Social Psychology, 69, 1013–1027.CrossRefGoogle ScholarPubMed
Felson, R. B. (1991). Blame analysis: Accounting for the behavior of protected groups. The American Sociologist, 22, 5–23.CrossRefGoogle Scholar
Fiedler, K., Messner, C., & Bluemke, M. (2006). Unresolved problems with the “I”, the “A”, and the “T”: A logical and psychometric critique of the Implicit Association Test (IAT). European Review of Social Psychology, 17, 74–147.CrossRefGoogle Scholar
Fiske, S. T., & Neuberg, S. L. (1990). A continuum of impression formation, from category-based to individuating processes: Influences of information and motivation on attention and interpretation. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 23, pp. 1–74). Elsevier.Google Scholar
Fryer, R. G. Jr (2016). An empirical analysis of racial differences in police use of force (Tech. Rep.). National Bureau of Economic Research.Google Scholar
Fryer, R. G. Jr, & Levitt, S. D. (2010). An empirical analysis of the gender gap in mathematics. American Economic Journal: Applied Economics, 2, 210–240.Google Scholar
Fyfe, J. J. (1980). Geographic correlates of police shooting: A microanalysis. Journal of Research in Crime and Delinquency, 17, 101–103.CrossRefGoogle Scholar
Gawronski, B. (2019). Six lessons for a cogent science of implicit bias and its criticism. Perspectives on Psychological Science, 14, 574–595.CrossRefGoogle ScholarPubMed
Gawronski, B., & Sritharan, R. (2010). Formation, change, and contextualization of mental associations: Determinants and principles of variations in implicit measures. In Gawronski, B. & Payne, B. K. (Eds.), Handbook of implicit social cognition: Measurement, theory, and applications (pp. 216–240). The Guilford Press.Google Scholar
Geller, W. A., & Karales, K. J. (1981). Shootings of and by Chicago police: Uncommon crises – part I: Shootings by Chicago police. Journal of Criminal Law & Criminology, 72, 1813–1866.CrossRefGoogle Scholar
Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506–528.CrossRefGoogle Scholar
Gottfredson, L. S. (1997). Mainstream science on intelligence: An editorial with 52 signatories, history, and bibliography. Intelligence, 24, 13–23.CrossRefGoogle Scholar
Gottfredson, L. S. (1998). The general intelligence factor. Scientific American Presents, 9, 24–29.Google Scholar
Gottfredson, L. S. (2004). Intelligence: Is it the epidemiologists’ elusive “fundamental cause” of social class inequalities in health? Journal of Personality and Social Psychology, 86, 174–199.CrossRefGoogle ScholarPubMed
Gray, N. S., Brown, A. S., MacCulloch, M. J., Smith, J., & Snowden, R. J. (2005). An implicit test of the associations between children and sex in pedophiles. Journal of Abnormal Psychology, 114, 304–308.CrossRefGoogle ScholarPubMed
Green, A. R., Carney, D. R., Pallin, D. J., Ngo, L. H., Raymond, K. L., Iezzoni, L. I., & Banaji, M. R. (2007). Implicit bias among physicians and its prediction of thrombolysis decisions for Black and White patients. Journal of General Internal Medicine, 22, 1231–1238.CrossRefGoogle ScholarPubMed
Greenwald, A. G., & Banaji, M. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4–27.CrossRefGoogle ScholarPubMed
Greenwald, A. G., & Krieger, L. H. (2006). Implicit bias: Scientific foundations. California Law Review, 94, 945–967.CrossRefGoogle Scholar
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464–1480.CrossRefGoogle ScholarPubMed
Haidt, J. (2011). The bright future of post-partisan social psychology. Talk given at the annual meeting of the Society for Personality and Social Psychology, San Antonio, TX. Retrieved from http://people.virginia.edu/~jdh6n/postpartisan.html.Google Scholar
Hakim, C. (2006). Women, careers, and work-life preferences. British Journal of Guidance & Counselling, 34, 279–294.CrossRefGoogle Scholar
Halpern, D. F., Benbow, C. P., Geary, D. C., Gur, R. C., Hyde, J. S., & Gernsbacher, M. A. (2007). The science of sex differences in science and mathematics. Psychological Science in the Public Interest, 8, 1–51.CrossRefGoogle ScholarPubMed
Heckman, J. J. (1998). Detecting discrimination. Journal of Economic Perspectives, 12, 101–116.CrossRefGoogle Scholar
Heriot, G. L., & Somin, A. (2018). The department of education's Obama-era initiative on racial disparities in school discipline: Wrong for students and teachers, wrong on the law. Texas Review of Law and Politics, Forthcoming; San Diego Legal Studies Paper No. 18-321.Google Scholar
Higgins, E. T. (1996). Knowledge activation: Accessibility, applicability, and salience. In Higgins, E. T. & Kruglanski, A. W. (Eds.), Social psychology: Handbook of basic principles (pp. 133–168). Guilford Press.Google Scholar
Higgins, E. T., & Bargh, J. A. (1987). Social cognition and social perception. Annual Review of Psychology, 38, 369–425.CrossRefGoogle ScholarPubMed
Hogarth, R. M. (1981). Beyond discrete biases: Functional and dysfunctional aspects of judgmental heuristics. Psychological Bulletin, 90, 197–217.CrossRefGoogle Scholar
Hsia, J. (1988). Asian Americans in higher education and at work. Lawrence Erlbaum Associates, Inc.Google Scholar
Hsin, A., & Xie, Y. (2014). Explaining Asian Americans’ academic advantage over whites. Proceedings of the National Academy of Sciences, 111, 8416–8421.CrossRefGoogle ScholarPubMed
Inbar, Y., & Lammers, J. (2012). Political diversity in social and personality psychology. Perspectives on Psychological Science, 7, 496–403.CrossRefGoogle ScholarPubMed
Inn, A., Wheeler, A. C., & Sparling, C. L. (1977). The effects of suspect race and situation hazard on police officer shooting behavior. Journal of Applied Social Psychology, 7, 27–37.CrossRefGoogle Scholar
James, L., James, S. M., & Vila, B. J. (2016). The reverse racism effect: Are cops more hesitant to shoot black than white suspects? Criminology & Public Policy, 15, 457–479.CrossRefGoogle Scholar
James, L., Klinger, D., & Vila, B. (2014). Racial and ethnic bias in decisions to shoot seen through a stronger lens: Experimental results from high-fidelity laboratory simulations. Journal of Experimental Criminology, 10, 323–340.CrossRefGoogle Scholar
James, L., Vila, B., & Daratha, K. (2013). Results from experimental trials testing participant responses to White, Hispanic and Black suspects in high-fidelity deadly force judgment and decision-making simulations. Journal of Experimental Criminology, 9, 189–212.CrossRefGoogle Scholar
Jarvis, S. N., & Okonofua, J. A. (2020). School deferred: When bias affects school leaders. Social Psychological and Personality Science, 11, 492–498.CrossRefGoogle Scholar
Johnson, D. J., Cesario, J., & Pleskac, T. J. (2018). How prior information and police experience impact decisions to shoot. Journal of Personality and Social Psychology, 115, 601–623.CrossRefGoogle Scholar
Johnson, D. J., Hopwood, C. J., Cesario, J., & Pleskac, T. J. (2017). Advancing research on cognitive processes in social and personality psychology: A diffusion model primer. Social Psychological and Personality Science, 8, 413–423.CrossRefGoogle Scholar
Jost, J. T. (2019). The IAT is dead, long live the IAT: Context-sensitive measures of implicit attitudes are indispensable to social and political psychology. Current Directions in Psychological Science, 28, 10–19.CrossRefGoogle Scholar
Jost, J. T., Rudman, L. A., Blair, I. V., Carney, D. R., Dasgupta, N., Glaser, J., & Hardin, C. D. (2009). The existence of implicit bias is beyond reasonable doubt: A refutation of ideological and methodological objections and executive summary of ten studies that no manager should ignore. Research in organizational behavior, 29, 39–69.CrossRefGoogle Scholar
Jussim, L. (2012a). Liberal privilege in academic psychology and the social sciences: Commentary on Inbar & Lammers (2012). Perspectives on Psychological Science, 7, 504–507.CrossRefGoogle Scholar
Jussim, L. (2012b). Social perception and social reality: Why accuracy dominates bias and self-fulfilling prophecy. Oxford University Press.CrossRefGoogle Scholar
Jussim, L., Cain, T. R., Crawford, J. T., Harber, K., & Cohen, F. (2009). The unbearable accuracy of stereotypes. In T. D. Nelson (Ed.), Handbook of prejudice, stereotyping, and discrimination (pp. 199–227). Psychology Press.Google Scholar
Jussim, L., Crawford, J., Anglin, S., Chambers, J., Stevens, S., & Cohen, F. (2015a). Stereotype accuracy: One of the largest and most replicable effects in all of social psychology. In T. D. Nelson (Ed.), Handbook of prejudice, stereotyping, and discrimination (pp. 31–63). Psychology Press.Google Scholar
Jussim, L., Crawford, J. T., Anglin, S. M., & Stevens, S. T. (2015b). Ideological bias in social psychological research. In J. P. Forgas, K. Fiedler, & W. D. Crano (Eds.), Social psychology and politics (pp. 107–126). Psychology Press.Google Scholar
Jussim, L., Crawford, J. T., & Rubinstein, R. S. (2015c). Stereotype (in)accuracy in perceptions of groups and individuals. Current Directions in Psychological Science, 24, 490–497.CrossRefGoogle Scholar
Kang, J., & Banaji, M. R. (2006). Fair measures: Behavioral realist revision of affirmative action. California Law Review, 94, 1063–1018.CrossRefGoogle Scholar
Kiefer, A. K., & Sekaquaptewa, D. (2007). Implicit stereotypes, gender identification, and math-related outcomes: A prospective study of female college students. Psychological Science, 18, 13–18.CrossRefGoogle ScholarPubMed
Klein, G. A. (1998). Sources of power: How people make decisions. MIT Press.Google Scholar
Kleven, H., Landais, C., Posch, J., Steinhauer, A., & Zweimüller, J. (2020). Do Family Policies Reduce Gender Inequality? Evidence from 60 Years of Policy Experimentation. NBER Working Paper (w28082).CrossRefGoogle Scholar
Kleven, H., Landais, C., & Søgaard, J. E. (2019). Children and gender inequality: Evidence from Denmark. American Economic Journal: Applied Economics, 11, 181–209.Google Scholar
Klinger, D., Rosenfeld, R., Isom, D., & Deckard, M. (2016). Race, crime, and the micro-ecology of deadly force. Criminology & Public Policy, 15, 193–222.CrossRefGoogle Scholar
Koch, A. J., D'Mello, S. D., & Sackett, P. R. (2015). A meta-analysis of gender stereotypes and bias in experimental simulations of employment decision making. Journal of Applied Psychology, 100, 128–161.CrossRefGoogle ScholarPubMed
Koenig, A. M., & Eagly, A. H. (2014). Evidence for the social role theory of stereotype content: Observations of groups’ roles shape stereotypes. Journal of Personality and Social Psychology, 107, 371–392.CrossRefGoogle ScholarPubMed
Kristof, N. (2014). Is everyone a little bit racist? New York Times. Retrieved from http://www.nytimes.com/2014/08/28/opinion/nicholas-kristof-is-everyone-a-little-bit-racist.html.Google Scholar
Krueger, J., & Rothbart, M. (1988). Use of categorical and individuating information in making inferences about personality. Journal of Personality and Social Psychology, 55, 187–195.CrossRefGoogle ScholarPubMed
Kunda, Z., & Spencer, S. J. (2003). When do stereotypes come to mind and when do they color judgment? A goal-based theoretical framework for stereotype activation and application. Psychological Bulletin, 129, 522–544.CrossRefGoogle ScholarPubMed
Kunda, Z., & Thagard, P. (1996). Forming impressions from stereotypes, traits, and behaviors: A parallel-constraint-satisfaction theory. Psychological Review, 103, 284–308.CrossRefGoogle Scholar
Lee, K., & Ashton, M. C. (2020). Sex differences in HEXACO personality characteristics across countries and ethnicities. Journal of Personality, 88, 1075–1090.CrossRefGoogle ScholarPubMed
Levine, J. M., Resnick, L. B., & Higgins, E. T. (1993). Social foundations of cognition. Annual Review of Psychology, 44, 585–612.CrossRefGoogle ScholarPubMed
Lewis, C. W., Butler, B. R., Bonner, F. A. III, & Joubert, M. (2010). African American male discipline patterns and school district responses resulting impact on academic achievement: Implications for urban educators and policy makers. Journal of African American Males in Education, 1, 7–25.Google Scholar
Lhamon, C., & Samuels, J. (2014). Dear colleague letter on the nondiscriminatory administration of school discipline. Washington, DC: U.S. Department of Education, Office of Civil Rights & U.S. Department of Justice, Civil Rights Division.Google Scholar
Lick, D. J., Alter, A. L., & Freeman, J. B. (2018). Superior pattern detectors efficiently learn, activate, apply, and update social stereotypes. Journal of Experimental Psychology: General, 147, 209–227.CrossRefGoogle ScholarPubMed
Lippa, R. (1998). Gender-related individual differences and the structure of vocational interests: The importance of the people–things dimension. Journal of Personality and Social Psychology, 74, 996–1009.CrossRefGoogle ScholarPubMed
Locksley, A., Borgida, E., Brekke, N., & Hepburn, C. (1980). Sex stereotypes and social judgment. Journal of Personality and Social Psychology, 39, 821–831.CrossRefGoogle Scholar
Logan, G. D. (2018). Automatic control: How experts act without thinking. Psychological Review, 125, 453–485.CrossRefGoogle ScholarPubMed
Lopez, G. (2017). Police shootings and brutality in the US: 9 things you should know. Vox. Retrieved from https://www.vox.com/cards/police-brutality-shootings-us/us-police-racism.Google Scholar
Loughlin, J. K., & Flora, K. (2017). Shots fired: The misunderstandings, misconceptions, and myths about police shootings. Simon and Schuster.Google Scholar
Lu, J. G., Nisbett, R. E., & Morris, M. W. (2020). Why East Asians but not South Asians are underrepresented in leadership positions in the United States. Proceedings of the National Academy of Sciences, 117, 4590–4600.CrossRefGoogle Scholar
Lubinski, D., & Benbow, C. P. (1992). Gender differences in abilities and preferences among the gifted: Implications for the math-science pipeline. Current Directions in Psychological Science, 1, 61–66.CrossRefGoogle Scholar
Lynn, R. (2004). The intelligence of American Jews. Personality and Individual Differences, 36, 201–206.CrossRefGoogle Scholar
Lynn, R., & Irwing, P. (2004). Sex differences on the progressive matrices: A meta-analysis. Intelligence, 32, 481–498.CrossRefGoogle Scholar
Ma, D. S., & Correll, J. (2011). Target prototypicality moderates racial bias in the decision to shoot. Journal of Experimental Social Psychology, 47, 391–396.CrossRefGoogle Scholar
Ma, D., Graves, S., & Alvarado, J. (2019). A spatial analysis of officer-involved shootings in Los Angeles. Yearbook of the Association of Pacific Coast Geographers, 81, 158–181.CrossRefGoogle Scholar
Madon, S., Jussim, L., Guyll, M., Nofziger, H., Salib, E., Willard, J., & Scherr, K. C. (2018). The accumulation of stereotype-based self-fulfilling prophecies. Journal of Personality and Social Psychology, 115(5), 825–844.CrossRefGoogle ScholarPubMed
Martell, R. F., Lane, D. M., & Emrich, C. (1996). Male–female differences: A computer simulation. American Psychologist, 51, 157–158.CrossRefGoogle Scholar
McCauley, C., Stitt, C. L., & Segal, M. (1980). Stereotyping: From prejudice to prediction. Psychological Bulletin, 87, 195–208.CrossRefGoogle Scholar
McLanahan, S., & Percheski, C. (2008). Family structure and the reproduction of inequalities. Annual Review Sociology, 34, 257–276.CrossRefGoogle Scholar
Mekawi, Y., & Bresin, K. (2015). Is the evidence from racial bias shooting task studies a smoking gun? Results from a meta-analysis. Journal of Experimental Social Psychology, 61, 120–130.CrossRefGoogle Scholar
Mentch, L. (2020). On racial disparities in recent fatal police shootings. Statistics and Public Policy, 7, 9–18.CrossRefGoogle Scholar
Miller, A. L. (2019). Expertise fails to attenuate gendered biases in judicial decision-making. Social Psychological and Personality Science, 10, 227–234.CrossRefGoogle Scholar
Miller, T. R., Lawrence, B. A., Carlson, N. N., Hendrie, D., Randall, S., Rockett, I. R., & Spicer, R. S. (2017). Perils of police action: A cautionary tale from US data sets. Injury Prevention, 23, 27–32.CrossRefGoogle ScholarPubMed
Mitchell, G. (2012). Revisiting truth or triviality: The external validity of research in the psychological laboratory. Perspectives on Psychological Science, 7, 109–117.CrossRefGoogle ScholarPubMed
Mitchell, G. (2018). Jumping to conclusions: Advocacy and application of psychological research. In Crawford, J. & Jussim, L. (Eds.), The politics of social psychology (pp. 139–155). Routledge.Google Scholar
Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379–387.CrossRefGoogle Scholar
Morrison, G. (2006). Police department and instructor perspectives on pre-service firearm and deadly force training. Policing: An International Journal of Police Strategies & Management, 29, 226–245.CrossRefGoogle Scholar
Mortensen, C. R., & Cialdini, R. B. (2010). Full-cycle social psychology for theory and application. Social and Personality Psychology Compass, 4, 53–63.CrossRefGoogle Scholar
Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty's subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109, 16474–9.CrossRefGoogle ScholarPubMed
Musu-Gillette, L., Zhang, A., Wang, K., Zhang, J., Kemp, J., Diliberti, M., & Oudekerk, B. A. (2018). Indicators of school crime and safety: 2017. National Children's Advocacy Center.Google Scholar
Neal, D. A., & Johnson, W. R. (1996). The role of premarket factors in Black–White wage differences. Journal of Political Economy, 104, 869–895.CrossRefGoogle Scholar
Nix, J., Campbell, B. A., Byers, E. H., & Alpert, G. P. (2017). A bird's eye view of civilians killed by police in 2015: Further evidence of implicit bias. Criminology & Public Policy, 16, 309–340.CrossRefGoogle Scholar
Nock, M. K., & Banaji, M. R. (2007). Prediction of suicide ideation and attempts among adolescents using a brief performance-based test. Journal of Consulting and Clinical Psychology, 75, 707–715.CrossRefGoogle ScholarPubMed
Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). Math = male, me = female, therefore math ≠ me. Journal of Personality and Social Psychology, 83(1), 44–59.CrossRefGoogle ScholarPubMed
Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The implicit association test at age 7: A methodological and conceptual review. Automatic Processes in Social Thinking and Behavior, 4, 265–292.Google Scholar
Nosek, B. A., & Smyth, F. L. (2011). Implicit social cognitions predict sex differences in math engagement and achievement. American Educational Research Journal, 48, 1125–1156.CrossRefGoogle Scholar
Nosek, B. A., Smyth, F. L., Sriram, N., Lindner, N. M., Devos, T., Ayala, A., … Greenwald, A. G. (2009). National differences in gender-science stereotypes predict national sex differences in science and math achievement. Proceedings of the National Academy of Sciences, 106, 10593–7.CrossRefGoogle ScholarPubMed
Okonofua, J. A., & Eberhardt, J. L. (2015). Two strikes: Race and the disciplining of young students. Psychological Science, 26, 617–624.CrossRefGoogle ScholarPubMed
Okonofua, J. A., Walton, G. M., & Eberhardt, J. L. (2016). A vicious cycle: A social–psychological account of extreme racial disparities in school discipline. Perspectives on Psychological Science, 11, 381–398.CrossRefGoogle ScholarPubMed
Oswald, F. L., Mitchell, G., Blanton, H., Jaccard, J., & Tetlock, P. E. (2013). Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies. Journal of Personality and Social Psychology, 105, 171–192.CrossRefGoogle ScholarPubMed
Oswald, F. L., Mitchell, G., Blanton, H., Jaccard, J., & Tetlock, P. E. (2015). Using the IAT to predict ethnic and racial discrimination: Small effect sizes of unknown societal significance. Journal of Personality and Social Psychology, 108, 562–571.CrossRefGoogle ScholarPubMed
Palfai, T. P., & Ostafin, B. D. (2003). Alcohol-related motivational tendencies in hazardous drinkers: Assessing implicit response tendencies using the modified-IAT. Behaviour Research and Therapy, 41, 1149–1162.CrossRefGoogle ScholarPubMed
Petty, R. E., & Wegener, D. T. (1999). The elaboration likelihood model; current status and controversies. In Chaiken, S. & Trope, Y. (Eds.), Dual process theories in social psychology (pp. 41–72). Guilford.Google Scholar
Plant, E. A., Goplen, J., & Kunstman, J. W. (2011). Selective responses to threat: The roles of race and gender in decisions to shoot. Personality and Social Psychology Bulletin, 37, 1274–1281.CrossRefGoogle Scholar
Plant, E. A., & Peruche, B. M. (2005). The consequences of race for police officers’ responses to criminal suspects. Psychological Science, 16, 180–183. doi: 10.1111/j.0956-7976.2005.00800.x.CrossRefGoogle ScholarPubMed
Platt, J. R. (1964). Strong inference. Science (New York, N.Y.), 146, 347–353.CrossRefGoogle ScholarPubMed
Pleskac, T. J., Cesario, J., & Johnson, D. J. (2018). How race affects evidence accumulation during the decision to shoot. Psychonomic Bulletin & Review, 25, 1301–1330. https://doi.org/10.3758/s13423-017-1369-6.CrossRefGoogle Scholar
Pleskac, T. J., & Hertwig, R. (2014). Ecologically rational choice and the structure of the environment. Journal of Experimental Psychology: General, 143, 2000–2019.CrossRefGoogle ScholarPubMed
Pleskac, T. J., Johnson, D. J., Cesario, J., Terrill, W., & Gagnon, G. (under review). Modeling police officers’ deadly force decisions in an immersive shooting simulator.Google Scholar
Price-Williams, D. R., & Ramirez, M. I. I. I. (1974). Ethnic differences in delay of gratification. The Journal of Social Psychology, 93, 23–30.CrossRefGoogle Scholar
Ring, K. (1967). Experimental social psychology: Some sober questions about some frivolous values. Journal of Experimental Social Psychology, 3, 113–123.CrossRefGoogle Scholar
Rocque, M., & Paternoster, R. (2011). Understanding the antecedents of the “school-to-jail” link: The relationship between race and school discipline. The Journal of Criminal Law and Criminology, 101, 633–665.Google Scholar
Rooth, D. (2007). Implicit discrimination in hiring: Real world evidence (IZA Discussion Paper No. 2764). Bonn, Germany: Forschungsinstitut zur Zukunft der Arbeit (Institute for the Study of Labor).Google Scholar
Ross, C. T., Winterhalder, B., & McElreath, R. (2021). Racial disparities in police use of deadly force against unarmed individuals persist after appropriately benchmarking shooting data on violent crime rates. Social Psychological and Personality Science, 12, 323–332.CrossRefGoogle Scholar
Roth, P. L., Bevier, C. A., Bobko, P., Switzer, F. S. III, & Tyler, P. (2001). Ethnic group differences in cognitive ability in employment and educational settings: A meta-analysis. Personnel Psychology, 54, 297–330.CrossRefGoogle Scholar
Rozin, P. (2009). What kind of empirical research should we publish, fund, and reward?: A different perspective. Perspectives on Psychological Science, 4, 435–439.CrossRefGoogle ScholarPubMed
Rudman, L. A., & Ashmore, R. D. (2007). Discrimination and the implicit association test. Group Processes & Intergroup Relations, 10, 359–372.CrossRefGoogle Scholar
Rudman, L. A., & Glick, P. (2001). Prescriptive gender stereotypes and backlash toward agentic women. Journal of Social Issues, 57, 743–762.CrossRefGoogle Scholar
Schimmack, U. (2020). The implicit association test: A method in search of a construct. Perspectives on Psychological Science, 16, 396–414.Google Scholar
Scott, K., Ma, D. S., Sadler, M. S., & Correll, J. (2017). A social scientific approach toward understanding racial disparities in police shooting: Data from the department of justice (1980–2000). Journal of Social Issues, 73, 701–722.CrossRefGoogle Scholar
Shjarback, J. A., & Nix, J. (2020). Considering violence against police by citizen race/ethnicity to contextualize representation in officer-involved shootings. Journal of Criminal Justice, 66, 101653-63.CrossRefGoogle Scholar
Sim, J. J., Correll, J., & Sadler, M. S. (2013). Understanding police and expert performance: When training attenuates (vs. exacerbates) stereotypic bias in the decision to shoot. Personality and Social Psychology Bulletin, 39, 291–304. doi: 10.1177/0146167212473157.CrossRefGoogle Scholar
Skiba, R. J., Michael, R. S., Nardo, A. C., & Peterson, R. L. (2002). The color of discipline: Sources of racial and gender disproportionality in school punishment. The Urban Review, 34, 317–342.CrossRefGoogle Scholar
Smith, E. R., & DeCoster, J. (2000). Dual-process models in social and cognitive psychology: Conceptual integration and links to underlying memory systems. Personality and Social Psychology Review, 4, 108–131.CrossRefGoogle Scholar
Sowell, T. (2005). Black rednecks and white liberals. Encounter Books.Google Scholar
Sowell, T. (2008). Discrimination and disparities. Basic Books.Google Scholar
Srivastava, S. (2016). Everything is fucked: The syllabus [Blog post]. Retrieved from https://thehardestscience.com/2016/08/11/everything-is-fucked-the-syllabus/.Google Scholar
Stanley, D. A., Sokol-Hessner, P., Banaji, M. R., & Phelps, E. A. (2011). Implicit race attitudes predict trustworthiness judgments and economic trust decisions. Proceedings of the National Academy of Sciences, 108, 7710–7715.CrossRefGoogle ScholarPubMed
Stickle, B. (2016). A national examination of the effect of education, training and pre-employment screening on law enforcement use of force. Justice Policy Journal, 13, 1–15.Google Scholar
Su, R., & Rounds, J. (2015). All STEM fields are not created equal: People and things interests explain gender disparities across STEM fields. Frontiers in Psychology, 6, 1–20.CrossRefGoogle Scholar
Su, R., Rounds, J., & Armstrong, P. I. (2009). Men and things, women and people: A meta-analysis of sex differences in interests. Psychological Bulletin, 135, 859–884.CrossRefGoogle ScholarPubMed
Taylor, A. (2011). The influence of target race on split-second shooting decisions in simulated scenarios :a Canadian perspective. (Doctor of Philosophy, Carleton University, Ottawa, Ontario). doi: 10.22215/etd/2011-09571.CrossRefGoogle Scholar
Tetlock, P. E. (1985). Accountability: The neglected social context of judgment and choice. Research in Organizational Behavior, 7, 297–332.Google Scholar
Tregle, B., Nix, J., & Alpert, G. P. (2019). Disparity does not mean bias: Making sense of observed racial disparities in fatal officer-involved shootings with multiple benchmarks. Journal of Crime and Justice, 42, 18–31.CrossRefGoogle Scholar
Uhlmann, E. L., Brescoll, V. L., & Paluck, E. L. (2006). Are members of low status groups perceived as bad, or badly off? Egalitarian negative associations and automatic prejudice. Journal of Experimental Social Psychology, 42, 491–499.CrossRefGoogle Scholar
Valla, J. M., Ceci, S. J. (2014). Breadth-based models of women's underrepresentation in STEM fields: An integrative commentary on Schmidt (2011) and Nye et al. (2012). Perspectives on Psychological Science, 9, 219–224.CrossRefGoogle Scholar
von Hippel, W., Brener, L., & von Hippel, C. (2008). Implicit prejudice toward injecting drug users predicts intentions to change jobs among drug and alcohol nurses. Psychological Science, 19, 7–11.CrossRefGoogle ScholarPubMed
Wheeler, A. P., Phillips, S. W., Worrall, J. L., & Bishopp, S. A. (2017). What factors influence an officer's decision to shoot? The promise and limitations of using public data. Justice Research and Policy, 18, 48–76. doi: 10.1177/1525107118759900.CrossRefGoogle Scholar
Wittenbrink, B., Judd, C. M., & Park, B. (2001). Spontaneous prejudice in context: Variability in automatically activated attitudes. Journal of Personality and Social Psychology, 81, 815–827.CrossRefGoogle ScholarPubMed
Worrall, J. L., Bishopp, S. A., Zinser, S. C., Wheeler, A. P., & Phillips, S. W. (2018). Exploring bias in police shooting decisions with real shoot/don't shoot cases. Crime Delinquency, 64, 1171–1192. doi: 10.1177/0011128718756038.CrossRefGoogle Scholar
Wright, J. P., Morgan, M. A., Coyne, M. A., Beaver, K. M., & Barnes, J. (2014). Prior problem behavior accounts for the racial gap in school suspensions. Journal of Criminal Justice, 42, 257–266.CrossRefGoogle Scholar
Zytkoskee, A., Strickland, B. R., & Watson, J. (1971). Delay of gratification and internal versus external control among adolescents of low socioeconomic status. Developmental Psychology, 4, 93–98.CrossRefGoogle Scholar