No CrossRef data available.
Article contents
The “‘Crisis’ Crisis” in psychology
Published online by Cambridge University Press: 10 February 2022
Abstract
The recent trend to label dilemmas in psychology as “crises” is insidious. The “‘Crisis’ Crisis” in psychology can distract us from actionable practices. As a case in point, “The Generalizability Crisis” offers the valuable central thesis that verbal-quantitative gaps imperil psychological science. Focusing on the key issues rather than crisis narratives can lead to progress in our discourse and research.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2022. Published by Cambridge University Press
References
Amrhein, V., Trafimow, D., & Greenland, S. (2019). Inferential statistics as descriptive statistics: There is no replication crisis if we don't expect replication. The American Statistician, 73(S1), 262–270.CrossRefGoogle Scholar
Bourne, H. R. (2018). Opinion: Expansion fever and soft money plague the biomedical research enterprise. Proceedings of the National Academy of Sciences, 115(35), 8647–8651.CrossRefGoogle ScholarPubMed
Chopik, W. J., Bremner, R. H., Defever, A. M., & Keller, V. N. (2018). How (and whether) to teach undergraduates about the replication crisis in psychological science. Teaching of Psychology, 45(2), 158–163.CrossRefGoogle Scholar
Dictionary, C. (n.d.). Crisis. In dictionary.cambridge.org dictionary. Retrieved February 13, 2020 from https://dictionary.cambridge.org/us/dictionary/english/crisis.Google Scholar
Dworkin, J. D., Linn, K. A., Teich, E. G., Zurn, P., Shinohara, R. T., & Bassett, D. S. (2020). The extent and drivers of gender imbalance in neuroscience reference lists. Nature Neuroscience, 23(8), 918–926.CrossRefGoogle ScholarPubMed
Fanelli, D. (2018). Opinion: Is science really facing a reproducibility crisis, and do we need it to? Proceedings of the National Academy of Sciences, 115(11), 2628–2631.CrossRefGoogle ScholarPubMed
Gelman, A., & Loken, E. (2016). The statistical crisis in science. In Pitici, M. (Ed.), The best writing on mathematics (pp. 305–318). Princeton University Press.Google Scholar
Kell, D. B., & Oliver, S. G. (2004). Here is the evidence, now what is the hypothesis? The complementary roles of inductive and hypothesis-driven science in the post-genomic era. Bioessays, 26(1), 99–105.CrossRefGoogle ScholarPubMed
King, M. M., Bergstrom, C. T., Correll, S. J., Jacquet, J., & West, J. D. (2017). Men set their own cites high: Gender and self-citation across fields and over time. Socius, 3, 2378023117738903.CrossRefGoogle Scholar
Lakens, D., Hilgard, J., & Staaks, J. (2016). On the reproducibility of meta-analyses: Six practical recommendations. BMC Psychology, 4(1), 1–10.CrossRefGoogle ScholarPubMed
Lilienfeld, S. O. (2017). Psychology's replication crisis and the grant culture: Righting the ship. Perspectives on Psychological Science, 12(4), 660–664.CrossRefGoogle Scholar
Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science, 355(6325), 584–585.CrossRefGoogle ScholarPubMed
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487.CrossRefGoogle ScholarPubMed
Merriam-Webster. (n.d.). Crisis. In Merriam-webster.com dictionary. Retrieved February 13, 2020 from https://www.merriam-webster.com/dictionary/crisis.Google Scholar
Moed, H. F., Colledge, L., Reedijk, J., Moya-Anegon, F., Guerrero-Bote, V., Plume, A., & Amin, M. (2012). Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way. Scientometrics, 92(2), 367–376.CrossRefGoogle Scholar
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606.CrossRefGoogle ScholarPubMed
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Almenberg, A. D., … Vazire, S. (2021). Replicability, robustness, and reproducibility in psychological science. Retrieved from https://psyarxiv.com/ksfvq/.Google Scholar
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251).Google Scholar
Ruscio, J. (2016). Taking advantage of citation measures of scholarly impact: Hip hip h index!. Perspectives on Psychological Science, 11(6), 905–908.CrossRefGoogle ScholarPubMed
Shneiderman, B. (2016). Opinion: The dangers of faulty, biased, or malicious algorithms requires independent oversight. Proceedings of the National Academy of Sciences, 113(48), 13538–13540.CrossRefGoogle ScholarPubMed
Shrout, P. E., & Rodgers, J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69, 487–510.CrossRefGoogle ScholarPubMed
Soderberg, C. K., Errington, T., Schiavone, S. R., Bottesini, J. G., Thorn, F. S., Vazire, S., … Nosek, B. A. (2020). Research quality of registered reports compared to the traditional publishing model. Retrieved from https://osf.io/aj4zr/.Google Scholar
Trafimow, D. (2018). An a priori solution to the replication crisis. Philosophical Psychology, 31(8), 1188–1214.CrossRefGoogle Scholar
Wahls, W. P. (2018). Point of view: The NIH must reduce disparities in funding to maximize its return on investments from taxpayers. Elife, 7, e34965.CrossRefGoogle ScholarPubMed
Whitaker, K., & Guest, O. (2020). #bropenscience is broken science: Kirstie Whitaker and Olivia guest ask how open “open science” really is. The Psychologist, 33, 34–37.Google Scholar
Target article
The generalizability crisis
Related commentaries (38)
A crisis of generalizability or a crisis of constructs?
Addressing a crisis of generalizability with large-scale construct validation
An accelerating crisis: Metascience is out-reproducing psychological science
Causal analysis as a bridge between qualitative and quantitative research
Causal complexity demands community coordination
Citizen science can help to alleviate the generalizability crisis
Disentangling paradigm and method can help bring qualitative research to post-positivist psychology and address the generalizability crisis
Exposing and overcoming the fixed-effect fallacy through crowd science
From description to generalization, or there and back again
Generalizability challenges in applied psychological and organizational research and practice
Generalizability in mixed models: Lessons from corpus linguistics
Generalizability, transferability, and the practice-to-practice gap
Impact on the legal system of the generalizability crisis in psychology
Improving the generalizability of infant psychological research: The ManyBabies model
Increasing generalizability via the principle of minimum description length
Is formalism the key to resolving the generalizability crisis? An experimental economics perspective
Lessons from behaviorism: The problem of construct-led science
Look to the field
Measurement practices exacerbate the generalizability crisis: Novel digital measures can help
Mechanistic modeling for the masses
Mismatch between scientific theories and statistical models
Observing effects in various contexts won't give us general psychological theories
Psychologists should learn structural specification and experimental econometrics
Publishing fast and slow: A path toward generalizability in psychology and AI
Random effects won't solve the problem of generalizability
Science with or without statistics: Discover-generalize-replicate? Discover-replicate-generalize?
Separate substantive from statistical hypotheses and treat them differently
The cost of crisis in clinical psychological science
The crisis from above: Gatekeepers need better standards
The four different modes of psychological explanation, and their proper evaluative schemas
The role of generalizability in moral and political psychology
The stimulus-response crisis
The “‘Crisis’ Crisis” in psychology
There is no generalizability crisis
There is no psychology without inferential statistics
We need to be braver about the generalizability crisis
We need to think more about how we conduct research
Without more theory, psychology will be a headless rider
Author response
Replies to commentaries on the generalizability crisis