Hostname: page-component-cd9895bd7-lnqnp Total loading time: 0 Render date: 2024-12-27T13:55:31.462Z Has data issue: false hasContentIssue false
  • English
  • Français

Prestige does not equal quality: Lack of research quality in high-prestige journals

Published online by Cambridge University Press:  11 November 2020

Tine Köhler ,
Justin A. DeSimone  and
Jeremy L. Schoen
Tine Köhler*
Affiliation:
The University of Melbourne
Justin A. DeSimone
Affiliation:
The University of Alabama
Jeremy L. Schoen
Affiliation:
The University of Auckland
*
*Corresponding author. Email: tkoehler@unimelb.edu.au
Get access Rights & Permissions [Opens in a new window]

Abstract

An abstract is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'
Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 13 , Issue 3 , September 2020 , pp. 321 - 327
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of the Society for Industrial and Organizational Psychology.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

All authors contributed equally to the work on this paper.

References

Addo, C. K. (2017). Assessing the assessor: Using journal source as proxy for quality of article damaging scholars’ career. International Journal of Education and Evaluation, 3(8), 1–6.Google Scholar
Aguinis, H., Cummings, C., Ramani, R. S., & Cummings, T. (2019). “An A is an A:” The new bottom line for valuing academic research. Academy of Management Perspectives, 34(1), 135–154.10.5465/amp.2017.0193CrossRefGoogle Scholar
Aguinis, H., Ramani, R. S., & Alabduljader, N. (2018). What you see is what you get? Enhancing methodological transparency in management research. Academy of Management Annals, 12, 83–110.CrossRefGoogle Scholar
Banks, G. C., Field, J. G., Oswald, F. L., O’Boyle, E. H., Landis, R. S., Rupp, D. E., & Rogelberg, S. G. (2019). Answers to 18 questions about open science practices. Journal of Business and Psychology, 34, 257–270.CrossRefGoogle Scholar
Bar-Ilan, J., & Halevi, G. (2017). Post retraction citations in context: A case study. Scientometrics, 113, 547–565.CrossRefGoogle ScholarPubMed
Burchell, B., & Marsh, C. (1992). The effect of questionnaire length on survey response. Quality and Quantity, 26, 233–244.CrossRefGoogle Scholar
Camerer, C. F., Dreber, A., Forsell, E., Ho, T-H, Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., & Wu, H. (2016). Evaluating replicability of laboratory studies in economics. Science, 351, 1433–1436.CrossRefGoogle ScholarPubMed
Chandler, J. J., & Paolacci, G. (2017). Lie for a dime: When most prescreening responses are honest but most study participants are imposters. Social Psychological and Personality Science, 8, 500–508.CrossRefGoogle Scholar
Christensen, G., & Miguel, E. (2018). Transparency, reproducibility, and the credibility of economics research. Journal of Economic Literature, 56, 920–980.CrossRefGoogle Scholar
Cortina, J. M., Aguinis, H., & DeShon, R. P. (2017). Twilight of dawn or of evening? A century of research methods in the Journal of Applied Psychology . Journal of Applied Psychology, 102, 274–290.CrossRefGoogle ScholarPubMed
Cortina, J. M., Green, J. P., Keeler, K. R., & Vandenberg, R. J. (2017). Degrees of freedom in SEM: Are we testing the models that we claim to test? Organizational Research Methods, 20, 350–378.CrossRefGoogle Scholar
Cortina, J. M., Sheng, Z., List, S., Katell, L., & Keeler, K. (2019). From α to Ω: Reliability assessment in JAP. In G. C. Banks (Chair), Scale adaptation and reliability in I-O research: The good, the bad, and the ugly [Symposium]. Society of Industrial and Organizational Psychology 34th Annual Conference National Harbor, MD, United States.Google Scholar
Credé, M., & Harms, P. D. (2015). 25 years of higher-order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations. Journal of Organizational Behavior, 36, 845–872.CrossRefGoogle Scholar
Harms, P. D., & DeSimone, J. A. (2015). Caution! MTurk workers ahead—fines doubled. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8, 183–190.CrossRefGoogle Scholar
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biology, 13, e1002106.CrossRefGoogle Scholar
Heggestad, E. D., Scheaf, D. J., Banks, G. C., Hausfeld, M. M., Tonidandel, S., & Williams, E. B. (2019). Scale adaptation in organizational science research: A review and best-practice recommendations. Journal of Management, 45, 2596–2627.CrossRefGoogle Scholar
Highhouse, S., Zickar, M. J., & Melick, S. R. (2020). Prestige and relevance of the scholarly journals: Impressions of SIOP members. Industrial and Organizational Psychology: Perspectives on Science and Practice, 13(3), 273–290.Google Scholar
Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21, 967–988.CrossRefGoogle Scholar
Hunter, J. E., (2001). The desperate need for replications. Journal of Consumer Research, 28, 149–158.CrossRefGoogle Scholar
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124 CrossRefGoogle ScholarPubMed
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995). Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Controlled Clinical Trials, 16, 62–73.CrossRefGoogle ScholarPubMed
Morís Fernández, L., & Vadillo, M. A. (2019). Retracted papers die hard: Diederik Stapel and the enduring influence of flawed science [Unpublished manuscript].University Pompeu Fabra. https://doi.org/10.31234/osf.io/cszpy CrossRefGoogle Scholar
Murphy, K. R., & Aguinis, H. (2019). HARKing: How badly can cherry-picking and question trolling produce bias in published research? Journal of Business and Psychology, 34, 1–17.CrossRefGoogle Scholar
Neuroskeptic. (2012). The nine circles of scientific hell. Perspectives on Psychological Science, 7, 643–644.CrossRefGoogle Scholar
Nuzzo, R. (2014). Science method: Statistical errors. Nature, 506, 150–152.CrossRefGoogle ScholarPubMed
O’Boyle, E. H. Jr., Banks, G. C., & Gonzales-Mulé, E. (2017). The chrysalis effect: How ugly initial results metamorphosize into beautiful articles. Journal of Management, 43, 376–399.CrossRefGoogle Scholar
Open Science Collaboration (OSC) (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.Google Scholar
Porter, C. O. L. H., Outlaw, R., Gale, J. P., & Cho, T. S. (2019). The use of online panel data in management research: A review and recommendations. Journal of Management, 45, 319–344.CrossRefGoogle Scholar
Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.CrossRefGoogle Scholar
Rubin, M. (2017). When does HARKing hurt? Identifying when different types of undisclosed post hoc hypothesizing harm scientific progress. Review of General Psychology, 21, 308–320.CrossRefGoogle Scholar
Saylors, R., & Trafimow, D. (in press). Why the increasing use of complex causal models is a problem: On the danger sophisticated theoretical narratives pose to truth. Organizational Research Methods. Retrieved from https://journals.sagepub.com/doi/full/10.1177/1094428119893452 Google Scholar
Schoen, J. L., Köhler, T., DeSimone, J. A., Davison, H. K. (2019, April). Distrust and verify: Qualities of perceptual measures and measurement reporting in organizational research. In Banks, G. C. (Chair), Scale adaptation and reliability in I-O research: The good, the bad, and the ugly [Symposium]. Society of Industrial and Organizational Psychology 34th Annual Conference, National Harbor, MD, United States.Google Scholar
UTD (University of Texas at Dallas). (2020). The UTD Top 100 business school research rankings. https://jindal.utdallas.edu/the-utd-top-100-business-school-research-rankings/northRankings#20142018.Google Scholar
Wright, T. A., Quick, J. C., Hannah, S. T., & Hargrove, M. B. (2017). Best practice recommendations for scale construction in organizational research: The development and initial validation of the Character Strength Inventory. Journal of Organizational Behavior, 38, 615–628.CrossRefGoogle Scholar