Hostname: page-component-cd9895bd7-hc48f Total loading time: 0 Render date: 2024-12-26T05:12:35.243Z Has data issue: false hasContentIssue false
  • English
  • Français

Novelty versus Replicability: Virtues and Vices in the Reward System of Science

Published online by Cambridge University Press:  01 January 2022

Felipe Romero
Get access Rights & Permissions [Opens in a new window]

Abstract

The reward system of science is the priority rule. The first scientist making a new discovery is rewarded with prestige, while second runners get little or nothing. Michael Strevens, following Philip Kitcher, defends this reward system, arguing that it incentivizes an efficient division of cognitive labor. I argue that this assessment depends on strong implicit assumptions about the replicability of findings. I question these assumptions on the basis of metascientific evidence and argue that the priority rule systematically discourages replication. My analysis leads us to qualify Kitcher and Strevens’s contention that a priority-based reward system is normatively desirable for science.

Type
Values in Science
Information
Philosophy of Science , Volume 84 , Issue 5 , December 2017 , pp. 1031 - 1043
Copyright
Copyright © The Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

I am grateful to Carl Craver, Frederick Eberhardt, Barton Moffatt, Anya Plutynski, Mark Povich, Jan Sprenger, and Roy Sorensen for comments on previous drafts. I also thank Teresa Ai, Thomas Boyer-Kassem, Matteo Colombo, John Doris, Julia Staffel, and Michael Strevens for helpful discussion. Previous versions of this work were presented at the American Philosophical Association Pacific Division meeting, San Francisco, April 2016, and the Philosophy of Science Association Biennial Meeting, Atlanta, November 2016. This research was partially supported by ERC grant 640638, “Making Scientific Inferences More Objective.”

References

Begley, C. Glenn, and Ellis, Lee M. 2012. “Drug Development: Raise Standards for Preclinical Cancer Research.” Nature 483:531–33.CrossRefGoogle ScholarPubMed
Bem, Daryl J. 2011. “Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect.” Journal of Personality and Social Psychology 100 (3): 407–25.CrossRefGoogle ScholarPubMed
Bem, Daryl J., Utts, Jessica, and Johnson, Wesley O. 2011. “Must Psychologists Change the Way They Analyze Their Data?” Journal of Personality and Social Psychology 101 (4): 716–19.CrossRefGoogle ScholarPubMed
Bones, Arina K. 2012. “We Knew the Future All Along: Scientific Hypothesizing Is Much More Accurate than Other Forms of Precognition—a Satire in One Part.” Perspectives on Psychological Science 7 (3): 307–9.CrossRefGoogle ScholarPubMed
Campbell, Keith E., and Jackson, Thomas T. 1979. “The Role and Need for Replication Research in Social Psychology.” Replications in Social Psychology 1 (1): 3–14.Google Scholar
Cartwright, Nancy. 1991. “Replicability, Reproducibility, and Robustness: Comments on Harry Collins.” History of Political Economy 23 (1): 143–55.CrossRefGoogle Scholar
Chen, Xiang. 1994. “The Rule of Reproducibility and Its Applications in Experiment Appraisal.” Synthese 99 (1): 87–109.CrossRefGoogle Scholar
Evanschitzky, Heiner, Baumgarth, Carsten, Hubbard, Raymond, and Armstrong, J. Scott. 2007. “Replication Research’s Disturbing Trend.” Journal of Business Research 60 (4): 411–15.CrossRefGoogle Scholar
Fanelli, Daniele. 2009. “How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-analysis of Survey Data.” PLoS ONE 4 (5): 1–11.CrossRefGoogle ScholarPubMed
Fanelli, Daniele 2010. “Positive Results Increase Down the Hierarchy of the Sciences.” PLoS ONE 5 (4): e10068.CrossRefGoogle ScholarPubMed
Fanelli, Daniele 2012. “Negative Results Are Disappearing from Most Disciplines and Countries.” Scientometrics 90 (3): 891–904.CrossRefGoogle Scholar
Fisher, Ronald A. 1926. “The Arrangement of Field Experiments.” Journal of the Ministry of Agriculture 33:503–13.Google Scholar
Galak, Jeff, LeBoeuf, Robyn A., Nelson, Leif D., and Simmons, Joseph P. 2012. “Correcting the Past: Failures to Replicate Psi.” Journal of Personality and Social Psychology 103 (6): 933–48.CrossRefGoogle ScholarPubMed
Heisenberg, Werner. 1975. “The Great Tradition: End of an Epoch?” Encounter 44 (3): 52–58.Google Scholar
Hull, D. L. 1988. Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science. Science and Its Conceptual Foundations. Chicago: University of Chicago Press.CrossRefGoogle Scholar
John, Leslie K., Loewenstein, George, and Prelec, Drazen. 2012. “Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.” Psychological Science 23 (5): 524–32.CrossRefGoogle ScholarPubMed
Kerr, Norbert L. 1998. “HARKing: Hypothesizing after the Results Are Known.” Personality and Social Psychology Review 2 (3): 196–217.CrossRefGoogle ScholarPubMed
Kitcher, Philip. 1990. “The Division of Cognitive Labor.” Journal of Philosophy 87 (1): 5–22.CrossRefGoogle Scholar
Kitcher, Philip 1993. The Advancement of Science: Science without Legend, Objectivity without Illusions. Oxford: Oxford University Press.Google Scholar
Longino, Helen. 2015. “The Social Dimensions of Scientific Knowledge.” In Stanford Encyclopedia of Philosophy, ed. Zalta, Edward N. Stanford, CA: Stanford University. https://plato.stanford.edu/entries/scientific-knowledge-social/.Google Scholar
Machery, Edouard. 2012. “Power and Negative Results.” Philosophy of Science 79 (5): 808–20.CrossRefGoogle Scholar
Makel, Matthew C., and Plucker, Jonathan A. 2014. “Facts Are More Important than Novelty: Replication in the Education Sciences.” Educational Researcher 43 (6): 304–16.CrossRefGoogle Scholar
Makel, Matthew C., Plucker, Jonathan A., and Hegarty, Boyd. 2012. “Replications in Psychology Research: How Often Do They Really Occur?” Perspectives on Psychological Science 7 (6): 537–42.CrossRefGoogle ScholarPubMed
Merton, Robert K. 1957. “Priorities in Scientific Discovery: A Chapter in the Sociology of Science.” American Sociological Review 22 (6): 635–59.CrossRefGoogle Scholar
Muldoon, Ryan, and Weisberg, Michael. 2011. “Robustness and Idealization in Models of Cognitive Labor.” Synthese 183 (2): 161–74.CrossRefGoogle Scholar
Neuliep, James W., and Crandall, Rick. 1990. “Editorial Bias against Replication Research.” Journal of Social Behavior and Personality 5 (4): 85–90.Google Scholar
Neuliep, James W., and Crandall, Rick 1993. “Reviewer Bias against Replication Research.” Journal of Social Behavior and Personality 8 (6): 21–29.Google Scholar
Collaboration, Open Science. 2015. “Estimating the Reproducibility of Psychological Science.” Science 349 (6251): aac4716.CrossRefGoogle Scholar
Pashler, Harold, and Harris, Christine R. 2012. “Is the Replicability Crisis Overblown? Three Arguments Examined.” Perspectives on Psychological Science 7 (6): 531–36.CrossRefGoogle ScholarPubMed
Popper, Karl R. 1959/1959. The Logic of Scientific Discovery. Classics Series. London: Routledge.Google Scholar
Ritchie, Stuart J., Wiseman, Richard, and French, Christopher C. 2012a. “Failing the Future: Three Unsuccessful Attempts to Replicate Bem’s ‘Retroactive Facilitation of Recall’ Effect.” PLoS ONE 7 (3): e33423.CrossRefGoogle Scholar
Wiseman, Richard, and French, Christopher C. 2012b. “Replication, Replication, Replication.” Psychologist 25:346–57.Google Scholar
Rosenthal, Robert. 1979. “The File Drawer Problem and Tolerance for Null Results.” Psychological Bulletin 86 (3): 638–41.CrossRefGoogle Scholar
Rouder, Jeffrey N., and Morey, Richard D. 2011. “A Bayes Factor Meta-analysis of Bem’s ESP Claim.” Psychonomic Bulletin and Review 18 (4): 682–89.CrossRefGoogle ScholarPubMed
Schmidt, Stefan. 2009. “Shall We Really Do It Again? The Powerful Concept of Replication Is Neglected in the Social Sciences.” Review of General Psychology 13 (2): 90–100.CrossRefGoogle Scholar
Scotchmer, Suzanne. 2004. Innovation and Incentives. Cambridge, MA: MIT Press.Google Scholar
Simmons, Joseph P., Nelson, Leif D., and Simonsohn, Uri. 2011. “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.” Psychological Science 22 (11): 1359–66.CrossRefGoogle ScholarPubMed
Simonsohn, Uri. 2015. “Small Telescopes: Detectability and the Evaluation of Replication Results.” Psychological Science 26 (5): 559–69.CrossRefGoogle ScholarPubMed
Smith, Nathaniel C. 1970. “Replication Studies: A Neglected Aspect of Psychological Research.” American Psychologist 25 (10): 970–75.CrossRefGoogle Scholar
Stephan, Paula E. 2012. How Economics Shapes Science. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Strevens, Michael. 2003. “The Role of the Priority Rule in Science.” Journal of Philosophy 100 (2): 55–79.CrossRefGoogle Scholar
Strevens, Michael 2011. “Economic Approaches to Understanding Scientific Norms.” Episteme 8 (2): 184–200.CrossRefGoogle Scholar
Wagenmakers, Eric-Jan, Wetzels, Ruud, Borsboom, Denny, and van der Maas, Han L. J. 2011. “Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi.” Journal of Personality and Social Psychology 100 (3): 426–32.CrossRefGoogle ScholarPubMed
Winerman, Lea. 2013. “Interesting Results: Can They Be Replicated?” Monitor on Psychology 44 (2): 38–41.Google Scholar
Yarkoni, Tal. 2011. “The Psychology of Parapsychology; or, Why Good Researchers Publishing Good Articles in Good Journals Can Still Get It Totally Wrong.” Blog post, January 10. http://tinyurl.com/694ycam.Google Scholar