Hostname: page-component-6dbcb7884d-t8hx9 Total loading time: 0 Render date: 2025-02-13T21:16:42.035Z Has data issue: false hasContentIssue false
  • English
  • Français

PAC Learning and Occam’s Razor: Probably Approximately Incorrect

Published online by Cambridge University Press:  01 January 2022

Daniel A. Herrmann
Get access Rights & Permissions [Opens in a new window]

Abstract

Computer scientists have provided a distinct justification of Occam’s Razor. Using the probably approximately correct framework, they provide a theorem that they claim demonstrates that we should favor simpler hypotheses. The argument relies on a philosophical interpretation of the theorem. I argue that the standard interpretation of the result in the literature is misguided and that a better reading does not, in fact, support Occam’s Razor at all. To this end, I state and prove a very similar theorem that, if interpreted the same way, would justify the contradictory Anti-Occam’s Razor—the principle that we should favor more complex hypotheses.

Type
Research Article
Information
Philosophy of Science , Volume 87 , Issue 4 , October 2020 , pp. 685 - 703
Copyright
Copyright © The Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

I want to thank Jan-Willem Romeijn and Tom F. Sterkenburg for invaluable early discussions and feedback. I also want to thank Simon Huttegger, Brian Skyrms, Darcy Otto, Aydin Mohseni, Gerard Rothfus, Saira Khan, Bruce Rushing, Tom Colclough, and two anonymous referees for helpful comments on earlier drafts.

References

Aaronson, Scott. 2013. “Why Philosophers Should Care about Computational Complexity.” In Computability: Turing, Gödel, Church, and Beyond, ed. Copeland, B. Jack, Posy, Carl J., and Shagrirj, Oron, 261–328. Cambridge, MA: MIT Press.Google Scholar
Blumer, Anselm, Ehrenfeucht, Andrzej, Haussler, David, and Warmuth, Manfred K.. 1987. “Occam’s Razor.” Information Processing Letters 24 (6): 377–80..CrossRefGoogle Scholar
Domingos, Pedro. 1999. “The Role of Occam’s Razor in Knowledge Discovery.” Data Mining and Knowledge Discovery 3 (4): 409–25..CrossRefGoogle Scholar
Kearns, Michael J., and Vazirani, Umesh Virkumar. 1994. An Introduction to Computational Learning Theory. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Kelly, Kevin T. 2004. “Justification as Truth-Funding Efficiency: How Ockham’s Razor Works.” Minds and Machines 14 (4): 485–505..CrossRefGoogle Scholar
Ming, Li, Tromp, John, and Vitányi, Paul. 2003. “Sharpening Occam’s Razor.” Information Processing Letters 85 (5): 267–74..Google Scholar
Ming, Li, and Vitányi, Paul. 1990. “Kolmogorov Complexity and Its Applications.” In Algorithms and Complexity, ed. Leeuwen, Jan van, 187–254. Amsterdam: Elsevier.Google Scholar
Moran, Shay, and Yehudayoff, Amir. 2015. “Proper PAC Learning Is Compressing.” Electronic Colloquium on Computational Complexity Report no. 40, University of Trier.Google Scholar
Romeijn, Jan-Willem. 2017. “Inherent Complexity: A Problem for Statistical Model Evaluation.” Philosophy of Science 84 (5): 797–809..CrossRefGoogle Scholar
Sober, Elliott. 2015. Ockham’s Razors. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Sterkenburg, Tom F. 2016. “Solomonoff Prediction and Occam’s Razor.” Philosophy of Science 83 (4): 459–79..CrossRefGoogle Scholar
Valiant, Leslie G. 1984. “A Theory of the Learnable.” Communications of the ACM 27 (11): 1134–42..CrossRefGoogle Scholar
Valiant, Leslie G.. 2013. Probably Approximately Correct: Nature’s Algorithms for Learning and Prospering in a Complex World. New York: Basic.Google Scholar
Wolf, R. M. de. 1997. “Philosophical Applications of Computational Learning Theory: Chomskyan Innateness and Occzam’s Razor.” Master’s thesis, Erasmus University.Google Scholar