Hostname: page-component-5b777bbd6c-2c8nx Total loading time: 0 Render date: 2025-06-19T14:27:01.130Z Has data issue: false hasContentIssue false
  • English
  • Français

Transparency in Complex Computational Systems

Published online by Cambridge University Press:  01 January 2022

Kathleen A. Creel
Get access Rights & Permissions [Opens in a new window]

Abstract

Scientists depend on complex computational systems that are often ineliminably opaque, to the detriment of our ability to give scientific explanations and detect artifacts. Some philosophers have suggested treating opaque systems instrumentally, but computer scientists developing strategies for increasing transparency are correct in finding this unsatisfying. Instead, I propose an analysis of transparency as having three forms: transparency of the algorithm, the realization of the algorithm in code, and the way that code is run on particular hardware and data. This targets the transparency most useful for a task, avoiding instrumentalism by providing partial transparency when full transparency is impossible.

Type
Research Article
Information
Philosophy of Science , Volume 87 , Issue 4 , October 2020 , pp. 568 - 589
Copyright
Copyright © The Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

†

I am grateful for helpful comments from and discussions with Holly Andersen, Robert Batterman, Nora Mills Boyd, Liam Kofi Bright, Mazviita Chirimuuta, Roger Creel, Javier Duarte, Mahi Hardalupas, Paul Humphreys, Benjamin Jantzen, Johannes Lenhard, Sabina Leonelli, Jake Levinson, Edouard Machery, Sandra Mitchell, Elinor Nichols, Kathleen Nichols, Aaron Novick, Olivia Ordoñez, William Penn, Rebecca Traber, Porter Williams, Eric Winsberg, and two anonymous reviewers. Thanks also to generous audiences at Philosophical Perspectives on Data-Intensive Science in Hannover; Models and Simulations 8 in Columbia, SC; the Machine Learning Workshop in Irvine, CA; and Science and Art of Simulation IV in Stuttgart.

References

Ananny, Mike, and Crawford, Kate. 2018. “Seeing without Knowing: Limitations of the Transparency Ideal and Its Application to Algorithmic Accountability.” New Media and Society 20 (3): 973–89..CrossRefGoogle Scholar
Angwin, Julia, Larson, Jeff, Mattu, Surya, and Kirchner, Lauren. 2016. “Machine Bias: There’s Software Used across the Country to Predict Future Criminals; And It’s Biased against Blacks.” ProPublica, May 23. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.Google Scholar
Buckner, Cameron. 2019. “Deep Learning: A Philosophical Introduction.” Philosophy Compass 14 (10): e12625.CrossRefGoogle Scholar
Burrell, Jenna. 2016. “How the Machine ‘Thinks’: Understanding Opacity in Machine Learning Algorithms.” Big Data and Society. https://doi.org/10.1177/2053951715622512.CrossRefGoogle Scholar
Caruana, Rich, Lou, Yin, Gehrke, Johannes, Koch, Paul, Sturm, Marc, and Elhadad, Noemie. 2015. “Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-Day Readmission.” In KDD ’15: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1721–30. New York: Association for Computing Machinery.Google Scholar
Castelvecchi, Davide. 2015. “Artificial Intelligence Called in to Tackle LHC Data Deluge.” Nature News 528 (7580): 18–19..Google ScholarPubMed
Chouldechova, Alexandra. 2017. “Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments.” Big Data 5 (2): 153–63..CrossRefGoogle ScholarPubMed
Clark, Andy. 2013. Mindware. Oxford: Oxford University Press.Google Scholar
Datta, Amit, Tschantz, Michael Carl, and Datta, Anupam. 2015. “Automated Experiments on Ad Privacy Settings.” Proceedings on Privacy Enhancing Technologies 1:92–112.CrossRefGoogle Scholar
Dobbs, Matt, Halpern, Mark, Irwin, Kent D., Lee, Adrian T., Mates, J. A. B., and Mazin, Benjamin. 2009. “Multiplexed Readout of CMB Polarimeters.” Journal of Physics: Conference Series 155 (1): 012004.Google Scholar
Dosovitskiy, A., and Brox, T.. 2016. “Inverting Visual Representations with Convolutional Networks.” In Proceedings, 30th IEEE Conference on Computer Vision and Pattern Recognition, 4829–37. Los Alamitos, CA: IEEE Computer Society.Google Scholar
Duarte, Javier, et al. 2018. “Fast Inference of Deep Neural Networks in FPGAs for Particle Physics.” Journal of Instrumentation 13 (7): P07027.CrossRefGoogle Scholar
Esteva, Andre, Kuprel, Brett, Novoa, Roberto A., Ko, Justin, Swetter, Susan M., Blau, Helen M., and Thrun, Sebastian. 2017. “Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks.” Nature 542 (7639): 115–18..CrossRefGoogle ScholarPubMed
Fair, Ray C. 1978. “The Effect of Economic Events on Votes for President.” Review of Economics and Statistics 60 (2): 159–73..CrossRefGoogle Scholar
Fink, Katherine. 2018. “Opening the Government’s Black Boxes: Freedom of Information and Algorithmic Accountability.” Information, Communication and Society 21 (10): 1453–71..CrossRefGoogle Scholar
Glennan, Stuart. 2002. “Rethinking Mechanistic Explanation.” Philosophy of Science 69 (Proceedings): S342–S353.CrossRefGoogle Scholar
Goodman, Bryce, and Flaxman, Seth. 2017. “EU Regulations on Algorithmic Decision-Making and a ‘Right to Explanation.’” AI Magazine, Fall.CrossRefGoogle Scholar
Harman, Gilbert, and Kulkarni, Sanjeev. 2011. An Elementary Introduction to Statistical Learning Theory. New York: Wiley.Google Scholar
Humphreys, Paul. 2004. Extending Ourselves: Computational Science, Empiricism, and Scientific Method. Oxford: Oxford University Press.CrossRefGoogle Scholar
Humphreys, Paul. 2009. “The Philosophical Novelty of Computer Simulation Methods.” Synthese 169 (3): 615–26..CrossRefGoogle Scholar
Khain, A., Rosenfeld, D., and Pokrovsky, A.. 2005. “Aerosol Impact on the Dynamics and Microphysics of Deep Convective Clouds.” Quarterly Journal of the Royal Meteorological Society 131 (611): 2639–63..CrossRefGoogle Scholar
Knuth, Donald E. 1977. “Algorithms.” Scientific American 236 (4): 63–81..CrossRefGoogle Scholar
Krause, E., et al. 2017. “Dark Energy Survey Year 1 Results: Multi-Probe Methodology and Simulated Likelihood Analyses.” ArXiv:1706.09359 [Astro-Ph], Cornell University. http://arxiv.org/abs/1706.09359.Google Scholar
Lenhard, Johannes. 2006. “Surprised by a Nanowire: Simulation, Control, and Understanding.” Philosophy of Science 73 (5): 605–16..CrossRefGoogle Scholar
Lenhard, Johannes, and Winsberg, Eric. 2010. “Holism, Entrenchment, and the Future of Climate Model Pluralism.” Studies in History and Philosophy of Modern Physics 41 (3): 253–62..CrossRefGoogle Scholar
Leonelli, Sabina. 2016. “Locating Ethics in Data Science: Responsibility and Accountability in Global and Distributed Knowledge Production Systems.” Philosophical Transactions of the Royal Society A 374 (2083): 20160122.Google ScholarPubMed
Leonelli, Sabina, Rappert, Brian, and Davies, Gail. 2017. Data Shadows: Knowledge, Openness, and Absence. Los Angeles: Sage.CrossRefGoogle Scholar
Lesgourgues, Julian, and Tram, Thomas. 2017. “CLASS: The Cosmic Linear Anisotropy Solving System.” class-code.net, March 25.Google Scholar
Lewis, Antony. 2017. “Python CAMB.” Code for Anisotropies in the Microwave Background (CAMB) 0.1.5.3 documentation, June 8. http://camb.readthedocs.io/en/latest/.Google Scholar
Lewis, Antony, and Challinor, Anthony. 2017. “Code for Anisotropies in the Microwave Background.” Documentation, January. http://camb.info/.Google Scholar
Lipton, Zachary C. 2016. “The Mythos of Model Interpretability.” Presented at the 2016 ICML Workshop on Human Interpretability in Machine Learning, New York, June 23. http://arxiv.org/abs/1606.03490.Google Scholar
Machamer, Peter, Darden, Lindley, and Craver, Carl F.. 2000. “Thinking about Mechanisms.” Philosophy of Science 67 (1): 1–25..CrossRefGoogle Scholar
Marr, David. 2010. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Miotto, Riccardo, Li, Li, Kidd, Brian A., and Dudley, Joel T.. 2016. “Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records.” Scientific Reports 6 (May): 26094.CrossRefGoogle Scholar
Mordvintsev, Alexander, Olah, Christopher, and Tyka, Mike. 2015. “Inceptionism: Going Deeper into Neural Networks.” Research Blog, June 17. https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html.Google Scholar
Ribeiro, Marco Tulio, Singh, Sameer, and Guestrin, Carlos. 2016. “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier.” In KDD ’16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–44. New York: Association for Computing Machinery.Google Scholar
Rosenblatt, Frank. 1958. “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain.” Psychological Review 65 (6): 386–408..CrossRefGoogle Scholar
Roxlo, Thomas, and Reece, Matthew. 2018. “Opening the Black Box of Neural Nets: Case Studies in Stop/Top Discrimination.” ArXiv:1804.09278, Cornell University. https://arxiv.org/abs/1804.09278.Google Scholar
Community, Scipy. 2017. “Integration (Scipy.Integrate).” SciPy v0.19.1 reference guide, June 21. https://docs.scipy.org/doc/scipy/reference/tutorial/integrate.html.Google Scholar
Simonyan, Karen, Vedaldi, Andrea, and Zisserman, Andrew. 2014. “Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.” Presented at the International Conference on Learning Representations, Banff, Canada.Google Scholar
S⊘rmo, Frode, Cassens, Jörg, and Aamodt, Agnar. 2005. “Explanation in Case-Based Reasoning: Perspectives and Goals.” Artificial Intelligence Review 24 (2): 109–43..Google Scholar
Tcheng, David K., Nayak, Ashwin K., Fowlkes, Charless C., and Punyasena, Surangi W.. 2016. “Visual Recognition Software for Binary Classification and Its Application to Spruce Pollen Identification.” PLoS ONE 11 (2): e0148879.CrossRefGoogle ScholarPubMed
Wagenknecht, Susann. 2014. “Opaque and Translucent Epistemic Dependence in Collaborative Scientific Practice.” Episteme 11 (4): 475–92..CrossRefGoogle Scholar
Weng, Stephen F., Reps, Jenna, Kai, Joe, Garibaldi, Jonathan M., and Qureshi, Nadeem. 2017. “Can Machine-Learning Improve Cardiovascular Risk Prediction Using Routine Clinical Data?” PLoS ONE 12 (4): e0174944.CrossRefGoogle ScholarPubMed
Winsberg, Eric B. 2010. Science in the Age of Computer Simulation. Chicago: Cambridge University Press.CrossRefGoogle Scholar