Chapter 6 - Black Boxes, Beetles and Beasts

Summary

Introduction

For a class of artificial intelligence (AI) systems, answers to the question of why – for some senses of ‘why’ – the system made a particular ‘decision’ are, as a matter of constitutive fact, unavailable. This discussion raises some issues that face Wittgenstein-inflected philosophers when thinking about this so-called Black Box problem. It offers reasons for thinking that Wittgensteinians should be intensely relaxed about such AI systems.

Black boxes

The black box problem arises because systems of the relevant kind are supposed to be ‘opaque’ to certain questions. I might ask, for example, why a system refused to give me a credit card (Bary n.d.), recommended a longer prison sentence, changed lanes on the motorway, offered good news about a cancer tumour (Gregory 2022), or determined a protein's shape (Trager 2022). Of course, if I were to query the system owners, I might, perhaps after some hesitancy, be answered: ‘Well, because our system determined that the target, you or your circumstances met (or failed to meet) criteria A, B and C’. The black box problem arises because, for a class of AI systems, this answer is false.

For the systems which do not interest us here – symbolic, procedural or GOFAI – this answer will be true. The input – the data about me and my circumstances – is an argument for the function here – which is calculated in this object here and so on. There is some value of a variable which, at some level of abstraction, is understood as the representation of some fact about you or your circumstances. Call these domain homogeneous systems. For such systems, someone can, in principle, run through the same inferential steps as the machine, with or without two kinds of pebble and a long enough toilet roll (Weizenbaum 1976, Ch 2). The result is rationally recoverable.

The AI systems which interest us here are convolutional neural networks with a primary discriminative (rather than generative) function. They are focused on finding a particular solution to a problem rather than modelling the data. Convolutional neural networks – ConvNets – are, we can say, domain heterogeneous. If someone asks why a certain result was forthcoming, we can describe how the system works, values of hyper-parameters, weights and activation biases, gradient descents, back-propagation algorithms and so.