On the hazards of relating representations and inductive biases

Abstract

The success of models of human behavior based on Bayesian inference over logical formulas or programs is taken as evidence that people employ a “language-of-thought” that has similarly discrete and compositional structure. We argue that this conclusion problematically crosses levels of analysis, identifying representations at the algorithmic level based on inductive biases at the computational level.

Over the last few decades probabilistic models of cognition, which explain human behavior in terms of Bayesian inference over a set of hypotheses, have been applied to a wide range of phenomena (Griffiths, Chater, Kemp, Perfors, & Tenenbaum, 2010). But what does the success of a particular probabilistic model in capturing human behavior imply? Probabilistic models of cognition are typically defined at Marr's (1982) “computational” level, characterizing the abstract problems human minds have to solve and their ideal solutions. More precisely, they characterize the ideal solutions to inductive problems, where an agent has to draw conclusions that go beyond the available data. The content of a probabilistic model of cognition, expressed via the set of hypotheses and their prior probabilities, is a claim about the inductive biases that guide such inferences – those factors other than the data that influence the hypothesis the agent selects (Mitchell, 1997). Here, we argue that drawing conclusions that go beyond these inductive biases – and in particular, inferring support for specific cognitive processes and representations – can be problematic.

Inductive biases are at a different level of analysis from cognitive processes and representations, which Marr (1982) located at the “representation and algorithm” level. Representations and algorithms are notoriously underdetermined by observable data (Anderson, 1978). This underdetermination motivated Anderson (1990) to develop rational analysis, the approach adopted in almost all applications of probabilistic models of cognition. This approach explicitly focuses on abstract problems and their ideal solutions rather than the processes and representations that implement them (inspiring critiques, e.g., Jones & Love, 2011). Probabilistic models need some way of representing hypotheses, but such representations do not necessarily guide human behavior. Rather, they are theoretical constructs that help scientists describe inductive biases.

Finding that a particular inductive bias seems to characterize human behavior places constraints on the representations and algorithms that might be involved, but those constraints rarely pick out a unique solution. To give a simple example, consider the problem of learning a linear relationship between two variables. A probabilistic model identifies a set of hypotheses (e.g., all linear functions), defines a prior distribution over those hypotheses, and then performs Bayesian inference. This kind of solution could be implemented by an agent that explicitly represents a set of linear functions and uses an algorithm to update its beliefs about the posterior probability of each hypothesis as new data are observed (see, e.g., Sanborn, Griffiths, & Navarro, 2010). The behavior of this agent will match that of the ideal Bayesian model. However, an agent that seems quite different – a neural network with one hidden layer and a linear output function that updates its weights by a few iterations of gradient descent – will also produce an answer that matches that Bayesian model (assuming a Gaussian prior; see Santos, 1996). Two very different representations and algorithms are consistent with the same computational-level account (for a real modeling example, see Feldman, Griffiths, & Morgan, 2009).

Quilty-Dunn et al. argue from the success of probabilistic models of cognition based on Bayesian inference over logical formulas and programs to the conclusion that people employ a similarly discrete and compositional “language-of-thought.” This argument crosses levels of analysis in the same problematic way. What we are licensed to conclude from the success of these models is that logical formulas and programs are useful in characterizing human inductive biases for certain problems, not that humans use these representations when solving those problems. Any stronger conclusion seems particularly problematic in light of the recent successes of deep neural networks that Quilty-Dunn et al. mention, because these systems may not require discrete or compositional representations. Metalearning – training a system to perform a set of related tasks – provides a way to create neural networks with specific inductive biases, and has formal connections to learning a prior for Bayesian inference (Grant, Finn, Levine, Darrell, & Griffiths, 2018). Metalearning has been used to train neural networks to perform tasks characterized at the computational level by Bayesian models based on symbolic representations, such as theory-of-mind (Rabinowitz et al., 2018) and causal learning (Dasgupta et al., 2019). Analysis of the internal representations of related systems shows that they contain information that can be used to reconstruct appropriate posterior distributions (Mikulik et al., 2020). It thus seems plausible that such systems might produce behavior that is just as consistent with Bayesian inference over logical formulas and programs as that of humans.

The possible existence of deep neural networks that can be analyzed at the computational level in terms of Bayesian inference blocks strong conclusions about the language-of-thought, as the representations learned by these networks could emulate the associated behavior without requiring discreteness. Our investigations of networks trained by metalearning show that they can emulate human performance on abstract tasks without explicit representations of the relevant abstractions (Kumar et al., 2022). In some cases, deep neural networks succeed on abstract tasks by learning compositionally structured representations (McCoy, Linzen, Dunbar, & Smolensky, 2019), but these representations remain continuous, making them importantly different from the inherently discrete ones postulated in the language-of-thought hypothesis (Smolensky, 1988). Such results align with theoretical work showing that compositional behavior does not require discrete representations (Smolensky, McCoy, Fernandez, Goldrick, & Gao, 2022). Indeed, the best current models of language itself – which is the prototypical example of a compositional domain (Pinker & Prince, 1988), as suggested by its use in the name language-of-thought – are deep networks that have continuous internal representations (e.g., Chowdhery et al., 2022).

Algorithms and representations may not be identifiable, but we can at least narrow down the equivalence class of possibilities through careful experimentation – behavioral work focused on response times and errors, neuroscientific studies of what the brain might be encoding, and computational simulations – designed to provide strong tests of alternative hypotheses. Until we can definitively do so, the fact that a discrete, compositional language-of-thought is useful as an abstract way of characterizing human inductive biases still allows the possibility that the actual representations and algorithms underlying human cognition may have a very different character.