On bodies, brains, and behaviour (and a little bit of magic)

Abstract

The impact the body has upon complex cognitive capabilities has long challenged cognitive scientists. Insights into the complex interplay between how we see, what we see, and how we interpret what we think we saw and remembered are offered by a surprising source: the effects magicians create.

Coombs and Trestman (C&T) name six “pivotal traits,” which correlate with complex cognitive capabilities across vertebrate, coleoid cephalopod mollusc, and euarthropod lineages. These traits are embodied and share a common application in high-resolution active sensing – the practice of directing motor outputs to obtain, or alter the sampling of, sensory information from the environment (Prescott, Diamond, & Wing, 2011). Compared to passive sensing, high-resolution active sensing requires increased computational resources (Ahmad, Huang, & Yu, 2014), and we agree with C&T’s position that these increased demands may have accelerated the emergence of complex cognitive capabilities.

However, the interplay between the sensorimotor system and complex cognition on a causal level remains elusive. Whilst adaptive filtering offers a possible neuroarchitecture for the relationship between distal sensors and complex cognition, how information is weighted in such networks is unknown, and the question of how a given agent’s body impacts its perception looms large (Barrett & Stout, 2024). We take the opportunity to discuss this topic, with a focus on two forms of sensory information facilitated by C&T’s pivotal traits – visual information and haptic feedback. In doing so, we find inspiration in an unlikely source: the effects produced by magicians.

Studies of magic have begun to reveal the inner workings of embodiment (Kuhn, Amlani, & Rensink, 2008; Garcia-Pelegrin et al., 2021; Garcia-Pelegrin et al., 2023; Rensink & Kuhn, 2015). Expert magicians make objects disappear before your eyes and balls multiply in your own hands – seemingly without you noticing. They are experts in misdirection, specifically in misdirecting active sensing, almost always with a visual component (Garcia-Pelegrin, Wilkins, & Clayton, 2022; Rensink & Kuhn, 2015). By pantomiming the cues of common biomechanical processes and utilising objects which are not all that they seem, a skilled magician uses audience’s expectations to amaze them. Whilst theatrical magic shows are a distinctly human affair, the efficacy of magic effects is not.

Comparative studies of magic have revealed that audiences of Eurasian jays and non-human primate species are successfully misdirected by sleight-of-hand magic effects, but only if the performed trick is tailored to the anatomy of the audience. A jay is only vulnerable to a trick in which the hand moves like a wing, and a marmoset is only fooled if the thumb is treated as non-opposable. Such studies represent evidence that species’ body plans are directly relevant in the weighting of integrated sensory information during the formation of mental representations and predictions (Garcia-Pelegrin et al., 2023, 2021; Garcia-Pelegrin et al., 2024).

The mental representation of objects is central to many magic effects (Garcia-Pelegrin et al., 2022). The utilisation of such representations in real-world settings depends on the ability to efficiently and accurately recognise objects across dynamic and non-standardised environments. C&T consider feature extraction critical to object-detection capabilities underpinning this process. Here we aim to extend the authors’ discussions by suggesting that encoding object-related features as discrete subunits is an efficient and effective method through which to generalise expectations across objects with shared characteristics, highlighting that these expectations are likely influenced by the types of features that can be mapped by an agent’s sensory system (Summerfield & De Lange, 2014; Wurm et al., 1993).

In the fork-twisting effect (Figure 1), a magician holds a solid steel fork in one hand and appears to effortlessly twist the head so that it is now facing in the opposite direction to the handle, ultimately revealing the twisted metal handle and astounding the audience. However, this effect would be much less impactful if the fork was made of an easily deformable material, such as silicone. The effect works because the audience visually recognise a steel fork, represent its component properties, and form expectations and predictions based upon them: they detect “steel” as a property, expect that steel is difficult to deform, and predict that twisting the fork would be effortful – perhaps informed by past experience of tactile interactions with steel objects (Friston, 2010; Prescott et al., 2011; Stevens, 2022).

Figure 1. Twisting fork effect (see text for further explanation).

Now, imagine the audience has a high-resolution visual system, but lacks analogous sensors to those which collect the low-level spatio-chromatic data utilised by the human visual system when discerning between metallic and non-metallic surfaces (Harvey & Smithson, 2021). The fork may still be recognised by its other features, but the expectations formed around its behaviour could be very different. In this way, magic effects become a helpful testbed to explore how the sensing capabilities of a given species may influence its recognition of discrete object subunits and impact the higher-level predictions formed around objects.

Constructing mental representations of objects from encoded features supports the generalisation of object-level expectations across multiple contexts based on object-similarities – an approach that has gained success in deep-learning models (Lake, Salakhutdinov, & Tenenbaum, 2015; Hebart et al., 2020). These expectations allow agents to form mental predictions for how objects will behave in space (for example, that steel will not twist without significant force), and may provide the basis on which novel features are mapped as the predictions formed on generalised near-neighbour traits are contra-indicated in real-time (Friston, 2010; Ranzato et al., 2007).

The types of integrated sensorimotor systems described by C&T allow agents to test their predictions in real-time, and facilitate learning and causal reasoning surrounding self-environment interactions as agents dynamically observe the impact of their actions on the environment (Figure 2); by attempting to twist a steel fork, an agent can learn the forces required as part of a visuomotor feedback loop (Jelbert et al., 2019; Pfeifer, Lungarella, & Iida, 2007). Combined with information generalised from stored object-based representations, such capacities may contribute to successful problem-solving in unfamiliar environments – a hallmark of complex behaviours such as tool-use, and a significant challenge in robotics (Yang et al., 2018).

Figure 2. A schematic diagram to illustrate how integrated sensorimotor systems may support learning in self-environment interactions, using the example of a steel fork.

As such, we support C&T’s perspective that key traits underpinning an integrated visuomotor feedback loop may promote the emergence of complex cognitive behaviours, and conclude by suggesting that future directions could utilise magic paradigms to investigate the role of such traits in object-related predictions.