The originality of Glenberg's theoretical account lies in the claim that memory works in the service of physical interaction with the three-dimensional world. Little consideration is given, however, to the role of memory in action. We present and discuss data on spatial memory for action. These empirical data constitute the first step of reasoning about the link between memory and action, and allow several aspects of Glenberg's theory to be tested.

Although Glenberg's theory benefits from the incorporation of a suppression concept, a more differentiated view of suppression would be even more effective. We propose such a concept (based on the attention framework first developed by William James in the late nineteenth century), showing how it accounts for phenomena that Glenberg describes and also for phenomena that he ignores.

Glenberg offers two different accounts of embodied conceptualization. The first fails in cases where no direct bodily interaction is possible. The second fails in cases where the object in question cannot serve as a discriminative stimulus; moreover, it yields inappropriate content even in cases where it can be applied. Glenberg's disregard for the conceptual agenda set by the social world is also disquieting.

Contrary to prevailing views, productivity and propositional construal are not problematic for perceptual views of representation. Glenberg's embodied representations contribute to our understanding of how these two important processes might be implemented perceptually.

While sharing the author's interest in the development of an action-based framework for memory research, I think the present version is neither new nor particularly productive. More differentiation is needed to describe memory functioning in a variety of domains and on the many levels of activity regulation. Above all, Glenberg's proposals seem to contradict empirical data.

Glenberg's account falls short in several respects. Besides requiring clearer explication of basic concepts, his account fails to recognize the autonomous nature of perception. His account of what is remembered, and its description, is too static. His strictures against connectionist modeling might be overcome by combining the notions of psychological space and principled learning in an embodied and situated network.

I address the commentators' calls for clarification of theoretical terms, discussion of similarities to other proposals, and extension of the ideas. In doing so, I keep the focus on the purpose of memory: enabling the organism to make sense of its environment so that it can take action appropriate to constraints resulting from the physical, personal, social, and cultural situations.

Some regularities enjoy only an attenuated existence in a body of training data. These are regularities whose statistical visibility depends on some systematic recoding of the data. The space of possible recodings is, however, infinitely large – it is the space of applicable Turing machines. As a result, mappings that pivot on such attenuated regularities cannot, in general, be found by brute-force search. The class of problems that present such mappings we call the class of “type-2 problems.” Type-1 problems, by contrast, present tractable problems of search insofar as the relevant regularities can be found by sampling the input data as originally coded. Type-2 problems, we suggest, present neither rare nor pathological cases. They are rife in biologically realistic settings and in domains ranging from simple animat (simulated animal or autonomous robot) behaviors to language acquisition. Not only are such problems rife – they are standardly solved! This presents a puzzle. How, given the statistical intractability of these type-2 cases, does nature turn the trick? One answer, which we do not pursue, is to suppose that evolution gifts us with exactly the right set of recoding biases so as to reduce specific type-2 problems to (tractable) type-1 mappings. Such a heavy-duty nativism is no doubt sometimes plausible. But we believe there are other, more general mechanisms also at work. Such mechanisms provide general (not task-specific) strategies for managing problems of type-2 complexity. Several such mechanisms are investigated. At the heart of each is a fundamental ploy – namely, the maximal exploitation of states of representation already achieved by prior, simpler (type-1) learning so as to reduce the amount of subsequent computational search. Such exploitation both characterizes and helps make unitary sense of a diverse range of mechanisms. These include simple incremental learning (Elman 1993), modular connectionism (Jacobs et al. 1991), and the developmental hypothesis of “representational redescription” (Karmiloff-Smith 1979; 1992). In addition, the most distinctive features of human cognition – language and culture – may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale.

Clark & Thornton are too hasty in their dismissal of uninformed learning; nonmonotonic processing units show considerable promise on type-2 tasks. I describe a simulation which succeeds on a “pure” type-2 problem. Another simulation challenges Clark & Thornton's claims about the serendipitous nature of solutions to type-2 problems.

I suggest that the difficulties inherent in discovering the hidden regularities in realistic (type-2) problems can often be resolved by learning algorithms employing simple constraints (such as symmetry and the importance of local information) that are natural from an evolutionary point of view. Neither “heavy-duty nativism” nor “representational recoding” appear to offer totally appropriate descriptions of such natural learning processes.

Clark & Thornton's type-1/-2 distinction is not well-defined. The classes of type-1 and type-2 problems are too broad: many nocomputable functions are type-1 and type-2 learnable. They are also too narrow: trivial functions, such as identity, are neither type-1 nor type-2 learnable. Moreover, the scope of type-1 and type-2 problems appears to be equivalent. Overall, this distinction does not appear useful for machine learning or cognitive science.

Uninformed learning mechanisms will not discover “type- 2” regularities in their inputs, except fortuitously. Clark & Thornton argue that error back-propagation only learns the classical parity problem – which is “always pure type-2” – because of restrictive assumptions implicit in the learning algorithm and network employed. Empirical analysis showing that back-propagation fails to generalise on the parity problem is cited to support their position. The reason for failure, however, is that generalisation is simply not a relevant issue. Nothing can be gleaned about back-propagation in particular, or learning in general, from this failure.

Clark & Thornton's “superficially distinct ploys and mechanisms” are in fact very different: there is a deep difference between (a) filters and feature detectors, which “let the information in,” and (b) contentful representations and theories, which reconfigure it into a computationally tractable form. (a) is bringing abilities to experience whereas (b) is bringing content to experience. Both have well known problems. I outline an evolutionary story that avoids these problems and begins to explain how representations and theories developed out of feature detectors and filters.

Analogical transfer in sequence learning is presented as an example of how the type-2 problem of learning an unbounded number of isomorphic sequences is reduced to the type-1 problem of learning a small finite set of sequences. The commentary illustrates how the difficult problem of appropriate analogical filter creation and selection is addressed while avoiding the trap of strong nativism, and it provides theoretical and experimental evidence for the existence of dissociable mechanisms for type-1 learning and type-2 recoding.

Learning a mapping involves finding regularities in a training set and generalization to novel patterns. Clark & Thornton's type distinction has been discussed in terms of generalization, but has limited value in this respect. However, in terms of detection of regularities in the training set, the distinction is more valid, as it provides a measure of complexity and correlates with the size of search space.

Evolution is not like an exam in which pre-set problems need to be solved. Failing to recognise this point, Clark & Thornton misconstrue the type of explanation called for in species learning although, clearly, species that can trade spaces have more chances to discover novel beneficial behaviours. On the other hand, the trading spaces strategy might help to explain lifetime learning successes.

A fundamental problem with the Clark & Thornton definition of a type-1 problem (requirement 2) is identified. An alternative classical statistical formulation is proposed where a type-1 (learnable) problem corresponds to the case where the learning machine is capable of representing its statistical environment.

Clark & Thornton (C&T) have demonstrated the paradox between the opacity of the transformations that underlie relational mappings and the ease with which people learn such mappings. However, C&T's trading-spaces proposal resolves the paradox only in the broadest outline. The general-purpose algorithm promised by C&T remains to be developed. The strategy of doing so is to analyze and formulate computational mechanisms for known cases of recoding.

The distinction between uninformed learning (type-1) and learning based on recoding using prior information (type-2) helps to clarify some long-standing psychological problems, including misunderstanding of mathematics by children, the need for active construction of concepts in cognitive development, and the difficulty of configural learning tasks. However, an alternative to recoding some type-2 tasks is to represent the input as separate dimensions, which are processed jointly. This preserves the original structure, but is subject to processing capacity limitations.

The computational notion of “trading spaces” is highly relevant to the psychological domain of categorization. The “theory” view of concepts can be interpreted as a recoding view. A design principle for exploiting learned recodings in order to handle the type-2 problem of forming sophisticated concepts is outlined.