Engineers use neural networks to control systems too complex for conventional engineering solutions. To examine the behavior of individual hidden units would defeat the purpose of this approach because it would be largely uninterpretable. Yet neurophysiologists spend their careers doing just that! Hidden units contain bits and scraps of signals that yield only arcane hints about network function and no information about how its individual units process signals. Most literature on single-unit recordings attests to this grim fact. On the other hand, knowing a system's function and describing it with elegant mathematics tell one very little about what to expect of interneuronal behavior. Examples of simple networks based on neurophysiology are taken from the oculomotor literature to suggest how single-unit interpretability might decrease with increasing task complexity. It is argued that trying to explain how any real neural network works on a cell-by-cell, reductionist basis is futile and we may have to be content with trying to understand the brain at higher levels of organization.

Prevailing engineering-inspired theories of motor control based on sequential/algorithmic or motor-programming models are difficult to reconcile with what is known about the anatomy and physiology of the motor areas. This is partly because of certain problems with the theories themselves and partly because of features of the cortical and basal ganglionic motor circuits that seem ill-suited for most engineering analyses of motor control. Recent developments in computational neuroscience offer more realistic, that is, connectionist, models of motor processing. The distributed, highly parallel, and nonalgorithmic processes in these models are inherently self-organizing and hence more plausible biologically than their more traditional algorithmic or motor-programming counterparts. The newer models also have the potential to explain some of the unique features of natural, brain-based motor behavior and to avoid some of the computational dilemmas associated with engineering approaches.

The posterior parietal cortex (PPC) is the most likely site where egocentric spatial relationships are represented in the brain. PPC cells receive visual, auditory, somaesthetic, and vestibular sensory inputs; oculomotor, head, limb, and body motor signals; and strong motivational projections from the limbic system. Their discharge increases not only when an animal moves towards a sensory target, but also when it directs its attention to it. PPC lesions have the opposite effect: sensory inattention and neglect. The PPC does not seem to contain a “map” of the location of objects in space but a distributed neural network for transforming one set of sensory vectors into other sensory reference frames or into various motor coordinate systems. Which set of transformation rules is used probably depends on attention, which selectively enhances the synapses needed for making a particular sensory comparison or aiming a particular movement.

We provide a threefold taxonomy of models in neurolinguistics: faculty models, which embrace the work of classical connectionists and holists and of such modern workers as Geschwind; process models, which are exemplified by the work of Luria, and which fractionate psycholinguistic tasks and ascribe the components to particular brain regions; and representational models, which use specific linguistic representations to build a psycholinguistic analysis of aphasic performance. We argue that further progress requires that neurolinguistics become more computational, using techniques from Artificial Intelligence to model the cooperative computation underlying language processing at a level of detail consonant with linguistic representations. Finally, we note that current neurolinguistics makes virtually no contact with the synapse-cell-circuit level of analysis characteristic of twentieth-century neuroscience. We suggest that the cooperative computation models we envisage provide the necessary intermediary between current neurolinguistic analysis and the utilization of the fruits of modern neuroanatomy, neurochemistry, and neurophysiology.

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities (including “distributed representations”) or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks.

This target article presents a new computational theory of explanatory coherence that applies to the acceptance and rejection of scientific hypotheses as well as to reasoning in everyday life. The theory consists of seven principles that establish relations of local coherence between a hypothesis and other propositions. A hypothesis coheres with propositions that it explains, or that explain it, or that participate with it in explaining other propositions, or that offer analogous explanations. Propositions are incoherent with each other if they are contradictory. Propositions that describe the results of observation have a degree of acceptability on their own. An explanatory hypothesis is accepted if it coheres better overall than its competitors. The power of the seven principles is shown by their implementation in a connectionist program called ECHO, which treats hypothesis evaluation as a constraint satisfaction problem. Inputs about the explanatory relations are used to create a network of units representing propositions, while coherence and incoherence relations are encoded by excitatory and inhibitory links. ECHO provides an algorithm for smoothly integrating theory evaluation based on considerations of explanatory breadth, simplicity, and analogy. It has been applied to such important scientific cases as Lavoisier's argument for oxygen against the phlogiston theory and Darwin's argument for evolution against creationism, and also to cases of legal reasoning. The theory of explanatory coherence has implications for artificial intelligence, psychology, and philosophy.

Beyond modularity attempts a synthesis of Fodor's anticonstructivist nativism and Piaget's antinativist constructivism. Contra Fodor, I argue that: (1) the study of cognitive development is essential to cognitive science, (2) the module/central processing dichotomy is too rigid, and (3) the mind does not begin with prespecified modules; rather, development involves a gradual process of “modularization.” Contra Piaget, I argue that: (1) development rarely involves stagelike domain-general change and (2) domainspecific predispositions give development a small but significant kickstart by focusing the infant's attention on proprietary inputs. Development does not stop at efficient learning. A fundamental aspect of human development (“representational redescription”) is the hypothesized process by which information that is in a cognitive system becomes progressively explicit knowledge to that system. Development thus involves two complementary processes of progressive modularization and progressive “explicitation.” Empirical findings on the child as linguist, physicist, mathematician, psychologist, and notator are discussed in support of the theoretical framework. Each chapter concentrates first on the initial state of the infant mind/brain and on subsequent domain-specific learning in infancy and early childhood. It then goes on to explore data on older children's problem solving and theory building, with particular focus on evolving cognitive flexibility. Emphasis is placed throughout on the status of representations underlying different capacities and on the multiple levels at which knowledge is stored and accessible. Finally, consideration is given to the need for more formal developmental models, and a comparison is made between representational redescription and connectionist simulations of development. In conclusion, I consider what is special about human cognition by speculating on the status of representations underlying the structure of behavior in other species.

Human agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency – as though these inferences were a reflexive response of their cognitive apparatus. Furthermore, these inferences are drawn with reference to a large body of background knowledge. This remarkable human ability seems paradoxical given the complexity of reasoning reported by researchers in artificial intelligence. It also poses a challenge for cognitive science and computational neuroscience: How can a system of simple and slow neuronlike elements represent a large body of systemic knowledge and perform a range of inferences with such speed? We describe a computational model that takes a step toward addressing the cognitive science challenge and resolving the artificial intelligence paradox. We show how a connectionist network can encode millions of facts and rules involving n-ary predicates and variables and perform a class of inferences in a few hundred milliseconds. Efficient reasoning requires the rapid representation and propagation of dynamic bindings. Our model (which we refer to as SHRUTI) achieves this by representing (1) dynamic bindings as the synchronous firing of appropriate nodes, (2) rules as interconnection patterns that direct the propagation of rhythmic activity, and (3) long-term facts as temporal pattern-matching subnetworks. The model is consistent with recent neurophysiological evidence that synchronous activity occurs in the brain and may play a representational role in neural information processing. The model also makes specific psychologically significant predictions about the nature of reflexive reasoning. It identifies constraints on the form of rules that may participate in such reasoning and relates the capacity of the working memory underlying reflexive reasoning to biological parameters such as the lowest frequency at which nodes can sustain synchronous oscillations and the coarseness of synchronization.

The appropriate methodology for psychological research depends on whether one is studying mental algorithms or their implementation. Mental algorithms are abstract specifications of the steps taken by procedures that run in the mind. Implementational issues concern the speed and reliability of these procedures. The algorithmic level can be explored only by studying across-task variation. This contrasts with psychology's dominant methodology of looking for within-task generalities, which is appropriate only for studying implementational issues.

The implementation-algorithm distinction is related to a number of other “levels” considered in cognitive science. Its realization in Anderson's ACT theory of cognition is discussed. Research at the algorithmic level is more promising because it is hard to make further fundamental scientific progress at the implementational level with the methodologies available. Protocol data, which are appropriate only for algorithm-level theories, provide a richer source than data at the implementational level. Research at the algorithmic level will also yield more insight into fundamental properties of human knowledge because it is the level at which significant learning transitions are defined.

The best way to study the algorithmic level is to look for differential learning outcomes in pedagogical experiments that manipulate instructional experience. This provides control and prediction in realistically complex learning situations. The intelligent tutoring paradigm provides a particularly fruitful way to implement such experiments.

The implications of this analysis for the issue of modularity of mind, the status of language, research on human/computer interaction, and connectionist models are also examined.

A number of ways of taxonomizing human learning have been proposed. We examine the evidence for one such proposal, namely, that there exist independent explicit and implicit learning systems. This combines two further distinctions, (1) between learning that takes place with versus without concurrent awareness, and (2) between learning that involves the encoding of instances (or fragments) versus the induction of abstract rules or hypotheses. Implicit learning is assumed to involve unconscious rule learning. We examine the evidence for implicit learning derived from subliminal learning, conditioning, artificial grammar learning, instrumental learning, and reaction times in sequence learning. We conclude that unconscious learning has not been satisfactorily established in any of these areas. The assumption that learning in some of these tasks (e.g., artificial grammar learning) is predominantly based on rule abstraction is questionable. When subjects cannot report the “implicitly learned” rules that govern stimulus selection, this is often because their knowledge consists of instances or fragments of the training stimuli rather than rules. In contrast to the distinction between conscious and unconscious learning, the distinction between instance and rule learning is a sound and meaningful way of taxonomizing human learning. We discuss various computational models of these two forms of learning.

A set of hypotheses is formulated for a connectionist approach to cognitive modeling. These hypotheses are shown to be incompatible with the hypotheses underlying traditional cognitive models. The connectionist models considered are massively parallel numerical computational systems that are a kind of continuous dynamical system. The numerical variables in the system correspond semantically to fine-grained features below the level of the concepts consciously used to describe the task domain. The level of analysis is intermediate between those of symbolic cognitive models and neural models. The explanations of behavior provided are like those traditional in the physical sciences, unlike the explanations provided by symbolic models.

Higher-level analyses of these connectionist models reveal subtle relations to symbolic models. Parallel connectionist memory and linguistic processes are hypothesized to give rise to processes that are describable at a higher level as sequential rule application. At the lower level, computation has the character of massively parallel satisfaction of soft numerical constraints; at the higher level, this can lead to competence characterizable by hard rules. Performance will typically deviate from this competence since behavior is achieved not by interpreting hard rules but by satisfying soft constraints. The result is a picture in which traditional and connectionist theoretical constructs collaborate intimately to provide an understanding of cognition.

Recent “connectionist” models provide a new explanatory alternative to the digital computer as a model for brain function. Evidence from our EEG research on the olfactory bulb suggests that the brain may indeed use computational mechanisms like those found in connectionist models. In the present paper we discuss our data and develop a model to describe the neural dynamics responsible for odor recognition and discrimination. The results indicate the existence of sensory- and motor-specific information in the spatial dimension of EEG activity and call for new physiological metaphors and techniques of analysis. Special emphasis is placed in our model on chaotic neural activity. We hypothesize that chaotic behavior serves as the essential ground state for the neural perceptual apparatus, and we propose a mechanism for acquiring new forms of patterned activity corresponding to new learned odors. Finally, some of the implications of our neural model for behavioral theories are briefly discussed. Our research, in concert with the connectionist work, encourages a reevaluation of explanatory models that are based only on the digital computer metaphor.