A neural network model is developed to explain how visual thalamocortical interactions give rise to boundary percepts such as illusory contours and surface percepts such as filled-in brightnesses. Top-down feedback interactions are needed in addition to bottom-up feed-forward interactions to simulate these data. One feedback loop is modeled between lateral geniculate nucleus (LGN) and cortical area V1, and another within cortical areas V1 and V2. The first feedback loop realizes a matching process which enhances LGN cell activities that are consistent with those of active cortical cells, and suppresses LGN activities that are not. This corticogeniculate feedback, being endstopped and oriented, also enhances LGN ON cell activations at the ends of thin dark lines, thereby leading to enhanced cortical brightness percepts when the lines group into closed illusory contours. The second feedback loop generates boundary representations, including illusory contours, that coherently bind distributed cortical features together. Brightness percepts form within the surface representations through a diffusive filling-in process that is contained by resistive gating signals from the boundary representations. The model is used to simulate illusory contours and surface brightnesses induced by Ehrenstein disks, Kanizsa squares, Glass patterns, and cafe wall patterns in single contrast, reverse contrast, and mixed contrast configurations. These examples illustrate how boundary and surface mechanisms can generate percepts that are highly context-sensitive, including how illusory contours can be amodally recognized without being seen, how model simple cells in V1 respond preferentially to luminance discontinuities using inputs from both LGN ON and OFF cells, how model bipole cells in V2 with two colinear receptive fields can help to complete curved illusory contours, how short-range simple cell groupings and long-range bipole cell groupings can sometimes generate different outcomes, and how model double-opponent, filling-in and boundary segmentation mechanisms in V4 interact to generate surface brightness percepts in which filling-in of enhanced brightness and darkness can occur before the net brightness distribution is computed by double-opponent interactions.

Goldfish (Carassius auratus) were trained to discriminate triangles and squares using a two choice procedure. In the first experiment, three goldfish were trained with food reward on a black outline triangle on a white background, while a black outline square was shown for comparison. In transfer tests, a Kanizsa triangle and a Kanizsa square were presented, perceived by humans as an illusory triangle- or square-shaped surface of slightly higher brightness than the background. The choice behavior in this situation indicates that goldfish are able to discriminate between both figures in almost the same way as in the training situation. In control experiments goldfish did not discriminate between shapes in which humans do not perceive the illusion. A series of generalization experiments was performed indicating the similarity between the tested shapes and the training triangle. From all these findings we conclude that goldfish are able to perceive an illusory triangle or square within the Kanizsa figures. In a second experiment, four goldfish were trained on a white outline triangle versus a white outline square, both on black background with white diagonal lines. In transfer tests in which the shapes were replaced by gaps within the white diagonal lines, goldfish were clearly able to discriminate between the two patterns based on the illusory contours. This was not the case in tranfer tests with phase shifted abutting lines.

Natural scenes contain a variety of visual cues that facilitate boundary perception (e.g., luminance, contrast, and texture). Here we explore whether single neurons in early visual cortex can process both contrast and texture cues. We recorded neural responses in cat A18 to both illusory contours formed by abutting gratings (ICs, texture-defined) and contrast-modulated gratings (CMs, contrast-defined). We found that if a neuron responded to one of the two stimuli, it also responded to the other. These neurons signaled similar contour orientation, spatial frequency, and movement direction of the two stimuli. A given neuron also exhibited similar selectivity for spatial frequency of the fine, stationary grating components (carriers) of the stimuli. These results suggest that the cue-invariance of early cortical neurons extends to different kinds of texture or contrast cues, and might arise from a common nonlinear mechanism.

The discontinuities of phase-shifted abutting line gratings give rise to perception of an “illusory contour” (IC) along the line terminations. Neuronal responses to such ICs have been interpreted as evidence for a specialized visual mechanism, since such responses cannot be predicted from conventional linear receptive fields. However, when the spatial scale of the component gratings (carriers) is large compared to the neuron's luminance passband, these IC responses might be evoked simply by the luminance edges at the line terminations. Thus by presenting abutting gratings at a series of carrier spatial scales to cat A18 neurons, we were able to distinguish genuine nonlinear responses from those due to luminance edges. Around half of the neurons (both simple and complex types) showed a bimodal response pattern to abutting gratings: one peak at a low carrier spatial frequency range that overlapped with the luminance passband, and a second distinct peak at much higher frequencies beyond the neuron's grating resolution. For those bimodally responding neurons, the low-frequency responses were sensitive to carrier phase, but the high-frequency responses were phase-invariant. Thus the responses at low carrier spatial frequencies could be understood via a linear model, while the higher frequency responses represented genuine nonlinear IC processing. IC responsive neurons also demonstrated somewhat lower spatial preference to the periodic contours (envelopes) compared to gratings, but the optimal orientation and motion direction for both were quite similar. The nonlinear responses to ICs could be explained by the same energy mechanism underlying responses to second-order stimuli such as contrast-modulated gratings. Similar neuronal preferences for ICs and for gratings may contribute to the form-cue invariant perception of moving contours.

In visual science the term filling-in is used in different ways, which often leads to confusion. This target article presents a taxonomy of perceptual completion phenomena to organize and clarify theoretical and empirical discussion. Examples of boundary completion (illusory contours) and featural completion (color, brightness, motion, texture, and depth) are examined, and single-cell studies relevant to filling-in are reviewed and assessed. Filling-in issues must be understood in relation to theoretical issues about neural–perceptual isomorphism and linking propositions. Six main conclusions are drawn: (1) visual filling-in comprises a multitude of different perceptual completion phenomena; (2) certain forms of visual completion seem to involve spatially propagating neural activity (neural filling-in) and so, contrary to Dennett's (1991; 1992) recent discussion of filling-in, cannot be described as results of the brain's “ignoring an absence” or “jumping to a conclusion”; (3) in certain cases perceptual completion seems to have measurable effects that depend on neural signals representing a presence rather than ignoring an absence; (4) neural filling-in does not imply either “analytic isomorphism” or “Cartesian materialism,” and thus the notion of the bridge locus – a particular neural stage that forms the immediate substrate of perceptual experience – is problematic and should be abandoned; (5) to reject the representational conception of vision in favor of an “enactive” or “animate” conception reduces the importance of filling-in as a theoretical category in the explanation of vision; and (6) the evaluation of perceptual content should not be determined by “subpersonal” considerations about internal processing, but rather by considerations about the task of vision at the level of the animal or person interacting with the world.