Perceptual Experience and Sensorimotor Dependencies

A summary of section 1.7 of Supersizing the mind by Andy Clark.

1.7. Perceptual experience and sensorimotor dependencies

This section mainly introduces Alva Noë’s account of perceptual experience. Having encountered it in his book Out of our heads, I found it fairly easy to follow.

The central claim of A. Noë, as presented by the author, is:

The central claim is thus that differences in what we perceptually experience correspond to differences in sensorimotor signatures (patterns of association between movements and the sensory effects of movement).

In other words, differences between perceptual experiences correspond to differences in sensorimotor signatures: the patterns connecting movement with its sensory consequences.

Let me unpack that. In the case of vision, common sense suggests that visual experience must arise as light information entering through the eye’s photoreceptors travels along the optic nerve to the visual cortex in the occipital lobe and is processed through various visual pathways.

A. Noë thinks differently. In one animal experiment, the optic nerve was surgically redirected to the auditory cortex rather than the visual cortex, yet visual perception reportedly worked well. In another experiment, blind participants were fitted with a small camera that converted visual information into two-dimensional tactile signals delivered to their backs or tongues through small patches. They reportedly felt that they were having visual experiences.

Combining these two experiments gives us the following:

Information need not be processed in the visual cortex for an experience to count as visual.
Information need not enter through the eyes for an experience to count as visual.

(This too seems, at first glance, not to fit evolutionary psychology very well. I intend to discuss that separately later.)

If neither the eyes nor the visual cortex are necessary, what makes an experience visual? A. Noë’s answer is that distinctive perceptual experiences arise from the characteristic ways sensory signals change with movement. For example, visual experience has patterns such as these:

Objects grow larger as we approach them and smaller as we move away.
We can obtain information only about an object’s surface.
We can obtain information about only one side of an object at a time. To learn about another side, we must turn the object or move our bodies.
…

Whatever process produces them, if these movement–sensation patterns are reproduced, the experience should count as visual.

I hope the following passage from A. Noë is now easier to understand:

what determines phenomenology is not neural activity set up by stimulation as such, but the way the neural activity is embedded in a sensorimotor dynamic.

What determines the phenomenon—the perceptual experience—is not the neural activity caused by stimulation in itself, but how that activity is embedded in sensorimotor dynamics.