Suppose we wanted to re-engineer the human brain in a way that would allow us to perceive entirely new kinds of qualities. How could that be done? Suppose for example that we have developed a new kind of neuron that is sensitive to magnetic fields, and we want to connect them to our brains in a way that would give us conscious awareness of magnetism, using qualities that differ from those generated by the other senses. What sort of structural alterations would be needed? The answer is unclear; and the fact that it is unclear is important, because it points to a serious gap in our understanding of the neurobiology of consciousness.
The purpose of this post is to justify those statements. Let me start with a bit of background.
We have, as everybody knows, several distinct sensory systems: vision, hearing, touch, olfaction, etc. Each of those systems is implemented differently in the brain, but at the level of conscious awareness, they have certain things in common. Most importantly, input from each sense is perceived in terms of qualities. For vision the qualities include color and brightness; for hearing they include pitch and loudness; etc.
Our full perceptual world can be decomposed into several orthogonal quality domains, one for each sensory system. We can compare qualities within a domain, but not across domains. Thus we can compare the color or brightness of two points in visual space, and we can compare the pitch or loudness of two tones, but we have no way of comparing a color with a pitch, except at a metaphorical level. (There is an interesting caveat: in people with synesthesia, the separation between sensory domains breaks down. That’s an important fact, but a discussion of it would lead me astray.)
The perceptual separation between sensory systems is mirrored at the brain level. Each sensory system is implemented by a distinct set of brain components. There are subcortical areas such as the cochlear nuclei; there are dedicated areas of the thalamus (for every sense except olfaction), and most importantly, there are dedicated primary and secondary areas of the cerebral cortex. Thus for vision we have the primary visual area V1 in the occipital lobe, and secondary areas V2, V3, etc. adjoining it. For hearing we have the primary auditory area A1 in the parietal lobe, and secondary areas A2 etc. adjoining it. And so on. The secondary areas project to higher-level parts of the cerebral cortex, which integrate information from multiple sensory systems.
Thus if we want to add a new sensory system to the brain, up to a certain point it is reasonably clear what we should do. We should allocate a portion of the cerebral cortex as a dedicated primary area. Our magnetic-sensing neurons can either project there directly, or, if they need to be reformatted, they can be routed through a newly allocated part of the thalamus. Our primary magnetic area can send its output to adjoining secondary magnetic areas which extract useful features, analogous to the way secondary visual areas extract features such as shape and color from raw visual signals.
But what then? What can we do with the resulting secondary signals so that the person whose brain this is will gain a conscious awareness of magnetism, perceived in terms of new qualities? Clearly we need to send the signals to other parts of the cerebral cortex, but which parts, and how should the projections be structured?
The answers are not at all clear. We can try “broadcasting” the output to every other part of the cortex, in an essentially unstructured way, but it seems very doubtful that that will work. But if not that, then what?
The fact that we don’t know how to do this points to an important gap in our understanding of sensory processing. A student of neuroscience, after learning about the structure of the visual system—how it extracts features such as color, motion, and binocular disparity—might get the impression that there are no longer any deep mysteries about perception: that it’s just a matter of working out all the niggly details. Nothing could be farther from the truth. We have substantial understanding of the low-level computational processing of sensory signals, but we don’t understand how they give rise to distinct perceptual qualities. In fact we don’t even understand, at the level of brain function, what a perceptual quality is.
That’s an important thing for philosophers to know. The “hard problem of consciousness”—as David Chalmers likes to put it—is not just hard at a philosophical level. It is also unsolved at an empirical level.