Biology 202
1999 final Web Reports
On Serendip

Neurobiological Correlates of Mental Imagery to Sensory Perception

David Mintzer

Understanding the brain's function in the human body involves examining how inputs are processed and outputs are generated. On a reductionist, neuronal level it is often difficult to conceive how such processes lead to our experience of the world. While sensory perception and motor output can be directly traced along neuronal pathways, science is still struggling to understand the roots of such internal, intangible processes as thinking, memory, I function and ultimately consciousness. It seems possible that these phenomena emerge through the complex integration of lower level processes, but our knowledge is far from being able to comprehend how this might occur.

Until recently, these abstract concepts have been the domain of cognitive psychology and philosophy. Relying on introspection to get at the nature of our experience, the early philosopher's excursions into these realms were necessarily highly subjective and were not concerned with biological or anatomical functionality. With the popularity of behaviorism in the early 1900's, mainstream psychologists avoided reference to such issues. The development of cognitive psychology pushed internal processes to the forefront, and examined them by utilizing behavioral indicators to theorize about the underlying concepts of thinking and consciousness (1).

However, only in the last two decades have psychologists been able to take advantage of technology which reveals the activity of the brain during cognitive tasks. This new approach, dubbed cognitive neuroscience, has attempted to corroborate theories on mental processes with empirical evidence of brain activity (2) . These scientists are now beginning to understand how the brain is responsible for such processes.

One of the prime candidates for neurobiological inspection is the phenomenon of mental imagery. Introspectively, this ability seems closely tied to perception, of which we have a firm biological grasp. Furthermore, mental imagery has implications regarding memory, thought, reasoning and emotion, which often seem intrinsically tied to imagery. Mental imagery is an interesting phenomenon because it blurs the line between inputs and outputs. On the one hand, imagery can be considered an input such as vision because we can observe elements of shape, size and color. On the other hand, we can actively manipulate mental imagery, which resembles a behavioral output of sorts. Either way, mental imagery is generated internally without any necessary external prime. Psychologists and philosophers have long pondered our ability to obtain, examine and manipulate a picture in our head without utilizing actual sensory input. The most obvious theory is that imagery utilizes a common mechanism as perception. When one imagines a sound or sight, the same phenomenological attributes are encountered as during actual perception. Hence visual imagery would be processed on a depictive level, with coded information relating to points in space. However, mental imagining is a cognitive process, in-line with thinking and reasoning whereas perception is a sensory process involving specific sensory pathways. It has been argued that imagery is not distinct from any of the other cognitive processes. This idea is known as the propositional theory of mental imagery (3). Here, all cognitive processing utilizes a language-like code by combining strings of concepts and relations. It should be noted that the depictive and propositional theories are metaphors for understanding what is actually occurring in the networks of neurons in the brain. Of course there is no space in the head filled with images or a list of symbols. These ideas show how information coded by neurons may be organized and interpreted as an entire system.

Before the advent of cognitive neuroscience, psychologists tested their hypotheses with behavioral experiments. The depictive theory was supported by the finding that the time it took to scan a mental image corresponded with the time it would take to scan an actual image in space. When subjects were asked to imagine a familiar scene and focus on one area, it took longer for them to describe an area further away than a closer area. This suggested that the scene in their mind was orientated in space and they were behaving as if they had to transfer their line of focus accordingly (4) . If the image were merely a propositional description, all aspects of the scene should have been equally accessible.

If these visual images are analogous to visual perception, it is quite possible that they share common neurobiological mechanisms. A common link was first detected in patients with brain damage resulting in visual impairment. Patients with posterior parietal lesions who were blind in one side of the visual field also were unaware of items on that side when imagining a familiar visual scene. If the patient were asked to turn the mental image around so that they were facing in the opposite direction, they reported items on the other side and ignored items which they had just previously reported "seeing" (3). However, since there have been case studies of brain damaged patients who have lost visual perception but have maintained the ability to create visual images, the two mechanisms may share only a partial link or may be hierarchical in nature so that they can operate independently of each other. For example, if imaging is a top-down procedure relying on previously processed visual inputs, it could still function without the bottom-up pathways of visual input.

With the advent of brain imaging devices it has become possible to actually determine which areas of the brain are most active at different times. The most widely used in studies of visual imagery has been positron emission tomography (PET). In a typical PET experiment, the subject is injected with a small amount of radioactively labeled water. When an area of the brain is working hard and processing information more blood flows through it and higher levels of the radioactive H2O are detected2.

Studies using this technology soon confirmed that areas of the visual cortex are active during mental imagery, suggesting that indeed visual imagery and visual perception may share a common neural basis. Kosslyn asked subjects to close their eyes and visual letters. He took PET measurements as the subjects visualized very small and very large letters. As suspected, imagining small letters increased blood flow to a small area of the visual cortex (corresponding to the fovea of the retina) but with greater activation within that region. Large letters activated neurons in the peripheral area of the visual cortex. This data implied that like visual perception, imagery maps topographically onto the visual cortex. The larger letter activated a larger region while the smaller letter was contained in a smaller region, but with more information concentrated in that area, more activity was observed (2).

A rather ingenious experiment by Kosslyn achieved the best direct comparison between mental imagery and visual perception. He presented a grid to the subject with a small cursive letter beneath it. There was an X in one of the squares of the grid. Kosslyn asked the subject to mentally picture the capital version of the letter, place it on the grid and determine if the letter fell on the marked square (see figure 1) He took PET recordings while the subject performed this task and then compared these measurements to those taken while the subject was presented with a picture of the grid filled in with the capital letter. The difference between the two measurements represented the part of the brain devoted to generating the image of the letter. Results showed that there was a higher level of activation of the primary visual area during mental visualization than during actual visual perception (3).

figure 1. Kosslyn's PET comparison of mental imagery versus perception.

At first it may seem surprising that the PVA shows greater activation when the subjects visualizes an image than when they actually see the image. However an examination of the brain anatomy of the visual system sheds some light on a possible theory explaining this phenomenon. Photoreceptors in the eye's retina send their info to the primary visual area (PVA) where information regarding orientation, texture, retinal disparity and color are processed independently. The information is then sent to the visual association cortex where it is assembled, interpreted and stored in long-term memory (6). According to M.J. Farah "visual processing during perception begins with retinotopic [mapped out topographically according to position of input on retina] representations in the occipital cortex, and progresses to memory representations of objects in the temporal cortex."(1) These pathways connecting the different areas of the visual cortex run in both directions. Afferent nerves carry information from the eyes to the PVA and through the visual association cortex while efferent nerves send signals in the opposite direction. This set-up allows for a top-down theory of visual imagery: stored visual information (memories) in the visual associate cortex travels backwards into the PVA where it evokes a pattern of activity in the topographically mapped area and produces an image (7). Since this image originated from memory, it would be perceived as a mental picture rather than a sensory one.

This theory is furthered by the concept of visual priming. Processing of certain visual inputs occurs faster if a related prime had been displayed prior to the input. This phenomenon is an example of the top-down influence on visual perception. The neural explanation of priming is believed to involve the efferent pathways from the visual association cortex. When the prime is displayed, its representation is activated in the association cortex which sends a corresponding signal down the efferent nerves to the subsystems which receive the input from the retina. When the input is received, it is more easily recognized because of the information supplied by the visual association cortex. Visual imagery is thought to be an extreme version of this process, where the priming is so great that information flowing down the efferent pathways actually triggers the pattern generator associated with the visual input in the PVA. Not surprisingly, it has been found that subjects visualize images quicker when they have been primed with an appropriate visual input7.

The relationship between visual perception and visual imagery became even clearer when researchers examined the specific pathways connecting the PVA to the visual association cortex. Visual sensory information leaving the PVA diverges into two streams in the extrastriate cortex. The ventral stream, which is directed towards the inferior temporal lobe of the visual associative cortex analyzes what the eye is seeing. The dorsal stream aims toward the posterior parietal lobe and analyzes where the input is located in space. Such a dichotomy has also been observed with mental imagery. PET experiments involving mental imagery of spatial tasks showed increased activation in the dorsal route. Imagery tasks involving object and face recognition have activated the ventral pathway (3).

It needs to be noted that several studies have contradicted Kossyln's findings in that they found no indication of PVA activation during mental imagery. The details of these studies can be found elsewhere (3), (8) , but it suffices to say that the discrepancies can not be blamed on methodological error alone. It seems that certain types of mental imagery activate the PVA while others do not. Mental tasks involving figure (shape, size, etc.) seem to activate the PVA while those involving spatial orientation do not (3). Further studies need to be conducted in order to determine the exact nature of this difference.

While the amount of evidence linking mental images with sensory perception and supporting the depictive hypothesis continues to grow, all studies involving PET and other imaging techniques must be weighed with a critical eye. These studies run into the same problem to which the early behaviorists objected. The experimenters do not actually know what the subjects are experiencing. They must trust the subjects' self-report that they are imagining what they are supposed to, that the mental images are consistent across subjects and that no extraneous thoughts enter the mind which could alter the PET measurements. The study of vision has been one of the most successful in the field of neurobiology, specifically because experimenters can control the input and observe and record brain activity. With mental imagery such manipulations are not as reliable; only through clever design can researchers attempt to study imagery and control for differences between subjects.

Cognitive neuroscience has made incredible progress during the last fifteen years in understanding the brain mechanisms of mental imagery. In some cases it has confirmed and enhanced early theories proposed by cognitive psychology and philosophy. Other times it has debunked once popularly held views. However, as is evident in this discussion, the field's understanding is also very limited. Mental imagery is not being studied on the level of individual neurons, rather the science examines groups of neurons and how they interact with other groups. Such experiential phenomenon as mental imagery may never be explained by the workings of single neurons, as it is not of the nature of these cells to "experience". Understanding how they work together to generate our experiences should be the ultimate goal of cognitive neuroscience. If there is no intangible, immaterial element of the brain which produces the intangible experience of our world, we must understand how the brain can be the image and experience the image. Computer programming and artificial intelligence have begun to examine this inherent problem, but as of yet, only the brain is capable of truly experiencing the world.

WWW Sources

1)"The Cognitive Neuroscience of Mental Imagery"Neuropsychologia,(1995) Vol 33, No 11. pp 1335-1582

2) "The Brain's Mind's Eye", Kosslyn, Steven. Harvard Mahoney Neuroscience Institute Letter, On the Brain. Vol4, Number 1 (1995).

3) "Reopening the Mental Imagery Debate: Lessons from Functional Anatomy." E. Mellet, L. Petit, B. Mazoyer, M. Denis, N. Tzourio. NeuroImage, v 8, n 2, August 1998, p129-139, 1998 Academic Press.

4) "Kosslyn: Mental Imagery" Baden, Denise.

5) "The Representing Brain: Neural Correlates of Motor Intention and Imagery" Jeannerod, Marc. Brain and Behavioral Sciences.

6) Physiology of Behavior. Carlson, Neil. Allyn and Bacon, (1998)

7) Image and Brain. Kosslyn, Stephen. MIT Press (1994)

8) "Functional Anatomy of Spatial Mental Imagery Generated from Verbal Instructions." Emmanuel Mellet, et al. "The Journal of Neuroscience". Volume 16, Number 20, October 15, 1996 pp. 6504-6512.

Additional Sources

1) "Primary Motor and Sensory Cortex Activation during Motor Performance and Motor Imagery: A Functional Magnetic Resonance Imaging Study". Porro, Carlo. et al. "Journal of Neuroscience". 1996, vol 16: 7688-7698.

2) "Imagery". MITECTS.

3) "Notes On Kosslyn 'Mental Imagery'". Dawson.

4) "Involvement of Primary Motor Cortex in Motor Imagery: A Neuromagnetic Study". Schnitzler, Alfons, et al, NeuroImage, vol 6, num 3, Oct 1997, p 201-208.

5) "Imagination, Mental Imagery, Consciousness, and Cognition: Scientific, Philosophical and Historical Approaches." Thomas, Nigel.