The Brain's Constructions and Deconstructions of "Reality"

Recent General Readership Articles Anticipating the Future to See the Present, NYTimes, 10 June, 2008 Essence of Thought, New Scientist, May 31-June 6, 2008 Relevant Professional Articles Perceiving the Present and a Systematization of Illusions Learning and Inference in the Brain The Bayesian Brain: The Role of Uncertainty in Neural Coding and Computation Free Energy and the Brain Relevant Serendip Materials Tricks of the Eye, Wisdom of the Brain Seeing More Than Your Eye Does Making Sense of the "Picture in the Head Ambiguous Figures Hofstadter's Road Sign Chance in Life and the World Variability in Brain Function and Behavior Getting It Less Wrong: the Brain's Way The Brain as a Learner/Inquirer/Creator: Some Implications of its Organization for Individual and Social Well Being

What one sees (or hears or feels or tastes of smells) is ... reality? Or a construction of the brain? A construction that might be different if the brain were different? And so could in fact be different in different individuals with different brains? And different for the same individual at different times? The more we learn about the brain, the more it appears that "reality" is indeed a construction of the brain. A disconcerting idea perhaps, one that challenges a whole host of age old assumptions and practices for many people. But the idea is also one that can help us to better understand ourselves and our relations to each other and to the world around us. And one that can give us an enhanced ability to play a significant role in our own lives and in the universe of which we are a part.

Perceptual "Illusions"

"Staring at a pattern meant to evoke an optical illusion is usually an act of idle curiosity, akin to palm reading or astrology" ... but its also entrancing to most people. And with good reason. "Illusions" provide us with a window into how our brains work. And the more they are studied, the clearer it becomes that brains have been designed (by evolution) not to discover "reality" but rather to make the best sense they can at any given time of inputs that are ambiguous, that might in principle be interpreted in any of an infinite number of different ways.

"Ambiguity and uncertainty are not ... the ripples of the imperfect glass through which the brain tries to perceive reality. They are instead the fundamental "reality", both the grist and the tool by which the brain (and, hence, all humans, you and I among them) creates all of its [perceptions]."

Given this ambiguity, what we perceive is not "reality", but rather the brain's best guess, an informed best guess, of what is out there that is meaningful to itself (and us). Everything we perceive is such a guess. "Illusions" are just special cases that help us to understand what the brain is using to make that guess, what makes it an "informed" guess. And that understanding, in turn, points us to ways we can move beyond our existing guesses to "less wrong" ones.

Informed Guessing: An Old Example

A large class of visual illusions reflects a tendency of the brain (in all senses, not only visual) to pay more attention to changes in input signals occurring over space and/or time than it does to the absolute values of those signals. As a consequence, one's perception of the brightness (as well as the color) of an object is heavily influenced by other objects in the scene one is looking at (as well changes from what one was previously looking at). This can yield quite dramatic changes in both the color and the brightness of particular objects depending on their spatial and temporal context (cf. Making Sense of the "Picture in the Head").

Why isn't the brain organized to show us things "as they are", instead of making their appearance dependent on other things? In what sense is paying more attention to changes than to absolute values a "best guess"? At least a partial answer to that question involves recognizing that the light reaching our eye from any given location depends not only on the properties of the object at that location but also on the properties and location of whatever is illuminating it (see Tricks of the Eye, Wisdom of the Brain). If our perception of objects depended only on the light we get from them, their appearance would change with every change in the illuminating source. A rose would look quite different in the morning than it does at noon or in the evening. Comparisons in the amount and character of light coming from different objects are much less sensitive to variations in the source of illumination ("a rose is a rose is a rose"). By paying more attention to comparisons than to absolute values, the brain is guessing that what is out there are "objects", things that have borders and stay pretty much the same from moment to moment. For most purposes, its a pretty good guess. For others (like if you want to know what fabric will look good on the sofa in your living room, or how a website will look to someone who is "color blind"), it pays to keep in mind that the brightness or color of something, like other aspects of perception, is always a guess. To put it differently, looking at differences is "useful" in some situations and less so in order. It creates stable order in what might otherwise be a less comprehensible fluidity, but in doing so it presumes the usefulness of certain features that may not be useful in other situations.

Informed Guessing: A New Example

"researchers ... argue that the brain's ability to see into the future creates many common illusions"

One might perhaps better say that "the brain's inclination to make guesses about the future creates many common illusions". Some people might be disappointed that there isn't evidence here that the brain can actually "see into the future" but its more than interesting enough to discover its apparent efforts to do so and how those seem to impact on what we perceive.

Changizi et al base their analysis on illusions like that shown to the left. In the upper image, the actually parallel thicker grid lines appear to curve in the middle, so the grid itself seems to bow out toward the viewer. The curving and bowing clearly depend on the starburst pattern of thinner lines, disappearing when it is absent (lower figure). Following earlier work, Changizi et al suggest that the starburst pattern causes the brain to presume motion of the observer toward the origin of the starburst pattern. The center square of the grid would be expected to become larger than surrounding squares as one gets closer to it. The curving and bowing can then be understood as reflecting the brain's prediction of the expected appearance of the grid at some (relatively soon) future time when the observer is closer to it. What one sees then is not the current appearance of the grid but the brain's prediction of its future appearance. Changizi et al provide significant support for this interpretation by showing that it can be applied to a large set of diverse illusions in an organized and systematic way.

Its an intriguing and not at all farfetched idea that in trying to make sense of ambiguous input the brain would tend to try and anticipate future appearances rather than generating perceptions of "things as they are". We do not normally sit stationary in a stationary world but instead typically are in motion in a world that itself may be continually changing. Interpreting signals in this context gives us a set of ways to make sense of the world that we wouldn't otherwise have, as first compellingly pointed out by J.J. Gibson. Moreover, neural processing takes time (see Time to Think), enough so that by the time a perception is created it may well be out of date.

Perhaps then the message is that the "informed guesses" that our perceptions are based on include not only some presumption that there are reasonably stable "objects" in the world but also on a presumption that one's relation to them is likely to be changing over time in somewhat predictable ways. Here too, its useful to know that what we perceive is an informed guess, but always a guess. Often it may be advantageous to anticipate what will be seen, but some times one might want to know what is now. And for that one needs to know that what one sees might be different if evaluated in differently informed ways (use a ruler to verify the squares are in fact flat squares).

Could We Do Without Informed Guessing?

But what's "real"? What would the world look like if one didn't make informed guesses? Supposing one could itemize all the ways that what we perceive is constructed, all the informed guesses that the brain makes, and then remove them. What would things look like? That question was posed by a student in a class on neurobiology and behavior I taught this year, and another student suggested the answer: "I think reality would be a very noisy place". And that may indeed be the best answer one can provide. "Eliminative reality" may be mostly randomness, waiting to have small statistical regularities noticed by a brain capable of best guesses (see Eliminative Reality and following comments/links, and Chance in Life and the World). Maybe "meaning" is what we make of small regularities in noise?

Who's Doing the Informed Guessing? and Why Should One Care?

Clearly there are things being revealed by illusions that, as suggested by how entrancing they are, teach us things about ourselves and our relation to the world around us. That we don't know about these things until we discover them through illusions also illustrates a more general important lesson about ourselves. The "informed guessing" is being done by processes in our nervous system of which we are largely unaware. Not only do we not know the basis of the informed guessing, until we notice and study it, but we aren't in fact usually aware that what we see involves guessing at all.

This is because our brains consist of two relatively distinct regions. One, the cognitive unconscious, makes informed guesses and delivers them to the second, conscious part, which supports our awareness of what we are seeing while knowing little or nothing of how what we see has been constructed. By trying to explain illusions, wondering how we come to see what we see, we can, using consciousness, deconstruct the process of perception, gaining a useful ability to appreciate things in ways other than those made immediately available by the informed guessing of the unconscious.

The "Bayesian Brain" and "Free-energy"

Just as we tend to see the world as consisting of clear and certain objects rather than a sea of ambiguity, brain researchers have tended to think of the brain itself as functioning in terms of a large and complex but ultimately machine like set of clear and certain processes. More recently, however, it has begun to be suggested that there is significant indeterminacy, and hence ambiguity, within the brain itself, and that brain function might be better understood not as a set of cogs and wheels but rather as an information processing system that continually and somewhat unpreditably generates and updates probabilistic descriptions not only of the world but of its own function as well.

"Making predictions and evaluating them seems to be a universal feature of the brain."

Thomas Bayes was an eighteenth century mathematician and minister who formalized a method of continuallly updating guesses based on new observations, without the assumption normally made by statisticians (and the rest of us) that uncertainty reflects imperfect information about an actual state. Bayes instead assumed only that all we ever know is the likelihood of something, and all we can do is to update that likelihood by new observations. Hence, the thought that it might be productive to look into the extent to which our brains are actually "Bayesian" inference machines.

Still more recently, it has been suggested that both interpreting sensory input and learning about the environment can be thought of similarly as processes of reducing the likelihood of substantial violations of predictions made within the nervous system or, roughly, as assuring that the nervous system has at all times reasonably good models of the world. In terms of trying to characterize brain function in the broadest possible terms, there are intriguing parallels between this and the minimization of free energy in chemical and physical systems.

Whether either the Bayesian or free energy formalisms prove in the long run to be productive routes to better understanding the brain, it seems clear that our brains are doing a lot more guessing and updating than most of us (neuroscientists included) have imagined in the past. Perhaps, as suggested, there is in fact nothing more than informed guessing, no describable "reality", no actual there there (at least nothing more than noise with some statistical regularities) without at least some informed guessing. Let's see whether taking that notion seriously takes us anywhere.

Ambiguous Figures: Beyond the Limits of Informed Guessing

Illusory figures provide a window into the informed guessing that goes on in the brain, and suggest that there is an underlying ambiguity in making sense of the world that makes informed guessing a useful feature of perception. Ambiguous figures give us a more direct look at the underlying ambiguity, and at the limitations of informed guessing. They also serve to further strengthen the argument that consciousness gives us the ability to move beyond the informed guessing that our cognitive unconscious provides us.

What's particularly intriguing about ambiguous figures is not only that one can see them in several different ways but that the several different ways are in some sense "equally good." One sometimes sees them one way and other times another way and there is no good reason for choosing one over another. Like with illusory figures, the informed guessing that gives a particular perception is being done by the cognitive unconscious without our awareness that it is happening. With ambiguous figures, however, the cognitive unconscious is capable of not only one informed guess but of several different ones. In this, ambiguous figures not only call our attention to ambiguity itself but also to an important feature of informed guessing as a way of making sense of ambiguity: sometimes there is more than one way to do it.

Like illusory figures, ambiguous figures can be enjoyed and then disregarded as "special cases" of perception, as, perhaps, amusements created by artists lacking any more general significance. Once one gets interested in them, however, they seem to turn up again and again, and in lots of places where there is no artist involved. In fact, one can find multiple ways to see almost anything (everything?), if one puts one's mind to it.

If one's in a hurry, and needs to act now, its nice to have the cognitive unconscious provide one an informed guess. That's a snake, or a car bearing down on me, so move! But we don't always need to act immediately. Sometimes it can be useful instead to reflect on what we're seeing, and to see what other kinds of informed guesses our cognitive unconscious can come up with. One might even suggest that that is a primary function of consciousness, not to decide what is actually there and act on it but instead to withhold judgement long enough to give the cognitive unconscious a chance to come up with some alternative informed guesses. Maybe, in fact, knowing that what one sees is no more, and no less, than an informed guess and making use of that in one's behavior is an important part of what we mean by personal responsibility or free will? One can use informed guesses but not be a slave to them?

 

 

Impossible Figures: Beyond the Limits of What Has Been

Deconstructing the process of perception using consciousness makes it possible not only to go beyond immediate informed guesses about the nature of things we say but to create things we would not otherwise perceive, to be creative. Visual artists have over centuries acquired great skill at using the brain's propensity to make informed guesses to cause viewers to see things other than what is actually there (a sense of three dimensions in two dimensional paintings for example). But one can, in addition, create representations of things that not only are not actually there but could not in fact be there. Escher is the best known of a large number of artists who create such impossible figures. To do so, they create images so as to cause the brain to make a best guess at one location in the image and a different and incompatible best guess at a different location. The upshot is that instead of their being multiple best guesses for the image as a whole, there is no satisfactory best guess at all. No single informed guess suffices for the entire picture. To put it differently, perceptions as informed guessing may be the foundations of our understanding but we are not limited by them. We can create things that go beyond what we have perceived and, in so doing, bring into existence in our world things that have never previously existed and would not but for our construction of them.

The Brain and "Reality"

Drawing by Roger N.Shepherd, Mindsights, W.H. Freeman and Co., 1990

"It is sometimes said that advances in empirical science not only necessarily alter our sense of humanness but invariably demean it ... I ... see the current and projected impact of empirical research on the brain in a quite different light."

Illusions, ambiguous figures, and impossible figures are fun, but they are more than that as well. They provide windows into how our brains work. And, in so doing, they help us see not only where our perceptions of "reality" come from but, even more importantly, the role we play in our constructions of reality and the capability we have to move beyond any particular construction. Yes, we may have to give up the idea of a fixed "reality" that we aspire to discover. But we are offered, to replace it, a sense of ourselves as active participants not only in making sense of our world but of shaping it as well. That is, it seems to me, not a bad tradeoff. To understand the brain is not to demean humanness, but rather to recognize that we have a continuing capacity to create in new ways, that any particular understanding is not an end but rather a foundation and resource for new understanding. No matter how much we learn about the brain (and the world) there will always be more to learn. The answer to the question "Is it possible to capture all the workings of your brain in one simple equation?" is a loud "no", because in learning we necessarily create new things to understand.