Emergeneers,

An interesting article on the statistics of "blogging" (web logs):

"Power Laws, Weblogs, and Inequality"

http://www.shirky.com/writings/powerlaw_weblog.html

-Doug
******************

Interesting indeed. Anne and I had quick go at this in conversation this morning, would be worth more talking through collectively ...

Question, in essence, is what "power law" behavior of complex systems might mean in social/cultural realm. Parenthetically, its noteworthy (for language group?) that technical term for nature of distribution of perturbation sizes in complex systems (most/all?) resonates for some people with social issue of "power' and its distribution, in sense of inequity.

General argument of paper, if I'm understanding it correctly, is that thinking of social organization as complex system implies that almost any characteristic of individuals (wealth, popularity, "power") will be distributed as a power law, implying the inevitability of substantial "inequity". Furthermore, the range of distribution ("amount of inequity") will necessarily increase as the population size increases.

On the face of it, this has a "fatalistic" character to it, and it is, at a minimum, an argument for keeping social systems small (an inference that one might well reach for other reasons, cf. Small is Beautiful). But ...

1- The analysis presumes that one WANTS to be wealthiest, or most popular, or most powerful. Many people (myself included) don't, for quite coherent reasons (the costs of being any of the above, among other things what my son calls the "budweiser effect").

2- The analysis presumes that a social "objective" is to eliminate "inequity". For me at least the objective isn't necessarily to eliminate inequity but rather to assure that distributions remain fluid so that individuals are not locked into particular places in them.

3- The analysis strongly depends on the presumption that interactions among individuals are such that particular choices made by some individuals bias other individuals to make the same choice (the "PC effect"). This is akin to ant stigmergy, noticing high concentrations of pheromones inclines ants to move along the same path. In fact, some people noticing a prevalence of individual choices to go in one direction use that as a cue to move in some other direction (yours truly?).

Bottom line:

I don't think fixed inequities are an inevitable outcome of complex systems in the social realm. On the other hand, the analysis does suggest the value of encouraging people to recognize in themselves inherent (?) biases toward following the stories of other people and do more striking out on their own.

The interactivity of the Web is perhaps its most important characteristic. For the first time in human history, it is becoming possible for all humans to play an active role in world-wide cultural and intellectual interchange.

See also The Place of the US in the World Community, a Serendip forum area:

... to tell and listen to each others' stories, to commit ourselves anew to finding new ways to tell our collective human story in a way from which no one feels estranged

"Social theorists have a name for these types of decentralized networks: heterarchies. In contrast to hierarchies, with top-down structures, heterarchies are made up of previously isolated groups that can connect to one another and coordinate. "

Yours in the (emerging) struggle,
Anne

Hi folks,

I mentioned at the end of my talk that Matthew Cook recently proved that RULE 110 is computationally universal. I just obtained a copy of his proof. Now that Wolfram's book is published, he's allowing Cook (who formerly worked for Wolfram and is currently a grad student at UCLA) to distribute his proof. Cook's theorem is ground breaking, and is the core fact supporting the philosophy that Wolfram develops in his text.

Would anyone like a copy? It's 40 pages, and seems to be very carefully written with lots of background and motivational explanations and pictures. Would anyone like to work though the proof together? I think that, if we're willing to skip some of the messier details, it would be a reasonable project. Perhaps it would be a good project to involve students in, with the goal of hiring a group of students to do a summer project deciding whether rule 30 is universal. Maybe this question is too large to solve in a single summer, but we could ask the experts their opinion on this.

Please let me know your level of interest,

Kris

An urge to organize
As individuals, army ants have almost no brain to speak of, just a clump of neurons inside their tiny heads. Working as a group, however, they rule the Amazon jungles, marching in formation over acres of land and flushing out thousands of insects, even scorpions, that are their prey. The ants move out and then file back in orderly lines, with the returning parties efficiently forming lanes inside the outgoing ants.
The full article will be available on the Web for a limited time:
http://www.philly.com/mld/inquirer/living/health/5409380.htm

(c) 2003 Philadelphia Inquirer and wire service sources. All Rights Reserved.

If real plants don't compute, then I don't know what computing is. I'd say the opposite: computing is the one thing that is being selected for, and adapted. I think we need to discuss what is "computing". Computers are the ones that don't do it very well, and don't do it very naturally.

Paul has agreed to lead a discussion on "information" in the Emergent Phenomena group. Maybe we can discuss this issue here and there.

-Doug

• Is VERY interesting idea that one can trace threads of history of emergent thinking to Adam Smith, a century before Darwin. Possible still earlier thread coming out of post discussion conversation with Anne: Ovid's Metamorphosis?
  
• There is wonderful resonance to a talk by Ted in the brown bag series last semester on modelling, what it can/cannot do. Bottom line, from today, one can ALWAYS create a model that will emulate a given phenomenon of interest. In fact one can always create multiple such models. Conclusion: models are not useful simply because they emulate a given phenomenon, nor challengeable on the grounds that they don't reflect "reality" nor adequately account for all the variables in reality. What they are useful for is to show that a given phenomenon COULD be accounted for is some surprisingly "simple" way.
  
• The micro/macro distinction in economics is a mirror of reductionist/wholist arguments in a whole variety of fields.
  

http://emergent.brynmawr.edu/wiki/index.cgi/EmergenceReadingList

Please feel free to help create/edit/organize that list. Panama has added many nice annotations to the list, so it is already a nice resource.

Early writers on the ideas of emergence include: Hume, Smith, and of course Darwin.

As a cognitive scientist, I always think of intelligence when I think of modeling. Whether or not we can even model that, though, is an open question. Maybe one would want to claim that predicate logic with a database of facts is a model, although it doesn't capture too much of the essence of intelligence, to me. But if we do model intelligence, I'd want to say that is more than a model: the system would have intelligence.

It seems that in many fields when they come to make a micro/macro distinction that it spawns a new field. It is interesting that economics decided to keep both levels in one. My advice to the economists: spawn, baby, spawn!

-Doug

Okay, I'll bite.
WAY before Adam Smith, WAY before Darwin, there was
Ovid. Metamorphosis (1-8 A.C.E.):

...In all creation
Nothing endures, all is in endless flux....
Nothing retains its form; new shapes from old
Nature, the great inventor, ceaselessly
Contrives. In all creation, be assured,
There is no death--no death, but only change
And innovation....
...the earth and all therein, the sky
And all thereunder change and change again,
We too ourselves, who of this world are part,
Not only flesh and blood but pilgrim souls....

Surely prescient to/for (for instance)
Loren Eiseley, "The Star Thrower" (1978):

"We are rag dolls made out of many ages and skins, changelings who have slept in wood nests or hissed in the uncouth guise of waddling amphibians. We have played such roles for infinitely longer ages than we have been men. Our identity is a dream. We are process..In modern terms, the dance of contingency, of the interdeminable, outwits us all.....Instability lies at the heart of the world...form is an illusion of the time dimension...the eternal struggle of the immediate species against its dissolution into something other...The power to change is both creative and destructive--a sinister gift which, unrestricted, leads onward toward the formless and inchoate void of the possible. This force can only be counterbalanced by an equal impulse toward specificity. Form, once arisen, clings to its identity. Each species and each individual holds tenaciously to its present nature....The evolutionists, piercing beneath the show of momentary stability, discovered, hidden in rudimentary organs, the discarded rubbish of the past. Man is himself, like the universe he inhabits, like the demoniacal stirrings of the ooze from which he sprang, a tale of desolations...But out of such desolation emerges the awsome freedom to choose--to choose beyond the narrowly circumscribed circle that delimits the animal being. In that widening ring of human choice, chaos and order renew their symbolic struggle.....

Last fall, Sharon Burgmayer, Andrea Friedman and I did a workshop on this topic: the delicate balance between stability and change, between safety and risk-taking, between security and novelty.

AND? while I'm trolling...
Paul's observation that "The micro/macro distinction in economics is a mirror of reductionist/wholist arguments in a whole variety of fields" also resonates for me in this--as well as in a range of other--contexts. It reminds me, for instance, of Ted Wong's discussion of Metaphor and Metonymy in the Brown Bags last semester, as well as a discussion this afternoon in the Language group, where we are now reading Jerome Bruner's Acts of Meaning. Bruner evokes C.S. Pierce's distinctions among "icon, index and symbol, the icon bearing 'resemblance' relationships to its referent as with a picture, the index a contingent one as in the relation between smoke and fire, the symbol depending upon a system of signs..." Icon is to index as metaphor is to metonymy as reductionism is to wholism as micro is to macro (as change is to stability...?)

Thank you, Mark, for teaching me "something about economics," by showing me how it re-plays patterns I recognize elsewhere.

Music of the Swarms: Who needs musicians when computers can think like bees? http://www.discover.com/mar_03/feattech.html

What might it take to create art? Can a flock of seagulls do it? Or does it take a soul, all that is human?

-Doug

There is clearly something that WORKS, in very satisfying and productive ways, in this working group. And something which, I think, is reflexively relevant to the explorations the group is engaged in. Looking inward, its worth noticing that the group itself is functioning as an emergent system, and trying to figure out what makes the group successful in this regard. My speculation:

• Each element brings to the group something different
  
• Each element in the group is open to being altered by all other elements in the group
  
• All elements in the group are simultaneously committed to the continuing elaboration of their own story and to the continuing elaboration of a collective story
  
• The group evolves collectively, in the absence of a leader or architect, because of an environment which encourages/supports the three previous items.
  

And looking outward: it is noteworthy that there is a sense of pleased "surprise" in many peoples' reactions (including mine) to what we're doing together. That suggests that the activity of collectively evolving stories which are satisfying/productive both individually and collectively is less common in academic life than it should perhaps be. That, of course, suggests that our activities could, beyond their local goals, play a useful role as well in suggesting/contributing to productive changes in the academic community at large. And that they could contribute to thinking about classroom environments/practices as well. Beyond this, my sense is that the lessons we're evolving are quite relevant to critical issues of finding better ways to conceive of national and world communities, and the relations among them.

I've heard about this project from time to time, but I didn't realize that anyone could participate. Maybe you already know about it? Anyway, it came up in a talk last week and I thought I would pass it along in case any of you are interested.

http://smallworld.sociology.columbia.edu/

I've attached an excerpt from the front page, below.

Best,
Panama

"In 1967, the Harvard Social Psychologist Stanley Milgram sent roughly 300 letters to randomly selected people in Omaha, Nebraska with the instruction to get the letter to a single "target" person in Boston using only personal contacts.

Milgram gave each "sender" some information about the target including name, location, and occupation, so that if the sender did not know the target (and it was extremely unlikely that they would), they could send the letter to someone they did know who they thought would be "closer" to the target. Thus began a chain of senders, each member of the chain attempting to zero in on the target by sending the letter to someone else: a friend, family member, business associate, or casual acquaintance.

Milgram's surprising finding was that for the 60 chains that eventually reached the target, the average number of steps in a chain was around six, a result that has entered folklore as the phrase "Six degrees of separation."

But is it really true? While Milgram's first experiment suggests it is, other experiments have been less conclusive, and no experiment has been done to test the theory on a global scale.

This is what we are trying to do and we need your help."

Among other things, it means that "emergence" is NOT a perspective which should be seen as discouraging "individual social action" (the issue came up several months ago in a grad idea working group discussion (which includes social work students) based on reading of Johnson's Emergence and earlier of Pinker's The Blank Slate; see http://serendipstudio.org/local/scisoc/grad/ideaforum/8oct02.html and http://serendipstudio.org/forum/newforum/gradideaforum02-read.html#3798)). The actions of an individual are in general NOT able to bring about arbitrary (wished for?) changes in collective properties, but they are also not, by the same token, irrelevant for such changes.

To put it differently, if individuals see themselves as contributing parts of social/cultural phenomena, then they will in fact be so. If they do not see themselves that way, they will continue to be so but will be less effective/"meaningful" contributors in terms of their own distinctive aspirations/desires. There are of course, as I suggested, immediate political significances to this conclusion (see The Place of the US in the World Community for an example of an effort to facilitate individually-influenced stigmergy and a modest proposal for an example of the possibilities and limitations of individual action).

And now returning, armed with the practical, to the theoretical ... Agency, in the sense of an agent being both influenced by and influencer of global properties, is a property of agents in any emergent system having stigmergic elements. It does NOT depend on either consciousness or free will. What DOES depend on those (and hence is a unique requirement for the internal instruction sets in the case of human interactions, and probably those of some other animals) is the existence of a "choice" as to whether to regard oneself as having agency. In THIS sense, agency is an emergent property (of evolution) and one which in turn makes possible (but not inevitable) the emergence of new phenomena (kinds of social organization) which were not previously possible (cf Free Will? and Variability in Brain Function and Behavior and The Brain's Images: Co-Constructing Reality and the Self and Getting It Less Wrong the Brain's Way: Science, Pragmatism, and Multiplism).

PS. What this makes me think of is a possible modification of the "segregated neighborhood" model which would evolve a solution to creating integrated neighborhoods. I THINK its doable, and if so would help to sharpen the question of what is/is not necessary for individual agency in the "free will" sense. Anyhow, thanks again to all for the stimulation, and I hope all this is stigmergically useful to someone.

For Mark, who (like me) is still trying to get a handle on this word, and (to do that) needs a short definition:

According to the Oxford English Dictionary (where I go for direction, whenever I am lost) stigmergy is "The process by which the results of an insect's activity act as a stimulus to further activity"; it was first used in 1959, in book called Insectes Sociaux: "The stimulation of the workers by the very performances they have achieved is a significant one inducing accurate and adaptable response, and has been named stigmergy." I can't reproduce the etymology of the word here in Greek, but loosely translated its roots mean "pricking" (stigma) + "ergon" (work). Mark was right--the root is the same as that for "stimga" (=the result of a mark; tatooed)--but w/out attaching any "stigma" (if you get my drift).

Which leads me (sort of!) to MY question of the week:

I want to separate what was put together far too casually for my taste during one moment of this morning's conversation, when identity was defined/identified w/ "following a certain rule set," and it was proposed that, in an emergent system, although behavior changes, those rules do not. My red flags went up. Like Karen, I'm ready to get beyond models now. I want to keep free will into the conversation--the will of thinking, self-reflective agents, and see if emergence, stigmergy, synergy are still useful ideas. I'm tired of talking about "stupid" agents (including the several instances of self-attribution that occured today).

Smartly yours,

A.

Taking a step back: why are we interested in stigmergy? Because we are looking for the mechanisms that explain how simple interacting agents can do far more as an interacting group than they can as a set of individuals. We are looking for the mechanisms that cause ants (for example) to do what they do, and also wondering in what ways we (for example) can affect our larger group/level/society.

But maybe the basic notions of cause and effect are too rooted in a reductionist world. Not that I'm advocating abandoning them completely, but merely suggesting that the way we normally think about cause/effect doesn't fit very well in describing an emergent phenomena.

Steve Grand has a sometimes-clever book, "Creation: Life and how to make it". In it he describes "the web of causality" which seems to be a better idea. The idea is that it is very difficult in an interconnected system to ascribe a particular cause to any particular effect. Agents can affect themselves. The experiences (historicity) of an agent affects its behavior. The environment affects its behavior. Social interactions affect its behavior. Newspapers can affect its behavior. The sum total of everything (locally accessable) potentially contributes something to the behavior of an agent.

If we give up on the simplistic notions of cause and effect, then we also will be left with very different "explanations" of a phenomena. In short, any abstract explanation will gloss over the details, and therefore miss the very core attributes that makes a complex system tick; it won't be an explanation at all! They will just be first-order approximations that don't have anything to do with how a system really works.

This may sound like I'm pessimistic on the future of science. But I'm not. I think that models are the key. Actual models don't just instantiate some abstract theory (if they did they wouldn't do anything interesting); they are the explanation.

So, I guess I'm wary of thinking about "stigmergy" as a tool that agents use to organize collective thoughts. Really, it is just us trying to abstract away from the gory details of a part of the web of causality. But the devil is in the details. Or, more optimisticly, god is in the details.

-Doug

http://www.mrwong.de/myhouse/index.htm

Well, I don't seem to have the right plug-in to add a floor to Mr. Wong's Soup'partments (emergence blocked by inadequate technology; access to said technology blocked by own stupidity), BUT Ted's invitation into that sort of "collaborative creativity" put me strongly in mind of two more architectural spaces in which you can play @ being the exploring ant, seeing if you can best find your way by re-tracing the path you yourself have just laid down or by finding a novel one (I AM going to keep free will--@ least in its "weak" sense as choice -- in this conversation!)

Try out both Serendip's House and Transformation . The latter site has one particular image "Inverting Power Games," which strikes me as a particularly wonderful example of stigmergy (and which Sharon Burgmayer, the artist, actually glossed over a year ago as "What might result if the power localized in a few players...was conjoined to involve all"). That god might be in THOSE details has already been imaged, too, @ Science and Spirit .

Wonder if ANY of this will have ANY relationship to the "emergent art" Tim will be talking about next week?
Eager to see,
pictorially yours,
A.

"Self-organization in social insects oftern requires interactions among insects: such interactions can be direct or indirect. Direct interactions are the "obvious" interactions: antennation, trophallaxis (food or liquid exchange), mandibular contact, visual contact, chemical contact (the ordor of nearby nestmates), etc. Indirect interactions are more subtle: two individuals interact indirectly when one of them modifies the environment and the other responds to the new environment at a later time. Such an interaction is an example of stigmergy. In addition to, or in combination with, self-organiztion, stigmergy is the other most important theoretical concept of this book. Grasse [157, 158] introduced stigmergy (from the Greek stigma: sting, and ergon: work) to explain task coordination and regulation in the context of nest reconstruction in termites of the genus Macrotermes. Grasse showed that the coordination and regulation of building activities do not depend on the workers themselves but are mainly achieved by the nest structure: a stimulating configuration into another confirguration that may trigger in turn another (possible different) action performed by the same termite or any other worker in the colony. Nest reconstruction consists of first building strips and pillars and finally the interpillar space is filled to make walls.... "

"Stigmergy is easily overlooked, as it does not explain the detailed mechanisms by which individuals coordinate their activities. However, it does provide a geral mechanism that relates individual and colony-level behaviors: individual behavior modifies the environment, which in turn modifies the behavior of other individuals."

[157] Grasse, P.-P. "La Reconstruction du nid et les Coordinations Inter-Individuelles chez Bellicositermes Natalensis et Cubitermes sp. La theorie de la Stigmergie: Essai d'interpretation du Comportment des Termites Constructeurs." Insect. Soc. 6 (1959): 41-80.

[158] Grasse, P.-P. "Termitologia, Tome II." Fondation des Societes. Construction. Paris: Masson, 1984.

Are two agents of the same kind because they share a rule set? I might be fine with this test, depending on how we define having the same rule set. I'm completely willing to say that the rule set for one cell in Conway's Game of Life is the same as the rule set for another cell, even if the two cells have different environments and therefore give rise to different behaviors. That's the whole point, after all. But I worry about the case of two different rule sets being tacked together into some big conditional. Like: follow rule-table A if timestep < 50; else follow rule-table B.

And like Karen and Anne, I want to think about this in terms of examples from nature. here are the examples I gravitate to. Ant-colony behavior. Harvester ants in the Arizona desert live in colonies which can get as old as fifteen years, even though individual ants don't live longer than one year. There is ant-level behavior (e.g., doing tasks, choosing tasks on the basis of frequency of interaction with other ants), and there's colony-level behavior (e.g., global foraging strategy, aggressiveness toward neighboring colonies). Here's what's cool: colony-level behavior changes even though ant-level behavior stays the same. Ants still choose their tasks the same way, but because the colony is larger and the nest has a different architecture (thereby changing interaction frequencies), colonies change their foraging strategies and aggressiveness over time. This, I think, is a perfectly good example of having the same rule set -- late ants share rules with early ants, but these rules play out differently because the environment or the global state has changed. Developmental genes. (And here I risk exposing my great ignorance of genetics, so please bear with me.) Some genes are for structures, different structures depending on where and when they're being expressed. The wheres and whens are determined by concentrations of signal chemicals, which are produced upon activation of other genes. The one example I have on hand is Antennapedia, which when rendered nonfunctional allows a leg to be produced in the developingfruit fly where an antenna ought to be. That is, it's a gene that somehow tells the developing structure to be an antenna instead of a leg.

In my schematic language, here's the rule: Make an antenna if you're in the head; else make a leg. Should that count as one rule? If so, then the differential behavior of the development process should be seen as emergent. I don't think it should count. Counting that would be like counting colony-level behavioral change if the ants had some time-dependent rule, or counting sandpiles if someone rigged it such that cascades obey a power law after some given time step.

"Are two agents of the same kind because they share a rule set? ... Here's what's cool: colony-level behavior changes even though ant-level behavior stays the same. Ants still choose their tasks the same way, but because the colony is larger and the nest has a different architecture (thereby changing interaction frequencies)..."

But how do interaction frequencies change the behavior of an ant? There must be a place in memory that is keeping track, and the rules must be sensitive to that change. We could represent that in a rule, like:

IF memory[4] > .45 THEN MOVE left

(consider memory[4] to be the 4th place in memory where some internal counter gets incremented with each encounter with another ant).

But if we allow that, then why not something like:

IF memory[4] > memory[5] * memory[1] THEN MOVE memory[2] direction

(where all of these memory positions are counters, sensors, and other internal states of all kinds).

In this manner, the rules could be the same for a million ants, but they would all have different behaviors... even if you had only one rule! Of course, if ants all had similar experiences, then they might all have similar (subtly different) behaviors.

I'm just pointing out that memory + memory-sensitive-rules can allow for much more individuality than those systems without memory, and the rules don't need to change a bit.

-Doug

P.S. - memory is just internal stigmergy, or stigmergy is just external memory :)

With all due respect to Bonabeau, Dorigo, and Theraulaz, and possibly (depending on what he means) to Doug, I think that the following way of talking about stigmergy:

"two individuals interact indirectly when one of them modifies the environment and the other responds to the new environment at a later time. Such an interaction is an example of stigmergy...individual behavior modifies the environment, which in turn modifies the behavior of other individuals" (B D & T)

"memory is just internal stigmergy, or stigmergy is just external memory" (DB)

misses something critical about the process, whether or not we want to use the term "stigmergy" to describe it.

It isn't JUST a case of individual behavior modifying the environment and in turn modifying the behavior of others (i.e., indirect or delayed feedback), it's rather that LOTS of individual behaviors pool into patterned aggregate changes in the environment (e.g., pheromone trails left behind by group activity, where individual variability tends to wash out) that in turn modify the behavior of groups of individuals. This is what links individual behavior to colony behavior. If we want to use "stigmergy" for the simpler case of delayed feedback, which is all that seems to me to be implied in "individual behavior modifies the environment, which in turn modifies the behavior of other individuals" fine; but then we need a new term for the fact that "individual behaviors pool into patterned aggregate changes in the environment...." "Colmergy"? (Anne where's your OED?) or perhaps "Durkmergy" in honor of Durkheim who may, if my memory serves me right and it may not, coined the term "collective representation."

A propos Doug's P.S., "colmergy/Durkmergy" is clearly much more than "just external memory." Internal (i.e., agent) memory, on the other hand, (at least what psychologists call "semantic memory" which is essentially memory for the generic rather than the episodic") may very well be a kind of "colmergy/Durkmergy" in that it is the patterned aggregate of "traces" (or activation levels between nodes or whatever is actually in there) in which individual variability tends to wash out. This is what gives memory its power and efficiency.

Rob

P.S. I hope that there is no stigma attached to those who neologize!

I agree with Rob that the collective nature of stigmergic memory (we should all have gone to Carol Bernstein's retirement events!) makes stigmergy different from memory. There's something here that's also why I disagree with the characterization of stigmergy as "delayed" whatever. What we describe as delayed is generally (always?) a discrete event. The effect of a stigmergic modification to the environment is no event -- rather, either it results in some event (the interaction, or contribution to behavior) after an unspecifiably long delay, or else its effects are distributed among (I want to say, smeared across) many future events.

I suppose this is the point of Rob's neologisms. If we take stigmergy to mean delayed feedback or delayed interaction, though, without the collectiveness, then I think we can simply dispose of stigmergy. Dynamical-systems people (my exposure is through population dynamics) have long known about the importance of time lags. There's no need to invent or invoke new ideas like emergence or even environment to account for time lags, even though time lags have been shown to be instrumental in producing chaos.

A few thoughts re: neologisms.

• Every new coinage seems like it might act as a sort of semantic bifurcation, and dynamical-systems people all know how bifurcations can lead to chaos.
• Neologists are what taxonomists call splitters, as opposed to lumpers. Neologists think the world is complex and that order can exist amidst apparent disorder. Paleologists (like Doug) think that complexity can emerge from a low-dimensional world. Rob and Doug are opposites and might cancel each other out.
• Are neologisms themselves emergence events? Or maybe stigmergic modifications of the environment. I suppose they could be both, depending on what the system is.

As per Ted's suggestion, I did go Carol Bernstein's retirement events, and found them quite rich and generative (see postings on Language and Time). I didn't find much there, though, to contribute to our discussion here: Carol's project (and those of her teacher and student who also spoke) has to do w/ re-capturing and re-presenting what has already happened (particularly that which is traumatic, catastrophic). The work is quite preservative in intent ("we must not forget") and quite melancholy in mode ("how can we bear to remember?"). The past pressed very hard on us during those discussions--and I found myself wanting the less time-bound work of emergence, which laments less the "absence of closure."

On the other hand, still musing over (one of the) questions left hanging last week, whether "criticality is critical," I found a melancholy note in The Chronicle of Higher Education about Gauging the 'Enough Point'" (and what happens when you fail to do so accurately):

"We are leaping across thresholds...That's how thresholds work: Up to a certain point something is good, and past that point there's trouble....Judging when you've reached this 'enough point' is, admittedly, no easy trick. You might stop short and miss some real improvement; you might overshoot and hit some wall....It's always going to be a guess, a question of feel, an art....But there are plenty of clues to alert us that we're near a technological saturation point, past which we will hit radically diminishing returns. The sheer speed of the world...overwhelms our ability to keep pace; we feel a kind of frantic restlessness which we sense will be alleviated only by slowing down."

More on neologisms later. Am slowing down for now.

Thanks, Ted, for what (to me, a word lover) is an amazing range of thoughts. To think of neologisms as "semantic bifurcations," as ways of making order out of apparent disorder, and as themselves emergent events, stigmergic modifications of the environment, just... blows me...back to the OED for more richness:

this term, for the "practice of using new words, innovation in language" first appeared in 1800: "Quaintness, the unavoidable companion of neologism, is...hostile to grace"
and again in 1858: "Neologism, in revolutionary times, is not an infirmity or caprice."
and 1867: "Since that day neologisms have fertilised the barrenness of our Saxon."

Most interesting, however, since the term was first introduced into this conversation by the psychologist in our group, is that it has a very particular meaning in Rob's discipline:

"An invented or concocted word or word-sound without recognizable meaning, freq. interpolated in otherwise correct sentences, and used by persons in a variety of neuropsychiatric disorders"; see

(1905): "Neologisms the meaning of which may remain absolutely enigmatical to the patient himself...are frequent in the period of dementia."
(1906): "In the typical form [of mania]...neologisms and symbols are found in great number."
(1932): "The verbal repetition of these 'new' words neologisms or senseless words invented by himself."
(1960): "Word-formations, which are so bizarre that they immediately bring to mind the neologisms of dementia praecox."
(1969): "Other patients refer the origin of neologisms to hallucinatory experiences."

So...assuming that emerging amongst us is...something really new, I propose we add a new (not @ all dismal) word, "neologismal," to our emerging vocabulary.  

Anne's reference to a passage decrying the "sheer speed of the world" brought to my mind William Carlos Williams's remarkable, breathless poem, To Elsie, or the Pure Products of America Go Crazy -- and especially the last stanza ("No one/ to witness/ and adjust, no one to drive the car"), in which Williams blames desperation, despair, and the disintegration of noble folk traditions on the absence of a central organizing agent.

We should occasionally pay more attention to the dark side of emegence -- how segregation emerges (thanks, Mark K.), how monopolies and dictatorships emerge, how Williams's Elsie, "her great/ ungainly hips and flopping breasts/ addressed to cheap/ jewelry/ and rich young men with fine eyes" emerges.

Of course, we'll think about all that with plenty of salt, noting that the passing of ethnic and cultural purity isn't all bad and that there may be extraordinary beauty in the "isolated flecks."

One possible point to argue in Rob's quote of B, D & T:

...individual behavior modifies the environment, which in turn modifies the behavior of other individuals

We should be careful to explore what causes what in the above example: is it the individual behavior of an ant that triggers the events? Or is it somehow the collection (e.g., the colony via the stigmergy)?

When the ant begins to move, we usually don't say that a cell in the ant's brain began making signals to get the whole body moving. Nope, we talk about the ant as a whole turning left by some rule.

But, when we talk about a group of ants, we talk about the individual ants modifying the environment. But we should be talking about the colony-level organization. For some reason we give special preference to talking about ants as agents, but not ant brain cells as agents, nor ant colonies as agents.

So, I see a direct analogy between what brain cells do, and what ants do. Do you want to call one stigmergy and the other not? I'm am a lumper and so I'd rather not try to make some artificial distinction between those two. Both of these systems are causing changes in the environment that can be interpreted as "information" in the proper context.

I'm not sure what the word is that desribes both of these situations, but here is the begining of a definition:

a signal generated by a part of a system that interacts with other parts (or other signals). The interaction causes two separate levels of a system to be integrated, causing emergent properties.

Here's a conjecture about the parts: it is necessary that the parts have limited capacity about what they are doing. For example, if the ants attempted to "understand" the "meaning" of the pheromone (and thus change their behavior), the colony-level organization would break down.

Said another way: a network of Einsteins would make a really lousy brain. "That Einstein is firing! Why? I don't understand what all the chatter is about; I'm thinking! You never fire when I want you to!"

-Doug

"No. The whole point is that there's no need for change to the agent's internal state. Colony behavior changes over the course of years, even though no ant (except the queen, and she seems not to be involved in these behaviors) lives even one year. Yes, memory is powerful and can allow for changes in behavior without changes in the rules. But memory must not always be necessary -- or stigmergy must substitute for memory."

So, just replace what I said amout memory[n] with sensor[pheromone]. You're right: one "rule" (no memory, just a pheromone sensor) and you have an infinite amount of variation in your ants, all with a single rule.

So, memory and stigmergy could perform the same function to a single ant. But stigmergy has the added benefit of being one-of-those-things-that-I-don't-have-a-word-for. Here, let me bifurcate:

• stigmergy + symbol or signal = stygmol
• interaction + information = intermation
• ...

http://www.cna.org/isaac/Glossb.htm

I'm assuming that these are definitions that those who worked on the ISAAC project developed. It seems that their use of words, like "agent" for example, is much more specific to their application than the sense in which we use the terms. I found it interesting to note the differences between their defintions of some terms and our use/meaning of them in our discussions.

My only objection: an agent's memory is accessible only to it, isn't it? Pheromone deposited on a trail is accessible to (and modifiable by) everyone.

More bifurcations (or coalescences?):

• stigmol + intermation = stigmermation
• stigmermation + Durkmergy = stigmermurkmergy
• beer + lemonade = shandy

SOMEWHAT INTERESTING
In the "Inventions" section he describes a couple of nifty games for generating comic-strip plots, 5-Card Nancy, and The Story Machine. Neither is actually implemented on the computer (this is why these are only somewhat interesting), but the graphics are cool.

HUGELY INTERESTING
In the "Online Comics" section, there's a reader-generated strip. First look at "Original Recipe Carl," which is a brilliant examination of narrative structure and modularity. Then look at "Choose Your Own Carl," which is the Carl strip implemented as a narrative network (multiple paths exist simulataneously), with panels suggested by readers.

Am also attracted by the similarity between such a "within range mismatch detector" and some ideas about how the nervous system fundamentally works (create model, test model by generating output, comparing resulting input to expectation ... cf http://serendipstudio.org/local/scisoc/mindbody/mindbody.html). Along related lines, it seems to me increasingly important to recognize that the unconscious/metonymic probably does NOT have temporal/narrative structure (a slight but important modification from http://serendipstudio.org/local/scisoc/time/time.html). So this has to emerge somehow from the bottom-up/top-down interaction.

Final point, I think there is something quite profound about the "AI doesn't know how to leave out the boring bits" remarks. In biological evolution, of course, there IS a more or less random generation of things, and what order one sees DOES result from leaving out the "boring bits" (this is what one actually means by 'natural selection'). Or, to put it differently, death is an essential element of evolution (and emergence?). See http://serendipstudio.org/sci_cult/philsci/s03/31march.html for an effort to think through the significance of evolution/emergence in a different context. And we'll hopefully get back to the "death" issue in an upcoming session on information.

Turns out it wasn't exactly free will I was longing for but rather a particular version of it: creativity (aka "aethestically useful surprise"). Thank you, Tim--and all the rest of you for further adumbrations.

I'm laughing with delight @ the neologismic (=orgasmic?) energy recently displayed by Doug and Ted. To paraphrase my friend Andrea Friedman, w/ whom I've been sharing these conversations: "I love the new words, love even more that your new words are new words about making up new words, and I love even MORE that you are making up words as you need them. That's chutzpah."

But now my attention thas been captured less by "emergent narrative" (which still seems to me indistinguishable from narrative-as-I-have-known-it) than by the emergence of new images. Paul and I are piloting a new course in the Bio, English and CSem Programs, called The Story of Evolution and the Evolution of Stories, so you can see why I was so engaged by Tim's presentation, this morning, on Creativity and Emergence (=Evolution, right??? RIGHT??)

The images we used to illustrate the syllabus were suggested by Andrea (see above re: chutzpah); each one is a "renga," or "linked image," generated when artists exchange and then modify computer graphics art works on telecommunication networks; they are VERY powerful individually and even more so in sequence. See http://www.renga.com for a display of a range of this work. It resembles in some fascinating ways the work Tim teased us w/ this morning: "Bush Soul," created by "Emergence."

Yum. Not sure yet about the death part of this, but...ready for more.

Well, I'm getting surer and surer about the death part of this (or @ least I've told my students I am).

Ted's talk this morning pushed the edges of my mathematical abilities/knowledge, but/so I found it a very rich one. I find myself wanting to understand better what was said early on, what "non-biologists can't seem to understand:" that "evolution does not optimize," is not meaningful, that evolving mechanisms "have no adaptive significance." And I very much want to understand better the implications of Ted's "suggestive" suggestion at the end of his talk about mutual information (about which I'd welcome further illumination/illustration/information...)

But what I found most generative was something Ted said in the middle: when "your cues are less good, you'll hedge your bets more"; that is, the MORE varied the signals, the LESS varied the resultant behavior. This is a very intriguing and troubling thought for me, as I think about applying it to human behavior. I'm reminded, for instance, of the conversation going on in the forum on The Place of the U.S. in the World Community, the notion that we respond to the world's increasing instability by (futilely) attempting to build constructions which will not change.

As Ann pointed out, there were comments this morning about frustration from those that study life (both biological and artificial) that others see evolution as an optimizer of some fitness function. Of course, when we use evolutionary models, that's exactly what we do: we optimize some function. We can also do it when we breed dogs or flowers: we can have in mind some measure of fitness (big and purple, or spotted) and select on those criteria. Purple flowers, spotted dogs.

But this is not what natural evolution does. Evolution has no "goal" or "fitness" of some task. Nonetheless, we have seen a "progression" from simple creatures to more complex ones. But it's not because complexity is nature's goal. It is a natural consequence of the process (limited resources).

Deepak and I are trying to develop artificial intelligence without having a fitness function to optimize. We see this paradigm in co-evolutionary models. In this way, the model's idea of fitness is grounded in the model. The problem is, though, we never seem to get the complexity to emerge out of these models like we see in real life. I suspect it is because of the limited ways in which the pieces can interact to produce new things.

-Doug

Bottom line is that evolution has had (even today) a bad wrap. As Doug says, it actually has "no goal", not even the "maximization of fitness", but there is a "progression" of SOMETHING. As Karen points out, "progression" does not mean improvement. But may, as both Karen and Doug say, have to do with "complexity" (see discussion of evolution of brain size). There's a clear explanation for this in what Gould called the "left-wall effect": with random change from any starting point, new things will appear in whatever space is currently empty (if one starts with maximal simplicity, hence at the left wall, the new things must appear to the right, ie at greater complexity). Note that it does NOT, contra Doug, depend on "limited resources", (this is the "bad wrap" which still hangs on, and which relates to altruism, as I'll get back to some time) but only on random change of a kind which can yield forms of increased stable complexity.

Upshot is that I suspect what's keeping our models from exhibiting the kind of progressive emergence they should given a consideration of emergence as evolution is a failure to understand what is required for "increased stable complexity" and to incorporate that into our models. I don't think its "limited resources" but it may be something having to do with the exploitation of entropy increases. I don't think I'll have this worked out in time for our next session but maybe some relevant background ... ?

I ask this because Gauss demonstrated, in what we now call the Principle of Competitive Exclusion, that two species cannot simultaneously occupy the same ecological niche. one species always outcompetes the other and drives it to extinction. It's only a slight exaggeration to claim that community ecology, one of the larger branches of ecology, sprouted and flourished nourished by the question that the Principle raises: why are there so many species in the world? Why are there a quarter million species of plant, when all plants are limited by only a handful of different nutrients? Why are there so many species of tropical tree, when the whole forest seems more or less uniform in moisture and soil quality? Why are there several species of marine invertebrates in the intertidal zone, when in the laboratory one species always eats the others to extinction? In general the answers have been versions of these two notions: (1) the world is composed of many, many niches, and (2) multispecies communities tend toward one species's exclusion of the others, but perturbations to the system always reset the clocks. That is, there are so many plants because plants make fine-scale distinctions among environments on the basis of subtle differences in the ratios of the available nutrients. Or, one plant would dominate the forest if not for the fact that hurricanes keep knocking back whatever species is ever ascendant.

Anyway, back to the point. Bacteria and we are around to enjoy each other's company, either because nature hasn't gotten around to killing bacteria or us off, or (more likely in my opinion) the environment is so complex that bacteria and we don't actually compete for the same set or resources. That is, if resources were limiting in a niche-relevant way, I think that either complexity wouldn't have evolved, or complexity would have evolved and we'd have lost the simple organisms.

Here's where I see a source of confusion: evolution isn't the same thing as natural selection. Any change in a population would be evolution and wouldn't necessarily be directional. Natural selection is directional and does take a population toward local fitness maxima.

I think evolution is best thought of as a phenomenon and not a process. The main process (or mechanism) is natural selection. (Whether natural selection is the main source of interestingness is open to debate these days. Emergence of order may also be important.) So I agree with Paul that evolution doesn't require scarcity of resources. But natural selection does require scarcity, and if we believe that selection is the main driver of evolution, then we should also believe that scarcity is required for most of the evolution we're interested in.

As you all remember from Intro Bio, the mitochondria in our cells are believed to be the descendants of free-living ancestors which were engulfed but (for some reason) not digested by the also free-living ancestors of our cells. Since that fateful First Supper, the two lineages (that of the host cell and that of the endosymbiont) have coevolved by a process which has probably been driven largely by natural selection.

Natural selection is of course all about the environment. What's interesting here is that the environment of the endosymbiont is the host cell. The host cell's environment includes more than the endosymbiont, but it's certainly the case that its environment is affected by the endosymbiont, though from the inside.

The endosymbiont lineage and the host-cell lineage both evolve. They move around in genotype spaces (where each location in the space corresponds to a genome sequence). How is one lineage's position determined by its position in the previous time step? By some selection rule, where the action of selection is itself partly determined by the position of the other lineage. So the endosymbiont lineage affects selection on the host-cell lineage (that is, on its environment!), and the host-cell lineage in turn affects the endosymbiont lineage's next position.

It's indirect, reciprocal stigmergy.

Now that I think about it, it seems to me that indirect, reciprocal stigmergy is common, not unique to endosymbiotic systems. Okay.

So... being the one who first "expressed some concern" about the pattern of death, and the one who has been worrying the matter incessently since--waffling between quoting Ovid's saying, there is "no death, but only change/And innovation," and giving sermons to my students about how death is essential to emergence--I'm impatient/eager/unsure I can wait til next week to find out whether it actually IS "necessary" or "just" a (bad? good?) side effect....

Anyhow, as a placeholder and sobering reminder of the LOCAL social relevance of these questions, see the e-mail we all just got advertising
*** BRYN MAWR COLLEGE ***
*** Better Dead Than Coed ***
***T-shirt now available in PINK!***

Am wondering, obviously, what role the insights of the Emergent Phenomena group can/could play/ARE playing in the "death" of the old/"evolution" of the new (could there BE a new?) Bryn Mawr....

When Anne first asked the question "is death necessary?" as I was doing the demonstration of the Genetic Algorithm and Genetic Programming, I said "No". The term "death" as an appropriate term for what happens to those "individuals" in the models: they are (usually) removed from the population and no longer contribute in any way to the "search".

I said that death wasn't necessary and maybe removing it form the model would be an interesting experiment. That is, keep all of the individuals around, letting the population size grow after each generation. But I think that that would be a very bad idea.

Death, in the model, does at least a few things:

1. Keeps the population size reasonable so that good combinations of two individuals are likely (probability is high they might "mate" using crossover)
2. Focuses resources (machine time and space) on the areas of the search space that look more promising compared to other areas
3. Makes room in the "probability space" so that novel things have a chance of catching on. Too much dead wood, and they get drowned (so to speak, because of course they wouldn't die) out

Death is as important in the models as limited resources and competition: without it, evolution of complexity couldn't happen. I believe that real evolution could have "solved" the "death problem" if that had been advantageous to do so. Keep in mind that the process we call evolution (and death) are an emergent properties of the system.

Natural death is the ultimate altruistic step of an organism taking itself out to make room for a fresh try. (Reminds me of a quip that Bill Wulf said when he was here last year: "Know how to change a department? One grave at a time.")

-Doug

Some comments on Paul's presentation.

• Outside the context of competition, fitness is meaningless. Fitness means reproductive success (including the survival ability necessary to allow the organism to reproduce), and unless we're talking about competition, there's no reason to talk about reproduction. If we're going to talk about evolution but not competition, we should be more precise in our language: evolution is or isn't directional in terms of size, complexity (defined how?), or aptness (apt as in ad-apt-ation).
  

• Isn't competition important in speciation? In allopatric speciation (so I'm excluding speciation that occurs as a result of weird chromosomal events, like polyploidy), a population is divided into two and the two subpopulations are prevented from interbreeding. While the subpopulations are separated, they become different from each other because of selection by different environmental pressures or simply because of random change. If the populations are reunited, they have differentiated enough that they are no longer able to interbreed -- or their offspring have reduced fitness.
  

  What does it mean for the hybrids to have reduced fitness? It just means that the hybrids have fewer viable offspring, on average, than do the nonhybrids. Because the hybrids and the nonhybrids compete, the hybrids are driven to extinction. What's more, since the production of hybrids is itself a behavior that should be subject to selection, the nonhybrids eventually evolve mechanisms that prevent their even trying to mate with members of the other subpopulation -- now species.
  

  What would happen in a world of unlimited resources -- where there would be no competition? I suggest that we'd see no species. We'd just see a huge blur of forms, and we'd all be mating (or trying to mate) with organisms which we'd otherwise call members of other species.
  

  And would evolution still work? As Doug suggests in his post about GAs, I think all that unorganized mating would probably just move us around already-explored regions of the phenotype or genotype space. Maybe. I think it's worth looking into in models.
  

  And anyway, is complexity or aptness meaningful or interesting if it's a property of individuals rather than of species? Because then it seems more haphazard: there's no guarantee that a particularly well adapted individual would have a similarly well adapted offspring. Complexity or adaptation or whatever would be the result of a random search, and we wouldn't be talknig about it in an emergence group (if such a group could exist).
  

• It's certainly the case that much directional evolution is driven by competition or its direct consequences. I'm mainly thinking here of plant-pollinator coevolution and other kinds of sexual selection. Sexual selection is when selection for sexual attractiveness or for other things that figure into frequency of mating drives evolution. Selection for pollinator-specialization has definitely been what's driven the diversification of many species of plant. Mate preferences and secondary sexual ornamentation is believed to be a big force in bird speciation. Peacock feathers, floral morphology, mating dances -- all traits that serve to isolate species, and all traits that probably evolved purely because of sexual selection.
  

  Why do I say that sexual selection is a direct consequence of competition? Because reproductive success is only important if there's competition. If there's no competition, then anything that reproduces at all will continue to be represented in the extant biota. What reproduces more might have greater representation, but if that matters then it's a kind of competition.
  

Ted

The great thing about species, though, is that it makes evolution all about discrete events. Branching events either happen or they don't -- and when they happen, they happen at precise moments in time. In fact, natural selection acts as a sort of noise filter immediately after branching events: behaviors that generate hybrids are selected against until species are so separated from each other (reproductively) that they don't even bother trying to interbreed. Noise filter -- exactly what an engineer might think to use to turn some analog signal into its digital representation.

Now, one thing we left off talking about earlier in the year is the relationship (is it a necessary one?) between emergence and discreteness. All our simulations are in discrete time. Our agents are discrete individuals, and their states are defined as sets not functions. If competition is what allows speciation to work, then competition is what makes evolution discrete and might therefore be responsible for emergence.

Is very interesting question, worth using to dissect terms and illustrate an argument I'm trying to develop. As Ted says, separated populations may come to be different "simply because of random change". To press the point further, separated populations may come, because of random change, to differ from one another even if the two environments in which they find themselves are identical. And in principle these randomly produced differences could be sufficiently great (or small but specific) so that individuals from each population are literally physically incapable of mating with individuals from the other. Bingo, "speciation" without anything even remotely resembling "competitition" and, for that matter, without any differences in "fitness". The story depends only on randomness and on the (perhaps revolutionary?) idea that different evolutions/emergences may occur under identical environmental conditions.

The story does, interestingly, depend on "death", ie it will yield reproductively isolated species only if the two founding populations of individuals die (otherwise they would provide opportunities for mating after the barrier was removed). And that's important, since it may help to illuminate the general significance of "death" (somewhat along the lies of one or more of Doug's suggestions). At the same time, there is no necessity in this story for death to relate to competitition or even to "differential reproductive success". It would be enough that things die after a fixed amount of time irrespective of their genes.

I trust Ted (and everyone else) recognizes that I am not asserting that this story is "representative" of actual biological evolution. There are lots of reasons to believe it is not. But the game here is to try and abstract some potentially generalizable ideas which in turn could help to account for similarities in global properties between biological evolution and other apparently quite different situations. If there are deep similarities then those similarities (randomness?), rather than idiosyncracies of particular phenomena (competition?), may help to better understand particular situations like biological evolution and be better guides to figuring out how to mimic emergence.

With that objective in mind, let me press the current story a little further in the case of biological evolution. Evolution has in fact created "aptness (apt as in ad-apt-ation)", and this must indeed, as Ted asserts, reflect "differential reproductive success" (assuming we discard the idea of a designer because we are impressed by evidence that differential reproductive success can yield "design", which I am and do). This is NOT, however, the same thing as asserting that is directional in terms of aptness. Indeed, my argument is that evolution is, in an important sense, demonstrably NOT directional in terms of aptness. Bacteria appeared first, but are no less "apt" than humans who appeared much later. Evolution by way of "differential reproductive success" does make "apt" things through local processes of differential gene contribtion to subsequent generations, but there is no strong evidence that things become more apt (or adapted or "fit") on larger scales or over longer time courses (while they DO both become larger and more complex (yes, interesting and productive definition problem) through a left wall effect). What this does is to pose an interesting question: is "differential reproductive success" at small scales and short times essential for particular aspects of the larger scale patterns, including, of most interest, appearance of new levels of organization? I'm not sure yet about the answer, but it is certainly possible that Ted's distinction between what is going on at different taxonomic levels could be pointing toward something quite significant.

At least for the sake of completeness (and perhaps for the general issues as well), let me make one other point in the biological context. "Differential reproductive success" should not be unthoughtfully equated with "competition". By the latter, many people tend to understand a process in which the success of one entity depends on the failure of another. To put it differently, what "competition" brings to many people's minds is the idea of "limited resources" and of a "zero sum game", so that one entity's increased "fitness" necessarily means decreased "fitness" for another. My assertion is that evolution in fact rarely proceeds under conditions of "limited resources" and is not generally best thought of as a "zero sum game". Instead, much of evolution involves cooperative adaptations which themselves increase the total possible payoff and the total available resources. This is important to understand as a counter-balance to "social darwinism" and related more subtle efforts to make use of the evolutionary idea in biology and other contexts. But my hunch is that it is equally important for the general emergence issue. Its my guess that the development of new levels of organization may in fact critically depend on systems NOT operating under limited resource, zero-sum game conditions. The appearance of multi-cellularity, for example, may have been the discovery of a way to increase the use of available "negentropy" rather than of something which could use a fixed amount of negentropy better than that which previously existed.

Yeah, we'll talk about "negentropy" tomorrow, along with information and death, and some of Doug's ideas (which, along with death, are ways of "avoiding unorganized mating" and hence maximizing exploration, which in turn is what the whole thing is about?, avoiding ). Very much enjoyed the shared journey, and looking forward to seeing where we get next. Here's to the new Bryn Mawr and beyond.

I think the problem is with the words "fitness" and "competition". Here's a story: two brands, er I mean, species of butterflys live in the same neighborhood. Let's say that, because of something having to do with soot and birch trees, the white butterflys get eaten. All of them. It wasn't really any "competition" of some measurable "fitness" quality (faster, stronger, smarter, etc) that led the black butterflys surviving. Rather it was something in the "web of causality" that led to one working better in the total system.

Out on a limb aside: from a scientific point of view, I don't think that we can ever say what "caused" the white butterflys to become extinct. It is too complicated and would involve such an interconnected chain of events that it becomes, in principle, impossible to identify (in a rigorous manner) the exact causation (see above on my ramblings on cause and effect).

However, the above situation is what I take to be meant when I hear the words "fitness" and "competition". The measurable fitness quality is defined exactly as the organisms fitness into the whole. "Survivor of the fittest" is a tautology, and that is exactly what gives it its meaning.

Is there a better word than "competition" that describes this scenario? I can't think of one, but it can bring to mind inappropriate analogies too. Where's George Lakoff when you need him...

-Doug

Yes, we DIDN'T get to "negentropy" nor even really to information. Sorry, but ... there's something interesting about that, in re pedagogy AND in re "emergence". With regard to the first, the issue of process versus content in the classroom is germane. The older I get, the more convinced I become that time spent seriously engaged in process is invariably more productive than time alloted in terms of content. For all concerned. To have "heard" about something is less meaningful than to have genuinely worked through the subtleties of important general concepts (which inevitably takes time). I learn better that way, whatever my classroom role, and don't think I'm at all unique (at least not in that particular respect). Moreover, getting through something one knows about is, as an objective, fundamentally inconsistent with the cardinal principle of "emergent systems": to create the circumstances within which the meaningfully novel can appear. Anne Dalke, Liz McCormack, and I have a manuscript on this, based in part on experiences with the emergent systems group, which some of you have seen and which we'd be delighted to make available to anyone else interested.

Having said all that, I still feel a need to reassure everyone that there WAS an imagined arc behind the conversation, and that it DID not only get through negentropy and information and death but even had at the end a pot of .... something. No sweat. Al and others will undoubtedly pick up some of the as yet unaddressed topics in future sessions, and, for the impatient, there are summaries and links in my notes. As for the pot ... That too is in the notes, but I'll summarize here.

My current hunch, based in part on recent reading of the beginning of Dennett's new Freedom Evolving (warning: the book is nominally about "free will" not about "emergence", and is probably misdirecting on that topic), is that we may actually already have the wherewithal in our hands to create true emergent systems which will generate multiple levels of organization, and this DOES depend, for very interesting subtle and not so subtle reasons, on killing Paul (metaphorically, and not exclusively but repeatedly). What has so far prevented us from doing so (creating emergent systems, not killing Paul) is not that we aren't "smart" enough (the universe was doing it long before "smart" organisms appear on the scene) but that we are in some ways TOO smart.

Thinking is not at all a bad thing. Some of my best friends do it, and I've even been know to engage in it myself on occasion. On the other hand, there are abundant reasons to recognize that thinking, at least in the formal sense, has serious and fundamental limitations (Hume, Kant, Godel, Turing, Popper, Penrose ... among others). In addition, it has some serious occupational hazards, among which one might mention tendencies towards conviction, arrogance, and impatience. Some times problems are better made sense of/solved by thinking less, by letting things flow a bit so that what one could NOT have thought of ... emerges. So maybe we're being too smart to let what could happen in our models happen? Maybe we'd see it given what we already know if we allowed/encouraged ourselves to engage in greater

• Humility, skepticism, patience, ambition
  
• Tolerance for uncertainty, tribes, conflict
  
• Willingness to onself engage in transcendence (change and/or death)
  

Sandy Schram passed on to me the syllabus for the doctoral seminar on Social Theory he'll be teaching this fall. On it I discovered, and read, Max Weber's 1918 speech, "Science As Vocation," which draws on the broodings of Leo Tolstoi to make a very interesting observation about the meaningfulness of death:

"for a civilized person death has no meaning...because...there is always a further step ahead of one who stands in the march of progress....Abraham, or some peasant of the past, died 'old and satiated with life'...because for him there remained no puzzles he might wish to solve....Whereas civilized man...may become 'tired of life' but not 'satiated with life'....what he seizes is always something provisional and not definitive and therefore death for him is a meaningless occurrence....civilized life...by its very 'progressiveness' ...gives death the imprint of meaninglessness."

In other words, death, as we've been using it (to mean erasure that creates space for the emergence of new things) is meaningless (at least to us as individuals). We need another word.

Neologists, to work.

This message is really addressed to Tim, but I thought others might enjoy listening in. I was blown away by this morning's discussion; it picked up on lots of things I've been thinking/reading/writing about for the past few years, in a few other working groups, about the nature and function of language, in particular regarding the different sorts of languages valued by scientists and humanists. See The Two Cultures for the initial articulation and the Language Group for a continued discussion of the idea that scientists aim for "no slippage," no "ambiguity," in their professional communications, while humanists not only revel in the play that ambiguity invites, but actually see the function of language as inviting and encouraging that sort of play, the sort of communication that can only happen when the receiver "doesn't quite get" what the sender sent.

What Tim's comments this morning added to these ideas, for me, was incorporating the whole matter of negentropy into what he and I fondly think of as "Derrida's Demon." Tim's very cute/acute questions:

• when there is a slippage of information, where does it go?
• how are multiple interpretations of a text (translated into my lingo: your guys' "multiple macrostates for one microstate" ) possible thermodynamically?
• IS the act of interpreting a "thermodynamic cheat"?
• is/what is the expenditure in consciousness?

Another note (and perhaps an oblique illustration of the above?): before the conversation took this turn, Tim was talking about the evolution of e-lingo, various shorthands used by some to speed up the typing process, and scorned by others (presumably the fast typists!) as impure and/or unnecessary. Tim suggested that these variations in accepted style are in some way a return to the era before printing, when there were multiple ways (and no standard way) of writing words. I was reminded of an account I just read in Stephen Jay Gould's The Hedgehog, the Fox and the Magister's Pox: Mending the Gap Between Science and the Humanities:

In the "early stage in the history of printing, publishers had not yet fully recognized the advantage and tranforming power of moveable type...[an early printed book] still uses cryptic and extensive abbreviations for many words, converting the entire text into a form of shorthand. These conventional abbreviations had greatly boosted the speed of production for texts, when each copy had to be written out by hand, but saved little time, and perhaps a little more space--but only at the cost of great ambiguity and difficulty in reading--when the type for each word only needed to be set once. Thus these abbreviations slowly faded from use, leading to our modern conventions of writing texts in full..." (127n.)

Such varieties of notation of course invite/insist on...
careful--and/but always uncertain--interpretation.

Better go get something to eat. Need to add energy in the system.
A.

Hm...I seem to be the only one w/ "energy" (=information) this morning. But, still chewing over yesterday's conversation, wanted to say a bit more...

Also quite resonant for me was Deepak's charge that we'd arrived, @ the very end of our session, at a contradiction: claiming BOTH that the generation of novelty "is the point" of emergent systems AND that the move (of the universe, of evolution, of exploration, of emergence) is always from improbable to more probable states....how can both those claims be true?

Since I find answers to abstract questions through concrete examples, I offer one here to the group. It's a gift to us from my friend Andrea Friedman (McBride 01), and evokes BOTH our delighted constructions of neologisms AND a process Doug described to us on 2/26/03:

"This guess-and-step methodology is actually called hill climbing due to the following metaphor. Start at a random place on a hill. Pick a direction to step. If the place you would step to is higher than where you are, make the step, otherwise stay where you are. This little algorithm will eventually take you to the top of the hill. However, it might might not take you to the highest place around because you could get trapped on a little plateau (i.e., you would have to step to a lower place before stepping to even higher ground)."

And here's Doug's hill-climbing, and ALL our neologizing, in poetic form:

The Hunt

Somewhere in the rolling hills and farm country
that lie beyond speech
Noah Webster and his assistants are moving
across the landscape tracking down a new word.

It is a small noun about the size of a mouse,
one that will be seldom used by anyone,
like a synonym for 'isthmus',
but they are pursuing the creature zealously

as if it were the verb 'to be',
swinging their sticks and calling out to one another
as they wade through a field of waist-high barley.

- Billy Collins

Kim Cassidy, Doug and I are co-directing a Summer Institute for Philadelphia Teachers: "Exploration and Emergence." When we met yesterday afternoon to continue our planning, Doug was puzzling about how "emergence" could work as a guiding principle for pedagogy. For me, that has been the clearest and most exciting application of the ideas we've been discussing, and "emergence" has become my newest shorthand for most of the progressive pedagogical strategies I employ: local hands-on interactions that produce unanticipated outcomes by validating multiple ways of knowing and thinking, and the relationships and interdependencies among the people doing the thinking.

I gave, as evidence of this process, what happened in our Emergence Group meeting yesterday morning : Paul came in w/ a series of (elaborately backed-up) propositions about the need for information loss and death, in order for new levels of complexity to emerge. By the end of a long hour's interaction among us all, w/ several of us providing the punch lines along the way, we arrived together @ several very different claims: that information, like energy, is not actually lost, but instead transformed, as it "moves up to another level," forming a category, an abstraction. This is not death, but resurrection after all!

While I'm reveling in this illumination, I'll take the opportunity to say that I'd like us to talk more about Rob's suggestion, just @ the end of our discussion, that Paul's "input-output" boxes are too static in their representation of how the nervous system works: it's an active "information-grabbing" device, and what it chooses to attend to is very important. I'd like to think about this matter of active attention.

A few afterthoughts. Rob is of course right in his assertion that I neglected two central aspects of what makes the brain an emergent system (in some ways even more paradigmatic than biological evolution, or at least more rapid). Both are in my notes (an autonomous, to some degree "random", generation of signals, and the output centered conception of the box which, with it, gives the nervous system a fundamentally exploratory character; the nervous system as Rob's "information grabber (see my earlier struggle to reach this conclusion). Both make the system in its function even more "irreversible", and hence history-dependent. In my brain/mind, we'd been over this stuff in connection with evolution and so, in presentation, I guess I let myself get preoccupied with the "new" point (the irreversibility at multiple levels which is both information "slippage"/loss and creation). But, its misleading to identify emergence in the brain ONLY with the latter. The ability of the nervous system to generate outputs independent of input, and to use the "reafferent loop" to test "expectations" are equally important in thinking of the brain as a paradigmatic emergent system. Similarly, Rob is right that the time parameter needs to be included (though I have some doubts, for other reasons, about the efficacy/significance of "contingency"; see Time talk and William James). I had the time embedded in the "irreversible" concept but should have made it more explicit.

Also re Rob: my GUESS is that "reflective consciousness" is in fact necessary for "consciousness", but that's a much longer conversation and the notion that there is information loss/gain in going from the unconscious to consciousness holds however it comes out. As for the idea that "attention"/"expectation" is an important emergent, not present in neurons, I need to think more about that one. Not EVERYTHING has to be found at the lowest level, as Mark noted, but as Karl Lashley wrote in 1951 "I am coming more and more to the conviction that the rudiments of every behavioral mechanism will be found far down in the evolutionary scale and also represented even in primitive activities of the nervous system". There is an interesting way in which the neuron might (like the nervous system itself) be seen as a device which generates output in a continuing effort to stablize its own state (the "expectation") in the face of input.

Back to the main theme (as it "emerged" in the talk): the idea that each level of organization "categorizes" the previous and so both gains/loses information (thanks again to Geoff and his leaky integrator question for triggering my thoughts along these lines). My intuition is that this is in fact an important piece of the "level of organization problem" (with the parallel in biological evolution/ecology being the appearance of new "niches" which in turn support new adaptive radiation? in developmental biology events like the early establishment of bilateral symmetry which in turn permit ... ? in social/cultural evolution new ideas/artifacts such as "christianity", "democracy", "communism"?).

If all this is anywhere near the right track, Doug (and Deepak) posed two important issues: is threshholding (and associated information "loss") essential for emergence? and are there particular "categories" which support further emergence or is that a general consequence of "categorization" irrespective of the particular categories created? My intuition is that there answer to the first question is "yes", that "edginess" (the category boundaries) is essential so that elements which simply record information are either very slow in creating new levels of organization or fail to do it entirely. My intuition is less strong for the second question. I THINK what happens, given enough time, is a kind of natural selection, that any category creating system will generate multiple levels but that it probably does so by exploring and allowing to "die" less generative categories.

And that, of course, brings us to the "death" and "information loss" questions (Anne, Tim). The information loss/creation link that we all noted sort of clarified and become stronger in my mind during the conversation. And I don't know at the moment exactly how to "concretize" it. There clearly IS in some sense information gain in the creation of categories, but I'm less sure that there is an existing form of information theory that would allow one to quantify and compare information loss and gain in, for example, edge detection (though I know some places worth checking). My intuition (again) is that it would turn out that in this case (and all others) that a modified form of the second law of thermodynamics continues to hold, ie that one is dealing with information "transformation" (rather than loss/gain), and that there is a necessary small information/free energy loss in all such transformations. I think working this out more formally might be generally productive, and hope Al's upcoming might help us with this. In any case, the point is indeed that "information" is "organized matter" and therefore as subject to the second law as any other matter/energy.

Viewed more generally (metaphorically?), "death" is not the same thing as "loss", in any system that has stigmergy. To put this less more ... poetically? .... entities in evolving/emerging systems continue to exist after individual "death" by virtue of their remaining influences/traces on other entities. I hesitate to call this "resurrection" (Anne's term), but do think we tend, in a very of contexts, to think too "essentially" ... and need to recognize that in evolving/emergent systems the meaning of entities is actually in their impact on other entities and these impacts can be quite extensive in time.

A final (for the moment) general point. Categorization=interpretation=abstraction, and there is LOTS of it going on UNconsciously.

Hi all!

A lovely, thoughtful follow-up to a great presentation, Paul. Thanks. One additional possible clarification...for what it is worth. I've been thinking about the difference between the neuron and the nervous system as input-output devices and the following undoubtedly somewhat strained analogy has suggested itself...the non-linear thresholding of the neuron forces it to act like a gate which is either open or closed...information is either passed along or it is not. With regard to the nervous system as a whole, however, a) information is actively gathered and selected rather than simply gated (although there is clearly an element of gating in that selection throws information away), i.e., the nervous system as a whole might better be compared to a paw, claw, beak, or hand than to a gate; and b) information can be gathered (i.e., attended to) at widely varying levels of detail from the virtually microscopic to the biggest of pictures. I don't see anything analogous to this at the level of the neuron.

Rob

Viewed more generally (metaphorically?), "death" is not the same thing as "loss", in any system that has stigmergy. To put this less more ... poetically? .... entities in evolving/emerging systems continue to exist after individual "death" by virtue of their remaining influences/traces on other entities. I hesitate to call this "resurrection" (Anne's term), but do think we tend, in a very of contexts, to think too "essentially" ... and need to recognize that in evolving/emergent systems the meaning of entities is actually in their impact on other entities and these impacts can be quite extensive in time.

Well, if we're going to be "poetic"...then let's not yet lay down "resurrection"--which refers to the act of rising from the dead or returning to life IN A DIFFERENT FORM (hence, "the resurrection body," the form in which we "will all" appear at the general resurrection--presumably not the one we now occupy, presumably something more "generalized," more "abstract" than our present particular concrete state).

Well, if we're going to be "poetic," then let's BE poetic. Here's a poet who observed in 1635 that death was NOT loss, that the "meaning" of (at least) one entity was indeed her "impact" on another (himself!):

THE DISSOLUTION.
by John Donne

    SHE's dead ; and all which die
        To their first elements resolve ;
And we were mutual elements to us,
        And made of one another.
    My body then doth hers involve,
And those things whereof I consist hereby
In me abundant grow, and burdenous,
        And nourish not, but smother.
    My fire of passion, sighs of air,
Water of tears, and earthly sad despair,
            Which my materials be,
But near worn out by love's security,
She, to my loss, doth by her death repair.
And I might live long wretched so,
But that my fire doth with my fuel grow.
        Now, as those active kings
    Whose foreign conquest treasure brings,
Receive more, and spend more, and soonest break,
This -which I am amazed that I can speak-
        This death, hath with my store
            My use increased.
And so my soul, more earnestly released,
Will outstrip hers ; as bullets flown before
A latter bullet may o'ertake, the powder being more.

If this sort of thing interests you @ all, you can read more @ The Works of John Donne, where you will find not only a poem entitled "Resurrection," and another called "Resurrection, Imperfect," but a sequence of twenty-three meditations, published in 1624, entitled Devotions on Emergent Occasions (?!).

Am thinking of penning a 21st version myself. Stay tuned.

Rob said, in part: "the non-linear thresholding of the neuron forces it to act like a gate which is either open or closed...information is either passed along or it is not."

Do we really mean that "information" is not passed on if the classifier doesn't fire? If we think of the neuron as a circuit gate, then the analogy would be that not firing is the same as outputing a 0 (rather than a 1). It is exactly this fact that sometimes the neuron fires and sometimes it doesn't that gives the firing its "meaning" and therefore makes both (firing and not firing) information.

So what is the litmus test to see if something is information? In information theory, the message is independent of the medium and would be a crazy thing to ask. It would be a confusion of levels. We create a message that we want to send via a channel, and as an outside observer, we can see the effects of noise on the signal. But when the message originates in and really *is* the medium, I don't think there is a method that would allow us to say that one thing is information and something else is noise.

We can easily see a single neuron as an information processing device because we can isolate the inputs and outputs, and see the firing of the neuron as a non-linear threshold function. But, we have a harder time doing that with an entire nervous system. When a system is sufficient complex enough that it can bend around and poke itself when it wants, then you can longer cleanly separate inputs from outputs. I am starting to think that information theory may not be the right way to think about active devices (rather than the passive in/out neuron devices).

In any event, neurons don't lose information, they concentrate it. They take a large number of inputs and summarize/generalize them. Given the appropriate organization of these categorizers, and a system can exhibit interesting behavior. But even if the neural system didn't do anything interesting, I think we have to say that it was processing the same amount of information.

Looking forward to Al's talk next week...

-Doug

Second, characterizing the firing of a neuron as input-output behavior and then to attribute that process to some kind of "information" transmittal is an even more gross approximation. Another viable alternative to the neuron's input-output metaphor is a "firing rate" metaphor. This is akin to Amplitude Modulation versus Frequency Modulation (better known as AM and FM on your radio dials). Some people claim that "information" is actually "represented" in the frequency of firing, rather than the transfer of a specific signal.

Third, trying to conclude anything from the input-output behavior of a single neuron is also imposing a "localized" information representation paradigm. It is generally conjectured that "information" is represented (and also transmitted) in a pattern of neuron firings and thus cannot be localized or even extracted from a study of individual neuronal behavior. It would, therefore, be mistaken to assume that a neuron is a "categorization" device. Perhaps a set of neurons are capable of acting as one.

Lastly, it may very well be that certain neurons in the body are specialized to specific functions. For example Paul talked about neurons in the retina being specialized as "edge detectors" (among other things). Not all neurons function this way. The more interesting thing about neurons in this context is the idea of "plasticity". While neurons in different parts of the brain differ (massively) from each other, there are instances, especially in developmental stages, where neurons from other parts of the brain can and do take over functions when there is physical damage. However, it is still not be concluded that all neurons function the same way (the neurons in the visual cortex are extremely specialized!).

Where do we go from this to physiological models of "information", "consciousness", "thinking", "death"? At best they are thought experiments....

As to the nature of "information", I need to think more about whether a failure to fire is equivalent to a "0" in a binary representation (Doug's idea). I'm not used to thinking about it in this way!

Consider a Cellular Automata, say Conway's Game of Life. Imagine two boards, one that has no cells "on", and the other that has one of those glider guns producing a pattern that regularly streams across the screen. I would claim that both of these boards are processing exactly the same amount of "information". That is, each cell on both boards either fires ("on") or it doesn't (like the neurons) and so the same amount of computation is occurring --- even though there is nothing "going on" on one board.

There is a difference between the two boards, but it is occurring at a level higher than that of the "information" processing of the cells. There are "signals" being propagated (the gliders), and those signals "can be seen" as information, when looked at from the right perspective.

The same can be said of neurons. Firing/not firing is information, but a "signal" sent either through frequency modulation, or some other method, is information "at another level".

What defines "another level"? Does it "really" exist? Does it take an observer?

Since it's clear that no one's willing (yet) to move UP several levels to talk about death, resurrection and the meaning of life...

I'm coming down to where you guys are playing. Doug said,

"In any event, neurons don't lose information, they concentrate it. They take a large number of inputs and summarize/generalize them. . . . even if the neural system didn't do anything interesting, I think we have to say that it was processing the same amount of information. "

I thought the point here was that, when information is compressed, some of it is lost, and lost irrecoverably; this is why we say emergence is one-directional, not reversible. When we were toying w/ some of these ideas in the Graduate Idea Forum this afternoon (yes, they are getting around....), Sam Glazier summed them up quite nicely by writing on the board

2+3+7=12

If you are given only the left-hand side of the equation, you can easily produce the information needed on the right. But if you have only the right-hand side, there are multiple possible configurations (12; or 24-12; or 1+2+3+4+3...) for the left; there's no reconstructing, no knowing from the sum alone, what the original equation looked like.

Moving up "a level" (yes, indeed--despite the best of intentions/the most open of minds, I have the humanist's prejudice against the essentializing, reductionist moves I see going on all 'round me....)--I wonder if we might return *sometime* to Rob's observations about the need for a distinction between self-consciousness and consciousness. I am in analysis now, and I had thought (til he made that observation) that the project was one of becoming more self-aware, of watching myself, my emotions as they arise, and thereby becoming less vulnerable to their movement, able to modulate them. But last night, as my daughter was talking about how self-conscious she is about her upcoming recital, her piano teacher suggested she aim for a "zen state." Might this be what Rob means by "awareness"? Marian might "try" (trick is, you can't "try") to be not self-aware, not self-conscious (not nervous!) but simply there, aware, conscious, @ ease w/ who she is, what she is doing....and might this also be a good outcome of analysis: to get beyond self-consciousness, to loop back to a kind of consciousness that is not self-aware/self-reflective? (To risk another poetic gesture: I find myself thinking here of Blake's "Songs of Innocence" and "Songs of Experience": a state of unconsciousness, then a "fall" into consciousness, then an emergence into a higher state of innocence...)

Hm. Wonder how long it will be til the Emergent Group is ready to move on "up" to these sorts of questions...??

Patiently,
Anne

I've been chewing on a number of the ideas that came up in this morning's conversation. I record them here, as a reminder to myself to continue thinking about them, and/or as an invitation to anyone else (in Al's teasing & resonant phrase, "in the fullness of time") to address/clarify ....

--the emergence of local complexity requires the 2nd law, but the 2nd law does not require (nor does it forbid) emergence (i.e.: the claim itself is not reversible);

--a single particle has no temperature and no entropy: is this simply definitional (that is, temperature/entropy are comparative states, quantifications/measurements located only in contrast?) or is it more "foundational" (i.e.: entropy, like emergence, arises only among particles/things in relationship w/ one another?)

--is this dependence on interactions related in any way to the observation that the message was in the "sequence," not in the "information" (the dots and dashes) of two lines of Morse code, one conveying a message, one scrambled, but both having the same entropy?

--"all interesting information processing has irreversibility": is this also just definitional (=circular), i.e.: what interests us is what we didn't know before? which is to say, everything that interests us is distinguished by its ability to surprise us?

--but that makes improbability (once again) simply observer-dependent, the result of our limitations as perceivers of knowledge;

--the scientists among us have been using the term "irreversible" to mean "not completely recuperable"; when we suggested that 2+5=7 could be thought of as "reversible," I think Tim and I were groping toward a less "technical" usage (that is, from just having access to the right-hand side of the equation, you can build back up the left; it may well not be the equation you started w/, but it actually is one of a fairly small sub-set of possibilities, and--depending on context--may serve just as "well" as the original);

--the query whether Shannon's theorem is "just a metaphor": if physics (like literature, like economics, like all disciplines) makes models of the world; if models are never "the same thing" as what they model, but always representations, or reductions, or metaphors; and if all model/representations/reductions/metaphors have their limits, the correspondence (and its predictive quality) will always be incomplete, will inevitably fail...

so...?

A Very Long Link

If that URL doesn't work try http://www.research.ibm.com/quantuminfo/ and click on the Landauer link, or just google the title of the paper.

From what I have surmised, computation (say a Turing Machine) can be implemented by reversible classic billiard ball-style physics. Building actual circuits that are reversible (or nearly reversible) creates computers that use less energy and generate less heat, because it is the erasure of information that requires it.

One might think that evolution would also build "computers" that use less energy. However, as Mark pointed out, there may be a trade-off between space (ie, the wiring to keep the info around) and expending the energy to erase it.

In that light, stigmergy is neither: an agent doesn't keep the information, and it doesn't get destroyed. It just gets left laying around.

-Doug

1 - I have, for the first time, clearly understood the limitations of Boltzmann's effort to make sense of the Second Law in terms of Newtonian physics (reversible), coupled with probability. I'm still a little murky on the history here (Loschmidt vs Poincare) but have very much on my mind Poincare's assertion (his "recurrence theorem"?) that Boltzmann may have accounted for a probabilistic trend but failed to account for "irreversibility".

2 - Its my hunch/intuition that this was/is a serious issue, one that has taken physics/information theory off on an inappropriate side-branch. My inclination is to try and back up to that branch point, and see if some existing problems can be avoided, and some open issues settled, by taking a different path ("two paths diverged in a yellow wood / and sorry I could not travel both ...").

3 - I THINK that "irreversible" is well-defined in information terms, if less well so in mechanics/physics. Sam Glazier's arithmetic equation (see Dalke above) is a pardigmatic example. In fact ALL of the basic arithmetic operators are "irreversible" functions in the sense that once they have operated it is not in general possible to recover exactly from the resultant the original terms (yes, they are limited but more often than not to an infinite set of possibilities, what I called many moons ago a "bounded variant" set; there are "traces" of the original terms but the one to one correspondence of inputs and outputs has been lost; the operator, as George Weaver puts it, is a function in the forward but not in the backwards direction). This is in general true of logical operators as well. And it is the sense of "irreversible" that I was referring to earlier when I talked about "information loss" associated with threshholding elements and behaviors in the nervous system. Most generally (perhaps?), irreversibility is a property of most transformations from higher dimensional to lower dimensional spaces (a line in 3D space collapses to a point in 2D space). While the Poincare argument persuades me that there is no necessary "information loss" in Boltzmann's reversible probabilistic system, I'm comfortable (I think) that there IS a real/inevitable "information loss" in the situations mentioned. And that "information" in these terms can at least in principle be defined in an "observer independent" fashion.

4 - The challenge then is (perhaps) to skip over Boltzmann/etc and try and define "information" with regard to the loss occuring in clear "irreversible" processes, and perhaps THEN go back and use that to see whether it can provide a clearer definition of both the second law of thermodynamics AND its resistance to Maxwell's Demon type phenomena. My guess is one would come out with the same linkage between "information" and "matter/energy", but have a clearer characterization of the nature of that relation.

I'm still dubious about fully reversible calculation (for reasons inherent in the above), but have yet to check the reference (or look for additional ones). Doug's C.H. Bennett is the same Bennett who, with Rolf Landauer, established the notion of "erasure" as the key constraint on Maxwell's Demon. Bennett, by the by, has an article on "Chain Letters and Evolutionary Histories" (perhaps relevant for Panama's upcoming presentatio) in the June Scientific American.

Some worthwhile links:

How Molecules Defy the Demon ... Review by Rolf Landauer of Maxwell's Demon by Hans Christian von Bayer, 1998 ... reviews history, including role of Charles Bennett and "algorithmic entropy", mentions earlier "authoritative" volume, Maxwell's Demon edited by Less, HF and Rex, F. (1990).

Mechanics and Irreversibility, a discussion including reference to Josef Loschmidt and his objection to Boltzmann's analysis

Henri Poincare, The Principles of Mathematical Physicsz, 1905

Notes on the History of Reversible Computation, C.H. Bennett, 1988

Trespassing Limits: Pynchon's Irony and the Law of the Excluded Middle ... The novelist Thomas Pynchon on entropy

What amuses me hugely here is that Paul's posting about the centrality of irreversibility centrally involves the action of backtracking (I have a hunch that I'm operating on a totally different level here, but this seems a pretty clear demonstration of reversibility). I've just finished an intense week of immersion in Doug's "Bible," Hofstadter's Gödel, Escher, Bach (so, arriving twenty years late, some of us are still open to revelation...) Among 1,000 other things, I have learned about the problem-reduction operations built into the AI language PLANNER: particularly the recursive process that's enabled by creating a tree of subgoals, subsubgoals, etc. If one path in the tree fails to achieve the desired goal, then the PLANNER program will backtrack and try another route. "Backtracking" seems to be the magic word as far as Planner is concerned...and, I've been convinced by Hofstadter, as far as emergence is concerned as well. So (@ whatever level) I'm revising Robert Frost's "Road Not Taken": It IS very much possible to "be one traveler" and take two paths, to jump outside the system, entertain the counterfactual, get the perspective of an overview, then "loop" back into it w/ an awareness of the "slippability" and alterability of the various nested contexts which define it....

I regretted having to leave Jim's presentation this morning, just as he was quoting Stephen Lansing's observation that "whereas in biology 'natural ecosystems evolve through a process of "blind" natural selection...[in anthropology] systems...are by definition shaped by conscious human intentions'"

--because this is precisely where the discussion gets puzzling (and so interesting) for me: what happens to/how useful is/how extended can the concept of emergence be, for describing the interaction (and evolution of action) among agents who are conscious: not ants, not termites, but thoughtful human beings? 

I hope that the distinction that Jim brought up this morning and that Anne has highlighted (emergence and/or intentionality) can be explored more fully next week when I lead a discussion on Emergence and Intelligence.

Here's a conjecture I'll throw out now and hope to defend next week: the more intelligent an agent is, the less likely it will be part of an emergent system. Or, said another way: the more "intentionality" an agent has, the smaller the effect it will have on higher level emergent systems.

(Although, I don't use the word intentionality as it seems to be beyond the reach of science.)

-Doug

My son Sam graduates from high school tonight. Searching for something meaningful to give him, prodded in part by our earlier conversation about "being one traveler" and having to chose between alternative paths (or not), I decided on Robert Frost, in particular his poem "Design'":

I found a dimpled spider, fat and white,
On a white heal-all, holding up a moth
Like a white piece of rigid satin cloth--
Assorted characters of death and blight
Mixed ready to begin the morning right,
Like the ingredients of a witches' broth--
A snow-drop spider, a flower like a froth,
And dead wings carried like a paper kite.

What had that flower to do with being white,
The wayside blue and innocent heal-all?
What brought the kindred spider to that height,
Then steered the white moth thither in the night?
What but design of darkness to appall?--
If design govern in a thing so small.

Which is to say, Doug, I very much want us to get to emergence, intelligence AND intentionality next week, and to think together--"scientifically" or not--about the relationship between the two latter terms, in the context of the former--
Anne

Thanks, Jim! Yesterday's discussion was fascinating, and it's exciting to see how some of our more abstract ideas play out in different settings.

I still have two concerns about Lansing and Kremer's claim to have demonstrated that the organization of traditional Balinese irrigation decisions is emergent. My first concern is the same one I brought up yesterday: if there's no tradeoff between subak-level payoff (for choosing one or another irrigation schedule), and if there's no competition for resources among neighboring subaks, then there's no reason to think that the resulting distribution of irrigation strategies reflects anything other than the distribution of what's optimal given local environmental conditions.

Have you ever seen those "Pin Art" toys they used to sell in executive-toy catalogues? (Here's a picture. Another. Here's a yellow one.) You press it against your hand or face or against some other object with an interesting shape. As some of the pins are pushed up higher than others, the toy reproduces the contours of the object and you get an accurate (but imprecise) reproduction of the object's shape. The accuracy of the reproduction does not emerge, though, from anything like among-pin dynamics or anything else interesting. Each pin gets its correct final position directly from the object. The object is completely responsible for the shape of the reproduction -- the toy contributes nothing. The toy processes or transforms no information. I guess all it does is lose information.

That's what might be happening with the irrigation decisions. Each subak is like a pin, and its final irrigation schedule is like the pin's final position. If a subak chooses its best irrigation schedule for its local environment, then the final pattern of schedules is just a point-for-point reproduction of the pattern of environmental conditions. That itself wouldn't really be emergence, any more than eery hands and faces appearing in Pin Art is emergence.

That's my first concern -- here's my second. In the simulation, each subak chose its next irrigation schedule on the basis of neighbors' schedules and resultant yields. But which subaks counted as neighbors? It's possible that the final clustering of irrigation strategies simply results from who counts as whose neighbor. If it turns out that the final clusters with like schedules are also the clusters of subaks that counted as each others' neighbors, then the final result was sort of built into the model at the start.

But if both of these problems turn out to be true, it's still a very intereting study, even in terms of emergence. It's at least as interesting as other searches of solution spaces, like in my plant-allocation searches or like in biological evolution.

I just came across this article by Scott Camazine in Natural History Magazine called "Patterns in Nature". You can read the article at, http://www.amnh.org/naturalhistory/0603/0603_feature.html . It summarizes some things that we have talked about over the past several months.

Also, Camazine is a co-author of "Self-Organization in Biological Systems", Princeton University Press (2001), which provides numerous case studies of biological self-organization. The link, http://www.scottcamazine.com/personal/research/index.htm , has more information.

-Panama

It's abstract states:
"Most models of natural selection assume either that the material environment remains constant or that it fluctuates in ways unrelated to changes in gene frequencies (and therefore changes in the distribution of phenotypes) of the organism undergoing selection. In this paper, we consider what happens when this assumption does not hold, that is, when ecological feedback between organism and environment is included in the evolutionary process. Specifically, we examine the unusual evolutionary dynamics that occur when changes in the distribution of phenotypes (resulting from selection) alter an environmental parameter in ways that, in turn, modify selection pressures. This process, which we term "system-dependent selection," produces stable phenotypic diversity which functions to regulate the relevant environmental parameter within a much narrower range than would occur in the absence of ecological feedback. This environmental regulation raises the mean fitness of the population and reduces variance in fitness among different phenotypes. Thus, system-dependent selection produces functional organization at the level of the system as a whole, rather than at the level of the individual organism. We use James Lovelock's model of the imaginary planet Daisyworld to describe the unusual dynamics of this selective process and then use a similar model to examine the structure of an ancient system of wet-rice farming on the Indonesian island of Bali. This model accurately predicts the actual structure of functional organization along two Balinese rivers. We investigate the stability of such systems by exploring the conditions under which mutant phenotypes can invade Daisyworld. The results suggest that the phenotypic diversity and functional organization produced by system-dependent selection may be maintained when there exists variation, over evolutionary time, in the environmental parameters underlying system-dependent dynamics."

For those of you with Macs, Lansing has posted his download simulation modelMacintosh only
.
He also has another study available on the Skokomish River.

I'm preoccupied with bringing an edited volume to press and have not had time to read through this thoroughly but will do so in August, when I return from Greece. Jim

I spent this weekend talking with (among other people) my brother-in-law, Pete Dalke, who works for the Oregon Department of Environmental Quality (DEQ). He was explaining to me how much of his work now acknowledges that traditional top-down governance systems can't respond to the challenges of sustainability; in their stead, all sorts of local decision-making structures are being established (see, for instance, Governance for Sustainability).

The totally different path towards emergence I found myself wandering down this weekend showed up in the 6/15/03 NYTimes Magazine article, "My son, the Cyborg":

"Rodney Brooks, the head of the M.I.T. Artificial Intelligence Lab, told NPR recently that while we could make a robot play chess, we were still flummoxed by the challenge of creating one that, like a human 2-year-old, could come into a place they've never seen and point to things and say 'chair,' 'cup,' 'table.' But the related dream of making humans a little more like cyborgs seems closer to reality ....Experiments w/ 'bionic technology,' like brain implants that can restore vision to the blind or mobility to the paralyzed, are now proceeding with human subjects....Superthumbs and Robocop-like visiual scanning could be regarded as virtual implants, incorporating machine intelligence into human bodies without surgery...."

My guess, for the sake of the record, is that both Anne's hope and Doug's conjecture reflect an understandable but not to be satisfied hope that "intelligent beings" (ie humans) are somehow qualitatively ifferent from ants and termites in their relation to "emergent" phenomena. "Thoughtfulness" (I agree with Doug that "intentionality" is confusing term, best avoided, because it can exist without "thoughtfulness") is an attractive, real, and ocassionally significant aspect of human nervous system function, but it is notorious for overestimating its own significance. One example of this is the general failure of "urban planning" efforts to achieve self-sustaining urban communities. Conversely, there are the examples of existing cities (and water networks) that are self-sustaining and that "emerged" in the absence of "thoughtfulness". It seems to me that to the extent that anthropologists actually believe that their objects of study "are by definition shaped by conscious human intentions" they are likely to be missing very much the largest part of the boat (and its explanation). For the moment, the most significant implication of emergent systems research is, it seems to me, how LITTLE of human activity depends on "thoughtfulness". Maybe once we get that straight, we can use it as a framework to help identify and study the much small part that does reflect thoughtfulness.

Along this line, another thought prompted by Jim's talk has to do with what I think is an important "assymetry" in thinking about levels of organization in emergent system (see Grad Idea Forum). One could in principle "explain" phenomena by moving either up or down among levels. BUT trying to account for lower levels in terms of upper ones can get one into serious trouble if the historical reality is (as it appears to be) that upper levels emerged from lower ones. There is always a conceivable upper level explanation for lower level properties (the god's eye view); if the systems are realy history dependent, the constraints on stories are better working from the lower to the upper levels (ie from neurobiology to anthropology rather than from politics or economics to anthropology).

Let me also beat a dead horse a little more. What is going on in the water distribution example is making the pie bigger; it is NOT competition in the zero sum game sense. And it is NOT "optimization" in any useful (ie small numbers of variables) sense. It is simply "exploring", with improvement as an outcome rather than a guide or objective. Indeed, when one ATTEMPTS to optimize in terms of some well defined variable or set of them one frequently ends up disturbing a previously well-functioning system (because it reflects a balance among lots and lots of variables).

Last thought, related to upcoming discussion of network architectures. Figure 6 of Jim's presentation distinguishes networks with different information flow patterns in a way that may be quite generally relevant/significant.

I'm misquoted and misunderstood above: am NOT invested in nor especially hoping that humans will turn out to be different from/better than ants and termites in our relation to emergent phenomena. I'm just trying "manfully" to UNDERSTAND HOW our being "thoughtful" agents works into/contributes to/interferes with the process of emergence (the notion that our faculty meetings are generally such miserable failures because too many thoughtful people are trying to control the "upper level" of the discussion is a marvelous case in point here!)

Anyhow, our conversations around Doug's presentation this morning helped me a lot in this thinking--and I want to lay out one of these strands. The first "bead on the string" is Jan's mention of "theory of mind," which was very resonant for me. It links directly to the second bead: in preparation for the upcoming Summer Institute on Exploration and Emergence, I've just finished re-reading Steve Johnson's Emergence, which postulates that our awareness of our own minds thinking is the result of our awareness of other minds doing so. The third bead is the insight provided by Turing's Imitation Game--whether he himself intended/was aware of it or not-- that we can determine a system's intelligence by asking it questions: that is, by INTERACTING w/ it using language. In other words, intelligence might be defined as a property identified by OTHER human beings in interaction w/ the "creature whose" intelligence is in question. As Paul suggested this morning, this notion links very nicely w/ the fourth "bead," the idea developed by Mikhail Bahktin in The Dialogic Imagination that language is always a dialogue, always social, always a struggle between a speaker and a listener.

Bingo. End of string: my current "definition" of intelligence is "the ability to imagine the mind of another...the ability to imagine that the mind of another is DIFFFERENT from one's own...." What's key here is NOT rationality but rather IMAGINATIVE INTERACTION.

Relationally yours
Anne

P.S. I've just noticed another "string"--which leads to a "correction" of Doug's observation that AI researchers find the Turing test a terrible method of judging intelligence because, among other things, "it leaves the self open for tricks." Doug also just lent me another book by Hofstadter, Le Ton beau de Marot: The Spark and Sparkle of Creative Translation, which is a profound reflection on the non-formulaic, fantasy-filled, always-tentative mental exploration underlying translation, which Hofstadter presents as paradigmatic for what it means to be human and alive. It is as remote from a computational approach as one can get --and it is full of tricks.

Above, Anne concluded:

Thanks to all for listening to my personal odyssey to EI! All feedback welcomed!

-Doug

There is a nice symmetry to Doug's position. Intelligence/rationality involves the sending of signals that you yourself understand, and by symmetry such an entity can comprehend and evaluate the signals of others (passing the Turing Test). But once an entity (understood as a level of emergence) becomes that self-aware, there are Two Impossibility Theorems:

1) No further levels of emergence can occur because the entities are now little Einsteins who are unfit as components for developing more complex systems.
2) It is impossible to understand, in the standard way, an entity this complex.

So self-awareness implies a level of development/complexity that prohibit's complete understanding of oneself.

This is cute, but is it true.

Impossibility Theorem 1) seems a lot like the "end of human evolution" view: that is, that humans by being able to manipulate their environment are able to prevent further human evolution from occurring. I don't know how true this is for human evolution (though I doubt it), but I don't see any good argument why the little Einsteins can't be merged into more complex systems. Impossibility Theorem 1) is, in a sense, contradicted by the existence of human society.

Impossibility Theorem 2) not only implies the impossibility of cognitive science (as one of Doug's critics said), but also social science (and I fully grant that social science does not deserve its name). My main point here is that it is way, way too early to throw in the towel on creating models that can adequately explain complex processes. All models are abstractions from reality, so they always do not fully represent the underlying processes. The great successes in intellectual history are precisely those models which somehow successfully explain things without merely replaying the things themselves. Finding the right set of abstractions and pushing through to their implications is what creativity in theory is all about. I see no inherent reason why this process cannot continue.

Theory of Mind
The branch of cognitive science that concerns our understanding of the minds of ourselves and others has come to be called "theory of mind," though it should perhaps be called "theory of theory of mind." http://cognet.mit.edu/MITECS/Entry/frith

http://cognet.mit.edu/MITECS/Entry/horst

(Sorry, long. Thought this was a helpful summary, but the URL doesn't seem to be working now.
http://host.uniroma3.it/progetti/kant/field/tom.htm
.... A "Theory of Mind" (often abbreviated in TOM) is a specific cognitive ability to understand others as intentional agents, that is, to interpret their minds in terms of theoretical concepts of intentional states such as beliefs and desires. It has been commonplace in philosophy (see Davidson 1984; Dennett 1987) to see this ability as intrinsically dependent upon our linguistic abilities. After all, language provides us a
representational medium for meaning and intentionality : thanks to language we are able to describe others people's and our own actions in an intentional way as in : "Ralph believes that Mary intends him to persuade George that p". According to this view, the intensionality of natural language, that is, its suitability for expressing meanings and thoughts, is the key for understanding the intentionality of our theory of mind.
A major challenge to this view came from studies on primate cognition and comparative psychology. In their 1978 famous paper : "Does the chimpanzee have a theory of mind?" D. Premack and G. Woodruff
argued that experimental evidence of chimpanzees' understanding of human behaviour could be interpreted as detection of intentions. Although Premack and Woodruff experimental data have been challenged by other primatologists (see Tomasello & Call, 1997, ch. 10), there is a growing evidence showing that non human primates have some intentional understanding of their social world (see Byrne & Whiten 1988; Tomasello & Call, 1997). The presence of such a capacity in non human (and obviously non-linguistic) species lead to the conclusion that it was possible to investigate TOM as a biological endowment independently of language.

The "False Belief Task"
A more focused perspective on TOM comes from developmental psychology. Children show a precocious ability to understand intentions and other important aspects of the mind (as gaze direction, attention, pretense). Nevertheless, in the early 80, the psychologists H. Wimmer and J. Perner showed that a full-fledged TOM doesn't develop before the age of 3/4. They set up a series of experimental tests in order to check whether children between 3 and 5 years of age were able to attribute a false belief to someone else. In one of these experiments, children see a scene in which a character, Maxi, puts chocolate in a drawer and goes away. While he is away, his mother takes a bit of chocolate for cooking and then puts it somewhere else and goes out. Then Maxi comes back, and the experimenter asks: "Where will Maxi look for the chocolate?". The 1983 original results showed that children over 5 did not have problems in attributing to Maxi a false belief, whereas younger child ren predicted indifferently that Maxi could look for the chocolate where his mother has put it. Further experiments lowered the threshold of attributing false beliefs to 3/4 years of age. The false belief task , as it is called, defines a sharp watershed between a stage of child's development in which children have a sort of "transparent" reading of mind and reality (people believe what it is the case), and a stage in which they show a capacity of having an "opaque" reading of mind and reality, that is, they can easily distinguish between what is the case and what people believe is the case. This has been taken as an important piece of evidence of the development of a domain specific ability in dealing with mentalistic concepts, such as believe, which doesn't seem to be available in earlier stages.

Which format for TOM?
Although there is general consensus that TOM is a domain specific theory whose inferences don't extend to other cognitive domains, there has been a lot of debate revolving around its format. Results on false belief task indicate an abrupt change during the third year of age. This lead many psychologists and philosophers (see Leslie 1997, Baron-Cohen 1995, Fodor, 1994) to describe the underlying cognitive structure responsible for TOM as an innate module, that is activated around three years of age. As in the case of language, the TOM module is dedicated, specific, fast, automatic, at least partly encapsulated, and its functioning is largely independent of intellectual general capacities of the individual. It can be specifically impaired or function in the presence of other mental impairments. This view fits with the evidence that comes from experimental studies of severe psychiatric impairments as autism, (see Baron-Cohen 1995, Frith, 1994). Autistic children have a significant lower performance on false belief task compared to other cognitive tasks for testing intelligence and language capacities. This lead to the hypothesis that autism could be the consequence of a specific deficit of the Theory of Mind Module (TOMM).
Other scholars (see Carey, 1985; Wellman, 1990) have argued for a "theoretical" model of TOM : instead of seeing it as a mental mechanism, they conceive it as a naive theory, with posits, axioms and rules of inferences. Mental states such as beliefs are theoretical entities, the posits of this theory. In this perspective, often called the Theory Theory it is not possible to pry apart our concepts of mental states from the set of inferences that individuate them within our theory of the mental world, as it wouldn't be possible to separate the concept of acceleration from those of speed and time in a physical theory.
Furthermore, theories change during development : this may lead to genuine conceptual "revolutions", to use the famous Thomas Kuhn's metaphor for theory change in science (see Kuhn . These radical changes of paradigm make a theory at a certain stage incommensurable with its earlier stages. Some advantages of this position are that it better explains the articulation of the development TOM with other children's abilities as mindreaders, as for example detection of desires (see Gopnik et al. 1994), pretense (see Perner 1991), emotions (see Harris, 1989).
A striking different hypothesis, suggested in the mid-80s by Robert Gordon (see Gordon 1986), is mental simulation, that is, the idea that our capacity of psychological understanding depends on our ability to run cognitive simulations. According to this view, it is possible to infer other people's intentions and future actions by using our own mind as a model for theirs. All we need is to be able to run a decision process "off-line" : to pretend to be in other people's shoes and see how our mind would resonate as if we were in the pretended context. Simulation doesn't involve a complex theory of mind : it involves a capacity of pretense and of putting oneself in the other's place. Its advantages are (1) that it can easily explain the emergence of pretense at a much earlier stage of development than that of TOM, given that it considers pretense as a completely different cognitive resource, and (2) that it is a much more economical explanation. Crucial evidence for this model may come from studies on the first-person/third-person ascription of beliefs in children. A series of experiments has investigated self-ascription of beliefs in order to check whether children were better psychologists of their own mental states than of others'. Evidence has been discussed (see Gopnik & Astington 1988, Gopnik 1993) that shows a symmetry between first-person and third-person grasp of intentional states. Children are no more reliable about their own mental states than they are about others'. This seems to suggest that our own mind is not a better model for mental life than others'. However, the discussion still goes on, and other results have been put forward in order to defend the simulation model (see Davies & Stone, 1995).

Evidence from autism
Research in clinical psychology is one of the main areas of application of theory of mind. Autism is one of the most severe psychiatric impairment that can occur during the early stages of development. It is a rare deficit, touching 4/5 children out of 10.000. Its symptoms range from anomalies in social communication, absence of imagination, isolation, lack of capacity to involve in social games, to an almost total impairment of cognitive functions.
In 1985, U. Frith, S. Baron-Cohen and A. Leslie advanced the hypothesis that the central symptoms of autism (anomalies in social interaction, communication and pretense) could be explained by a specific deficit of TOM. They adapted the false belief task to autistic children and ran experiments with a control group of Down children. Although autistic children had better cognitive performance than Down children in many cognitive tasks, they massively failed the false belief task. Furthermore, the amount of successful performance didn't increase significantly with age. This lead to the conclusion that one central component of autism is a specific deficit in mindreading, and not an impairment of general cognitive abilities.
Still, there is a small percentage of autistic children who actually succeed the false belief task. If autism is defined as a specific deficit of TOM, how is it possible? Experiments have shown that these "talented" autistic fail nonetheless in more sophisticated mindreading tasks, as the second-order false belief task (see Perner & Wimmer, 1985; Happé, 1994) in which subject are asked to attribute embedded mental states as in "Ralph believes that Peter wants that Mary thinks..."). Although they may have some rudimental mindreading ability, they lack the full-fledged metarepresetational capacity that is fundamental for communication (see Sperber 1994b).

Theory of Mind and evolution
Comparative studies with other primates lead psychologists and primatologists to speculate about the phylogenesis of TOM (see Byrne & Whiten 1988; 1997). If TOM is a specific cognitive module, whose function is to detect information within a particular cognitive domain (psychology), it could be the product of a selective pressure that conferred fitness advantages to individuals endowed with mindreading abilities. Furthermore, a complex cognitive module is constituted by sub-modules that may indicate some interesting facts about the phylogenetic history of the module. S. Baron-Cohen has argued (see Baron-Cohen 1995) that TOM recruits other modules for its functioning, as an Eye Direction Detection module (EDD) and a Shared Attention Detection module (SAD). These two modules are clearly present in other species. Comparative studies may lead to a more precise understanding of the evolution of these abilities.
The best known evolutionary hypothesis for theory of mind is the social intelligence or Machiavellian intelligence hypothesis , according to which : "the social environment might have been a significant selective pressure for primate intelligence" (see Byrne & Whiten, 1997, p. 2). Primates show a surplus of intelligence that overcomes the immediate survival needs, as eating, avoiding predators, feeding offspring, etc. According to the Machiavellian intelligence hypothesis this surplus intelligence might have en advantageous for social manipulation, deception and cooperation. This suggests a slightly independent evolutionary history of mindreading abilities from that of language. (see Sperber,
forthcoming)

Holding up my end of the discipline-spectrum, by passing a thin rope, made of poetry, across the chasm that stretches before us til "Emergence" meets again on July 9...

I wanted to share with you Howard Nemerov's "The Dependencies," which seems to me a beautiful illustration of so much we've been discussing these past months:

• the difficulty of laying out sequentially, in language, the interlocked webbiness of emergent systems,
• the necessity of death in their production,
• the moments of criticality, the thresholds when change occurs...

The route the poem took to get to you (from Cassandra Fraser @ UVA, to me, to Andrea Friedman, who just returned it to me this morning.... ) is itself a demonstration of interdependency (as is this linked posting...)

Also: the epigram, by the well-known physiologist Albrecht van Haller also provides an amusing link to our work: van Haller proved the concept of "irritability" of tissue, distinguishing between nerve impulse (sensibility) and muscular contraction (irritability). Shades of intentionality. Trying to read intelligence. What goes on Inside. Theory of Mind.

Anyhow: here's the poem.

The Dependencies

Natura in reticulum sua genera connexit,
non in catenam: homines non possunt nisi
catenam sequi, cum non plura simul
possint sermone exponere.

Nature knits up her kinds in a network, not
in a chain; but men can follow only by
chains because their language can't handle
several things at once.

-Albrecht van Haller

This morning, between two branches of a tree
Beside the door, epeira once again
Has spun and signed his tapestry and trap.
I test his early-warning system and
It works, he scrambles forth in sable with
The yellow hieroglyph that no one knows
The meaning of. And I remember now
How yesterday at dusk the nighthawks came
Back as they do about this time each year,
Grey squadrons with the slashes white on wings
Cruising for bugs beneath the bellied cloud.
Now soon the monarchs will be drifting south,
And then the geese will go, and then one day
The little garden birds will not be here.
See how many leaves already have
Withered and turned; a few have fallen, too.
Change is continuous on the seamless web,
Yet moments come like this one, when you feel
Upon your heart a signal to attend
The definite announcement of an end
Where one thing ceases and another starts;
When like the spider waiting on the web
You know the intricate dependencies
Spreading in secret through the fabric vast
Of heaven and earth, sending their messages
Ciphered in chemistry to all the kinds,
The whisper down the bloodstream: it is time.

- Howard Nemerov

Seems to me that this is indeed an issue relevant to thinking about emergent systems in general, as well as in the context of economics, politics, and world affairs. How DOES one avoid the "budweiser effect"? Could it be related to Doug's concerns from last week? And maybe to thinking more about US foreign policy?

Some of us are finding insights emerging in the beer we drink (or read about). Some of us are finding them @ church. I went this morning to Bryn Mawr Presbyterian w/ Paul Burgmayer, who is running a summer series on poetry in which I'll be presenting. The presenter today was Bill Brower, who "spoke" Robert Frost's poetry--from memory--for an hour, straight (and explained to me afterwards that the key is letting your mind make the associations--"naturally. You don't have to think--if you do, you'll forget"). Two of the poems Bill "spoke" put me delightfully in mind of emergent phenomena (misunderstood as beginning w/ "little Einsteins"); the poems are very cute both in imagining conscious intention on the part of small creatures AND in denying it to large ones (both are also--thereby--quite satiric about the academic enterprise).

In the first of these, "A Considerable Speck (Microscopic)," the speaker spies a dot on a sheet of paper and recognizes

...unmistakably a living mite
With inclinations it could call its own.
It paused as with suspicion of my pen,
And then came racing wildly on again
...again it turned to fly.
Plainly with an intelligence I dealt.
It seemed too tiny to have room for feet,
Yet must have had a set of them complete
To express how much it didn't want to die....

I have a mind myself and recognize
Mind when I meet with it in any guise.
No one can know how glad I am to find
On any sheet the least display of mind.

"Departmental" describes the activity of an ant who runs across a carcass:

...he no doubt reports to any
With whom he crosses antennae,
And they no doubt report
To the higher up at court.
Then word goes forth in Formic:
"Death's come to Jerry McCormic...
This is the word of your Queen."
And presently on the scene
Appears a solemn mortician,
And taking formal position
With feelers calmly atwiddle,
Seizes the dead by the middle,
And heaving him high in air,
Carries him out of there.
No one stands round to stare.
It is nobody else's affair.

It couldn't be called ungentle.
But how thoroughly departmental.

In today's NYT, a story about how high-concept movies and TV shows may be outcompeting the traditional blockbusters, which relied more on star power than on concept. Leonardo DiCaprio and Madonna used to dominate marquees and Billboard charts; today the biggest sellers are starless movies (e.g. Finding Nemo and 2 Fast 2 Furious) and no-name musical acts (e.g. Kelly Clarkson and Justin Guarini).

It's all about this self-organized marketing, like in the Pabst story which Mark pointed us to. Today's story makes two points that I think are relevant to our other discussions. First, the concept must be simple. The co-head of motion pictures at Dreamworks is quoted as saying, "You need concepts that can be easily grasped, that can be described quickly in the schoolyard or at the grocery store checkout. You need urgent word of mouth." So interactions among actors must be simple and efficient. Word-of-mouth marketing of TNN's Stripperella (the network's commercials say, "She's an exotic dancer by night, and a sexy superhero by later night") is bound to work better than word-of-mouth marketing of the excellent but hard-to-describe BBC workplace comedy The Office (which doesn't even have a promo sentence). Doug's Einsteins have no role in promoting Joe Millionaire.

Second point: the concept must be compelling. That is, the interaction (the cashier telling me that I should go see The Hulk) must be self-powered. People must want to recommend the product -- Paul might put it in terms of each recommendation's increasing the entropy of the universe. "Urgent" word-of-mouth.

(also, from a review: I haven't read yet)
Cultural Software, by J.M. Balkin (Yale, 1998)
Applies theories of cultural evolution and the theory of memes to the problems of ideology and justice. Instead of resting the concept of ideology on notions of "false consciousness," Cultural Software shows how ideological effects get produced through the spread and reproduction of forms of cultural know-how, or cultural software.
Human beings are the bearers of this cultural software, it helps constitute them and shapes them as persons with distinctive values and purposes. Yet cultural software reproduces whether or not it serves the interests of human beings. Rather, cultural conventions and institutions spread as if they had their own interests in survival and reproduction. And some kinds of cultural software can act like virtual parasites, breeding unhappiness and injustice as they reproduce in human minds and institutions.

More musings
(this my compensation for our not meeting this week...
I just take the conversation inside...
but then can't seem to keep it to myself....)

Anyhow, today, trying to think through a problem of my own, I found myself thinking about a key aspect of emergence, which Ted may have been the first to flag, when he called our attention to the "hugely interesting" aspect of reader-generated Online Comics, in which "multiple paths exist simultaneously." What I'm calling "key" here is the need for branching/exploring multiple avenues for possible solutions: a process which not only turns up unexpected answers, but also functions as a guard against the vulnerability of having only one.

I came across two examples of this today: the first is in a novel I just finished, Ursula LeGuin's The Telling, in which one of the beleagured protectors of all that is good says, "We were stupid...Carrying everything up here. We should have left it all over the place. Left the books with whoever had the books, and made copies. Spent our time copying, instead of bringing everything we have together where they can destroy it all at once....Now we' ve got our treasure where we can't use those technologies."

Now (stay w/ me here) that science fiction scenario has enormous resonance for contemporary world politics, which seems to be a current interest of our group. I'm on the mailing list for Tikkun, in which Michael Lerner just wrote an impassioned letter about "The Predictable Consequences of a Flawed Road Map":

"because the Road Map calls for the cessation of terror as a prerequisite to any peace negotiations, fundamentalists on both sides know that in order to pre-empt peace, all they need to do is perpetrate violence on the other side. Because of this, the Road Map, as it is currently formulated, guarantees the immediate escalation of terror and actually rewards violence. Their aim, to stop the peace process, the Road Map tells them, can only be achieved in one way: they must plant bombs! That is why there has been a serious escalation of violence since the signing of the agreement."

I find this analysis very striking, and very, very useful: it shows that in laying out a single plan one also lays out a single way of blocking it. The alternative offered by emergent systems thinking is the notion that, if there are multiple paths, the way can never be blocked. Shades of the Traveling Salesman Problem: a very different kind of Road Map.

Without losing sight of the need to model this way of thinking in campus politics (don't forget the group performance scheduled for the next BMC faculty meeting!), perhaps the emergence group might also want to write a group letter to our congressmen, to suggest a different way of thinking about both national and global interventions.

I'm serious: this kind of thinking really could make a difference in our political culture and in the way we do social action.

I think.

Re: Jan's inquiry re: memes, I've occasionally thought that the meme meme would make a nice case study for an essay on some of the differences between scientists and other kinds of researchers. The meme has been pretty successful as a meme: I don't hear people talking about memes as often as I hear them talking about weapons of mass destruction, say, but I've heard meme talk for many years -- probably with similar frequency year after year. It hasn't taken hold at all, though, in biology. As a bridge between disciplines, it seems to be sort of one-way. It's encouraged some cultural theorists and social scientists to co-opt some of the ideas and language of biology, but it hasn't brought any of the language of culture or society into biology. Why? I have a few ideas, but it's a topic probably better suited to a Science in Society Brownbag discussion.

As far as the emergence discussions go, I think memes are an interesting example of simple models which sacrifice mechanistic realism and quantitative precision for generality. Most of the models we discuss are simple and abstract: simple actors interacting by simple rules. Because they're so simple, they're as close to representing one real-life thing as another real-life thing. A cell in a cellular automaton could be a city block or a rainforest tree, since the only things that matter in the model are things both blocks and trees share: conditions, locations, neighbors. By proposing the notion of memes, Dawkins really just pointed out that organisms and ideas share several features -- and that if these features were to be represented in a simple model (like an epidemiological model), then this model might say things about the dynamics of all sorts of things, like viruses, songs, gossip, lice, metaphors.

Since reality television's been on my mind, Anne's recent post reinvoking the multiple-plots thing caused me to notice that most reality-television shows share some key features with emergent systems. (Have we discussed this before? I remember Tim talking about reality TV in one of the morning meetings, but I don't remember the context.) Most reality shows have two interlaced components. There's usually some sort of game in which the dozen or so (non-famous) participants compete, live, make out, or otherwise interact with each other. The real fun of the shows, however, is in the relatively unstructured scenes in which the participants all eat, drink, and sleep together with no scripts and little direct manipulation by the shows' producers. In these scenes, tensions from the game components blossom into rivalries, alliances, sexual liasons, and sexual betrayals. The audience gets much of the thrill of soap operas, but none of it's scripted.

So, it's all there: no author, many actors, structured interactions, unstructured consequences, unpredictable outcomes. Sometimes the enterprise succeeds, and we get compelling drama: Married by America was frequently gripping, sometimes heartbreaking. Sometimes we get pure boringness: as far as I know, nothing beyond an occasional flossing crisis has ever happened on Big Brother.

I agree that much of the drama in reality TV comes from editing. And I agree that this fact might hurt my claim that reality TV is emergent showmaking.

But editing occurs in several other processes which we've been calling emergent: evolution by natural selection, GAs, training of neural networks. Maybe we should consider reclassifying these as order-generating but not really emergent. Oh, and we also come back to the which-phenomena-do-we-single-out-as-interesting (or -ordered) problem. Is there anything inherently special about the things we've been calling "surprising"? Is Langton's ant highway really different from any other ant spaghetti? Are glider guns (in Conway's Life) really more interesting than traffic lights? And Wolfram's CA classes -- what did we decide about them, again?

Sorry to dredge up old questions. Maybe we should put together a FAQ list.

"Emergent pedagogy" = a new emergent, itself reshapable as it is further thought through by interacting individual/collective stories. For the moment, nothing more/nothing less than "teacher" and "student" both as active learners, engaged in somewhat unpredictable interactions out of which come new less wrong stories at both individual and collective levels and greater skill at creating useful new stories.

An old story (to some) in new trappings. Cf.

• BiCo Ed Program Philosophy
  
• Making Connections ...
  
• This Isn't Just My Problem ...
  
• Science and Education
  
• Bridging Cultures in K-12 Education: Exploration and Emergence.
  

Advantage of new trappings is that it provides broader connections (biological evolution, brain organization, lots of amusing computer models in lots of realms) to justify what good teachers know anyhow, and the occasional new insight by comparing education to other parallel contexts that might otherwise not have occurred to one (eg importance of noise/randomness in classroom?). And, in reverse, suggests that what works in classroom should in fact be recognized as more relevant in other contexts. For the latter, see War is a Bad Metaphor and Theorizing Interdisciplinarity.

As Alison and I were en route to the conference sponsored last week by the International Association for Learning Alternatives, I also found myself trying to explain "Emergent Pedagogy" to her. I discovered, in the process, that we already have "considerably more" than what Paul describes above. Under the rubric of "Resurrection," I'd listed a number of characteristics:

• local hands-on interactions that
• produce unanticipated outcomes
• by validating multiple ways of knowing and thinking,
• and the relationships and interdependencies among the people doing the thinking.

Such practices have long been common among the progressive educators @ the IALA conference; for instance, Alison and I shared a panel w/ Chris Mercogliano, who wrote Making It Up as We Go Along. This is the story of Albany Free School, the oldest inner-city independent alternative school in the United States. Alison and I were both astonished and delighted to learn that it was founded (in 1969) by a BMC alum, Mary Leue, and that it has @ its center a "council meeting system" which (shades of BMC faculty meetings!) is guided by Robert's Rules of Order. Any one can call for council at any time, to address any problem; a quorum is not required, and a simple majority rules.There is no problem this self-correcting system can't handle, Chris claimed, including the problem of its own dissolution, which has happened more than once!

I see Albany Free School, and its simple starting condition, the council system, as a great illustration of the sort of "editing" Ted was describing above, a process which is both order-generating and, I'd say, emergent.

Looking forward to exploring these possibilities further on Wednesday--

Anne

On another matter entirely...

My husband reads an entirely different set of journals than I do, but keeps his eye out for articles that might be of interest to me, including those that illustrate what he says are only sort of jokes I like: those about male pratfalls (even better, from his point of view: those in which the pratfallen men turn out to have the last laugh). Yesterday he gave me a piece from the "science and technology" section of The Economist, which is a perfect example of the latter, as well as of the necessity of death for emergence/evolution:

"the orb-weving spider...inserts...two sperm-carrying organs, known as pedipalps, into his mate and then actively terminates himself....Matthias Foellmer of Concorida University in Montreal and Daphne Fairbairn of the University of California, Riverside...explain in a paper in the Proceedings of the Royal Society that the palps of dead males are fixed in an inflated state, making them hard to remove. As a result, dead males may act as plugs to prevent other males from copulating, ensuring that the suidical male, not a rival, fathers the off-spring. In short, who dares win will die--but his genes are more likely to live on in the next generation."

...My only insight is that people themselves (ie their personalities) are emergent phenomena. They are continually being influenced by many small factors that each have imperceptible effects on their personalities (sometimes there are very big events, but I think we overemphasize their significance and I'll ignore them - by this I mean, if a child's parents get divorced, that is a very big event in that child's life, but how the child reacts to it is the product of the gazillion prior events and genetics, each with mostly small influence, that created that child's personality). The divorce, to be sure, has an independent effect, but I think we exaggerate these big events because we can see them. There are also huge positive feedback loops involved: people like what they are good at, they practice what they like, practice makes them good at it.

In the face of this, formal education has little independent influence on children's lives. They take from it what their personalties have predisposed them to take form it (some take a lot and some take almost nothing and it is hard to influence this).

I was delighted that we were able to spend this morning's session discussing the aspect/application of emergence which is for me by far the most interesting and potentially productive--that of pedagogy. It was a further delight to have new company in our conversation (Kim Cassidy, Alison Cook-Sather, Nia Turner).

Granting the "emergent" belief that the present is accounted for--albeit indeterminately--in terms of the past, I began the discussion by suggesting that we try thinking together about

• the significant initial conditions for "emergent" pedagogy (my claim: give up on "coverage," in order to gain space for "inefficient" discovery; Paul's refinement: choose content that illustrates/gives students experience in particular WAYS of thinking; that is, choose material that motivates discovery)

• how much editing/guidance/goal-direction is consistent w/ an "emergence" rubric for pedagogy?

• how important/significant (pushing @ the envelope of the usefulness of emergence for thinking about teaching) is it that pedagogy is enacted w/ 'n among the "active information-grabbing devices" that are the nervous systems of our students? how much difference does it make, in other words, if the actors in the system are thoughtful ones, each w/ their own agendas?

• the need for a materialist base in our teaching: an accretion of detail out of which thinking emerges
  
  
• we can't help our students develop thinking skills w/out the content to think with
  
  
• through the process of thinking for themselves, of learning to make and solve puzzles, students will inevitably acquire the content they need for doing so
  
  
• rather than exploring an alternative to traditional methods (replacing a "standard" w/ an "exploratory" curriculum, or "content" with "process"), we are searching for an effective means of integrating the two components: selecting content for its usefulness, because it facilitates a process of discovery (rather than because someone else has declared it "important")
  
  
• this is a means of addressing what Deepak called the "Olympic-games phenomena" (i.e.: since new sports are always being added, some must be dropped; in classroom terms: you can't cover it all!)
  
  
• it is also a way of facing the "illusion of efficiency" (shared by students and teachers, who think that delivering/being delivered content is "learning" it)
  
  
• is the "directive" way ever "the best"? in an emergent system, we can't ever know, for sure, what it is our students need to know
  
  
• how to handle the "uninformed self-editing" enacted by our students, as they choose deliberately not to engage in the classroom (out of fear of what might emerge, if they did?)
  
  
• how to reconcile conventional psychological notions of "readiness" (always constructed developmentally) w/ questions of motivation?
  
  
• children first recognize themselves as learners in kindergarten; part of the project here is preserving the exploratory nature of those early years thoughout the educational process
  
  
• is it possible to "go back" to that point in childhood where our students (hey: we ourselves) wanted to make their own categories, rather than accepting those designed by others?
  
  
• memorizing can be a useful learning technique, not just as a way of storing information, but as keeping labile various methods of "retrieving it," by keeping "in trim" the "associative" function of the unconscious
  
  
• might we think of the optimal classroom as one which gives access to a variety of intelligences, to everyone's ways of thinking? by insisting that the primary task of educators is to help students develop the ability to think independently, are we simply imposing a new orthodoxy?
  
  
• this sort of teaching may well involve a delay in gratification: assessment may well not be simple or immediate
  
  
• this sort of teaching and learning is both more different and more dangerous: giving both teachers and students more responsiblity means we are also giving both more room for error
  
  
• what is the psychology of taking "bubble tests" (multiple choice tests are actually designed to confuse; each range of answers includes, along w/ a right one and a wrong one, a "distractor").

As we closed, Alison recommended two books relevant to our discussion:

Jerome Bruner, The Process of Education and
Raymond Callahan, Education and The Cult of Efficiency.

I create this archive of our conversation both for myself, as a resource for the Summer Institute on Exploration and Emergence, and in the hopes that we will be able to continue the conversation both here in the forum (what did you students think about all this???) and in the group in the fall.

I've been at work this weekend on a new project about, oh, commodification of research and learning, and the cycle of reciprocity and community-building that is gift-giving, trying to reach for an alternative form of "gifting" that is free, no strings attached (following Derrida), "never (a) present."

En route to figuring all this out, I found myself reading an essay by Jean Lave and Ray McDermott. "Estranged Labor/Learning," Outlines 4 (1), 2002: 19-48 is a strong follow-up to a VERY important piece McDermott wrote some years ago w/ Herve Varenne, "Culture as Disability," which a number of us in the Education and CSem programs have found enormously valuable in our teaching.

Anyhow, in "Estranged Learning," which is a Marxist analysis of alienated educational praxis, Lave and McDermott make a number of comments which seem to me to pick up and expand on our exploration of emergent pedagogy last Wednesday. We had begun our discussion w/ the postulation that emergent pedagogues, rather than designing courses to provide complete "coverage" (impossible anyway), might choose content that illustrates/gives students experience in particular WAYS of thinking; that is, choose material that motivates discovery.

Lave and McDermott frame/describe (what I think is) the same idea somewhat differently, suggesting that we replace "objects" of learning (which can include skills) w/ "activities," which cannot then be "traded" or "distributed" as "objects":

categories of learning...by current practice, are treated instituitionally as objects--a stockpile of objects, really: attention, memory, problem solving, higher order skills, and so on--and not as activities well tuned to the relations among people and their world. So we say, over and against the mainstream, that learning is dependent, situated, contextual and emergent. But it is only the first half of a critique of learning theory as currently insitutionalized...and its market place as estranged, alienated, and mystified. Perhaps the most mystifying and in the end the most alienated and alienating assumption is specifically a matter of distribution--a widely and deeply felt distinction that separates the production of official knowledges (e.g. science, literature, national curricular frameworks)...from their distribution through school practice.... "The production of knowledge stocks" is carefully distinguished from what boils down to their apparently non-generative, unchanging distribution as they are "transmitted" through schooling, "learned," and "transferred" beyond. (pp. 29, 34)

My translation of their Marxism into our emergence? That such pedagogy is neither goal-or-product oriented, but action-driven. And cannot be "traded."

Back now to this piece/stand I'm taking against commodification.

Which brings me to the second question: once we have recast the descriptions in these articles in terms of emergence, where does that get us? I firmly believe that the Pabst Blue Ribbon article and the Lisa article are about emergent phenomena, but once I have recognized this, I don't know what to do with it. What deeper understanding or new insights do I get by recasting these phenomena as emergent (other than the act of recasting itself)? I really am at a loss here. Help!!!

The second point is that recognizing the emergent character of human phenomena gives one not only humility but also a set of tools that can be used more effectively to facilitate change (if that is what one is inclined to do). One needs to look for trigger and leverage points rather than attacking things head on, one needs to be patient, and one needs to be willing/able to manipulate instead of to assert/evangelize. One also needs to be alert for the Budweiser effect.

I agree with Paul and Mark about what some of the issues are, though I may disagree with Paul about the degree to which we have to recognize the highly limited nature of personal agency--in any event, it's a key issue for us to mess around with.

http://www.swarthmore.edu/SocSci/tburke1/agency.html

http://emergent.brynmawr.edu/eprg/?page=EmergentSchedule

From there you can also jump to the Serendip Forums, and everything else.

-Doug

At a minimum, this story illustrates the limits of education and the naive liberal view that people can be easily reformed. It also illustrates the meaning to me of the word "accident" and the possibilities of engineering outcomes in emergent systems, a subject that came up at this morning's breakfast.

People use the word "accident" for any bad outcome where the agent's actions were not directed toward creating that outcome (spilling milk while pouring, etc.). I'm not saying that this is a bad usage of the word, but to me there is a more interesting class of accidents that I first discovered when reading "The World According to Garp". If you haven't read the book, I won't gross you out with a description of the accident, but unlike the spilling milk example, this accident is an emergent phenomenon (though when I read the book 20 or so years ago, I had no idea what emergence was). Basically, the accident occurs because of a confluence of events, each slightly risky in themselves, but each of which in isolation would never have resulted in the accident.

If we go back to the Mississippi worker, his actions do not qualify as an accident because he intended the outcome, but it could be considered an accident from the firm's viewpoint (a bizarre kind of industrial accident caused by the ethics course). There undoubtably was a confluence of many factors that caused the worker to do what he did, so his action was in a sense emergent. Still, the "system" can be engineered by gun control. If the worker hadn't had access to guns, even though we still would not have been able to predict his reaction to the business ethics course, we could have limited the mayhem that resulted. "Surprise" is just another word for lack of mental computing capacity or lack of full understanding, but even in the absence of full understanding, we can engineer desired outcomes.

Without getting on my high horse about gun control, it is interesting how when faced with events like Columbine or this Mississippi disaster, Americans seek to turn the events into a simple single cause and effect non-emergent model. They search for a single explanation, like bad parenting (in Columbine) or racism (in Mississippi). These explanations are almost never correct (the racism explanation has already been discredited), but people gravitate to them to make the event understandable and predictable. It also gives people some sense of control, and safety, because if you can predict the event, you may be able to stop it short of gun control.

The evolution of biological complexity by natural selection -- as I understand evolved complexity's claim to emergence, it goes like this: complexity and aptness result from hundreds of thousands of selection events acting on organisms which do not actively (consciously?) influence the direction of evolution. There's no plan for what gets selected or for what variants exist to be selected from, and so all this complexity we see emerged from simple, unintelligent events. That's not enough for it to be called emergence. When a rough stone wheel is worn smooth and efficient by use, is the smoothness an emergent phenomenon? I say no. Here are my concerns:

• For one thing, selection is directed, just not by an intelligent driver. Selection benefits variants that have greater reproductive success. The fact that there are many ways to have greater fitness relative to your competitors -- or that what confers higher fitness changes with environment and with what traits a lineage has already picked up -- doesn't change the fact that at any given time, a demon (that is, "a being whose faculties are so sharpened" that he can map genotype faithfully to phenotype and map phenotype and environment to absolute fitness) could predict which variants die and which live to pass traits on to the next generation.
• But maybe you think that this isn't directedness -- it's just a deterministic rule, just like a CA transition rule. Well, I agree that each generation's selection by highest relative fitness is just a rule, just as the preferential breakage of the stone wheel's protruding corners is a rule. But built into the selection rule is reproductive success, so the rule includes global directionality. Just as we wouldn't count Conway's Life as emergent if the transition rules included built-in preferences for gliders or computation, we shouldn't count evolution because the outcome is in the rule.
• Or maybe you're not interested in how populations tend toward higher fitness -- you're interested in amazing patterns, structures, strategies. You rightly point out that selection has nothing built in that would allow us to predict zebra stripes or venus flytraps. Maybe you even concede that my earlier point about directedness casts doubt on the emergent nature of genetic algorithms, but not on that of jellyfish. Yeah, okay, but I've never been convinced that it isn't just about what appeals to us as amazing. I mean, why do we call it emergent instead of just noise? Or maybe edited noise? If Gould is right and evolution's tape replayed would yield different results, then it seems to me that biological amazingness is actually more about randomness than about emergence.
• Two literary quotes that have had me thinking about the notion of edited noise and about how it's hard for us -- ourselves examples of biological amazingness -- to judge what's amazing. First from Annie Dillard's Pilgrim at Tinker Creek: "We don't know what's going on here. If these tremendous events are random combinations of matter run amok, the yield of millions of monkeys at millions of typewriters, then what is it in us, hammered out of those same typewriters, that they ignite?" Second from Italo Calvino's Mr. Palomar: "Mr. Palomar manages to shift the world from in front of him and set it on the sill, looking out. Now, beyond the wondow, what do we have? The world is also there, and for the occasion has been split into a looking world and a world looked at. And what about him, also known as "I," namely Mr. Palomar? Is he not a piece of the world that is looking at another piece of the world? Or else, given that there is world that side of the window and world this side, perhaps the "I," the ego, is simply the window through which the world looks at the world."

I propose the following tightening of what counts as emergence.

• Emergent phenomena are observed in the global behavior of a population of identical actors (or agents or entities).
• These actors are distributed across some (possibly abstract) space, at the same time. (The number of actors might change in time, but the emergent phenomenon has to be observed in the entire population -- not just in the last of a temporal series of actors.)
• The actors interact with each other by rules which are not directly dependent on any variable which must be measured globally.
• All actors have the same rules, and the rules are constant in time.

Okay, lemme have it.

#s 1 & 4: "identical" or "same" by what criterion? I'm willing to grant that the cells in a CA are identical by almost (but not quite) any criterion (since they vary, at least in spatial position, relative to the rest of any finite grid); but ants surely aren't identical. Some are more aggressive than others, some move faster, some slower, some are larger, some smaller...how do we decide which variations can be overlooked in their effects on the operation of the rule system...even if the rule system is identical, which is arguable? And by this criterion all human social psychological phenomena would, it seems to me, be ruled out as emergent since it is probably impossible to identify any psychological characteristic of human beings on which they are identical, i.e., we all functions according to varying rule systems- varying over people at the same time; and, for cultural-historical-developmental reasons, varying within individuals and societies over time.

#2: seems reasonable

#3: if birds (like boids) regulate their movement in flocking behavior by calculating the average velocity of the flock (which they may not of course), family members regulate their interpersonal behavior by taking the emotional temperature of the family group as a whole at any point in time, and people their stock buying behavior by whether or not the market (or the economy) as a whole is going up or down, then this would seem to rule these out as emergent phenomena by this criterion.

I would be prepared to accept the idea that some emergent phenomena can be described as Ted has described them; but I would be very hesitant to restrict the definition of emergence this way in general. Doing so would, I think, define away much of what it most interesting about emergence.

Lucy (kerman@pobox.upenn.edu) has been following our conversations and sent me an email which seemed to me of general interest and so post here with her permission:

Ted suggested a Frequently Asked Question (FAQ) list not long ago, and I think that this is a great idea. Not necessarily so that it can provide answers, but, by creating one, it may help us see the issues. Which might help us to actually write an article (the so-called "White Paper") explaining why we are interested in emergence, and why we think it may be helpful for other disciplines to look at.

I've started a FAQ over at http://emergent.brynmawr.edu/wiki/index.cgi/EmergenceFAQ. Here, I am trying to reconcile all of the many aspects of emergence that we have wrestled with over the past year, and put into a concise definition. Here's a first stab:

• randomness
• simple, interacting agents
• distributed control
• limited resources
• surprising results

I must admit that my instinct is to completely avoid the ideas of "surprising" or "amazing"... I just have no idea how one would define those. But let's run with this for a moment. Adding that as an aspect to the very definition confronts the observer issue head-on. I've come to believe that about intelligence, so why not emergence too?

This perspective allows some of us to see emergence everywhere, while some of us see it rarely. But is this too wishy-washy to be useful as a scientific concept?

-Doug

I'd like to respond to Rob's response to my first pass at a narrow definition of emergence by suggesting that we all talk in more detail about levels of organization. I'll talk about specifics first and say some more general things along the way and at the end:

• Defining identical and same might not be all that difficult, since I think much of the confusingness comes from the multilayered nature of rules and of actors. As Rob suggests, individual ants behave differently in some ways and identically in other ways. I claim that we have to start with what colony behavior we're calling emergent, and choose the relevant behaviors based on that. Different behaviors are relevant to different phenomena, and the difference often comes down to level.

  Let's say we're interested in the ability of ants to calculate the areas of patches. A bunch of ants behaves differently in small patches than it does in big patches. In the models, it comes down to individual ants traversing the patch and adjusting their behavior based on how many pheromone trails they cross. This one is easy: this model works when each ant is identical in every way.

  Now let's look a another classic emergent colony behavior: the optimal allocation of individual ants to different tasks. The way the thinking here goes, individual ants are foragers, housekeepers, patrollers, etc. Each job occupies more or less the "right" fraction of the total worker population, even as colony size grows or as some researcher kills off a disproportionate number of ants of one or another job. How is this believed to work? In the models, it emerges from individual workers' decisions of what jobs to take on, and these decisions are made on the basis of frequency of contact with other workers of the same or different jobs. Now, it's true that the ants are different in what jobs they have, but the ants are also the same in how they choose their jobs. In this case the relevant thing is their identical job-choice behavior, since the differences among their job-specific behaviors is at the level of the phenomenon itself.

  No one brought this up, but it's been on my mind: computation by Conway's Life. You could imagine someone putting together a bunch of gliders and glider guns and whatnot in Conway's Life to create a sort of computer. Being a computer, it might accept different programs, or rules. These rules would be different at different times, and they might even change. So is it emergent? If the phenomenon we're talking about is computation itself, then the relevant actors are the individual cells following the regular Conway's Life rules. So yes, and the fact that the computer changes or has different rules is irrelevant.

  (My example here was originally going to be the evolution of plant branching rules. (We could also talk about the evolution of cognition.) The branching rules themselves change with time, but their evolution is a matter not of those ules but of the rule of natural selection. The selection rule is constant in time, and the fact of the specific branching rules' changing is irrelevant. But then I remembered that I've been arguing that evolution by natural selection isn't emergent.)

  As for all human social psychological phenomena being ruled out, I think that's tricky. We are all different, it's true, but I think any claim to emergence would have to posit some behavior we all share which we could claim is identical among all people contributing to the phenomenon in question. Real ants and real people are complicated, but emergence is a kind of explanation and explanations are supposed to be simpler than the things they explain.

• In Boids, individual birds adjust their behavior in response to the states of their neighbors, not in response to the global formation. So I'd still call that emergent. Same with stock market behavior. Individuals may look at global stock prices, but only because the global is locally relevant: it's how much money they have to shell out to buy a share. What individual investors don't do is buy or sell with an eye toward affecting any of the variables whose behaviors we'd call emergent.

  As for family members regulating their behaviors by taking the group's emotional temperature into account, I'd suggest excluding that from emergent phenomena. Unless everyone's trying to do it and you get into a game-theory situation in which the final outcome can't be predicted by individuals -- in that case, I'd suggest that individuals in the group have access only to local information, and so maybe it'd be emergent.

• The actors (and their traits, rules, etc.) must be defined (or identified) only after the phenomenon has been chosen.
• Maybe emergence is only something that definitely happens in models, and is only suggestive when talking about real life. I mean, real ants may not be as identical as they are in our models, but real ant-colony behavior may not be purely emergent, either.
• Some agents may have access to global information, but they only care about the global information because of its local effects. They certainly do not choose their behaviors in order to affect the global variable. That's why emergenauts talk about individual investors rather than about pension-fund managers.

Another way of putting my claim that actors can respond to the global if it's locally relevant is the way I put it here:

• Agents can't react to variable that must be measured globally.

I believe that the "perception of levels" is an important aspect to emergence (and have added it to the developing wiki definition). But, I think that this is one attribute that belongs to the observer.

How could one test to see if something is coming from (or being produced by) a "level"? In information theory, a bit is a bit. The bit might represent global information (number of ants in the colony, for example) or something more local to an actor, or a combination. But I don't think it will be useful to try to determine where the meaning of the bit resides (global or local).

-Doug

I'll be sorry to miss Ted's presentation, tomorrow morning, about levels of organization--I'd like to ask some questions (since not then, now!)about the levels of perception/perception of levels Doug mentions above.

I'm writing, this week, from down on the farm, where the notion of limited resources is also much w/ me (much more than when I'm on campus, free of the material constraints a farmer faces every day, freer to speculate...) Actually, the range of questions running through my head right juxtapose of my present location, an earlier life in WPhilly (where I got very discouraged about limited neighborhood resources)and Lucy's description of the current emergent planning taking place in Penn's West Philadelphia Initiatives.

Her description intersected, for me, w/ Doug's list of conditions for emergence, which include

• "limited resources" and the observation that
• "emergence is a perspective" involving two systems: the observer and that which is being observed..." as well as the
• "surprising results" which are "very real aspects of the paradigm."

I'm very intrigued by Penn's attempt, in collaboration w/ its neighbors, to help restore a neighborhood it once was quite active in destroying...and would nr interested to hear Lucy, w/ her experience working in Resources, talk more (in person, @ one of our sessions; or here, on the forum?) about how she sees her work intersecting w/ the range of factors Doug traces:

• the two systems: the observer and that which is being observed
  (=the folks re-designing the neighborhood and the folks living there?
  what happens when the positions of observer and observed are occupied by the same folks?)
• limited resources (again: from whose perspective? might what seems limited to the planners @ Penn seem relatively unlimited to folks living in West Philly?
• what counts as "surprising results"? (again: from what perspective? mightn't Penn planners find something "suprising" that neighborhood folks have known all along, and/or vice versa?)

Lucy, help me w/ this?
Anne

More from down this-a-way:

I read the article on Jane Jacobs which Lucy, via Paul, sent our way yesterday. Much there of interest to us/me. Certainly Jacobs´ vision of urban areas emerging as "the result of human action but not human design" fits nicely into our rubric of emergence. But several further points/turns of the screw to earmark for further exploration/discussion:

• she coined the term "locality knowledge";
• she thinks inductively, rather than by proceeding explicitly from grand philosophical or ideological principles;
• she is a "friend in the fight against tyranny, both local and global."

It´s this last matter which most captured my attention, and to which I invite us to return. Jacobs dismisses city planners as being "driven by the dictatorial complex: They want to deal with their fellow men in the way an enginneer deals with..materials...in all Utopias, the right to have plans of any significance belonged only to the planner in charge."

This seems REALLY key to me: Jacobs'comment returns us QUITE neatly to the questions of agency which we were discussing with Tim last week; importantly, it invites us to use the concept of emergence as a way of talking about the free will of the agents involved. What they do will not control the outcome. But (in part because they aren't controlling/can't control outcome) they are free to...do what they do. To do what they WILL.

I LIKE this.

• Lindsay discussed the distinctions between social and "cosmetic" integration (the latter involves physical intermixing/only superficial interactions;
• she discussed the pressure new students feel from upperclasswomen to identify w/ and participate in the cultural groups which express their ethnic identity (when they may initially prefer not to);
• (a related suggestion: might the Tri-Co Institute, like the Sustained Dialogue we've been discussing, include students of all ethnic identities, rather than targeted groups??)
• when the simulation runs as 80%-Similar-Wanted condition, homeostatsis is "never" reached;
• Lindsay played w/ the ramification of an "avalanche:" if you add one red turtle to the system, how many additional encounters will be needed for ALL the turtles to reach "happiness"? (this is essential when thinking about "critical mass" necessary for "real integration" to occur);
• she posited the (related) helpfulness of a "kill" function that would dispense w/ turtles that could not find a suitable environment after a certain limited number of steps;
• she observed that a more diverse pool (than just two colors) would interestingly complexify the simulation;
• she highlighted the fact that this is a closed system (which she finds usefully analogous to students sharing limited space on campus).

My favorite bits were two: her explanation, @ the end that "Segregation, whether voluntary or involuntary, is a restriction of opportunity; it hampers the flow of knowledge and experience and thus impedes diversification of interests and occupations."

Secondly, she thanks me for introducing her to emergence: "even though it was introduced as a ploy for us not to classify her class as chaotic, but rather realize that there may be some emergent properties involved."

So much for theorizing an emergent pedagogy!

Despite Ted's gesture, above (which I appreciate!) towards the humanist implications of "emergence," I'm finding myself a little non-plussed by what I learned in yesterday's discussion. I have for a number of years now been engaged in negotiating the Two Cultures Divide, beginning w/ the Conversation on that topic in winter 2001, and arriving, most recently, @ one of several papers on the generative productivity of the same, Theorizing Interdisciplinarity. But the distance between the project of the humanists and that of the scientists loomed suddenly very large to me this week, so large that I'm wondering how the bridge I've been working so hard to construct can hold.

On Tuesday afternoon, I participated in a wonderful discussion of the Graduate Idea Forum. At the suggestion of Roland Stahl, we were looking together @ Stephen Toulmin's Return to Reason, a philosophical exploration of our "intellectual obligation to abandon the Myth of Stability" (214).Toulmin's argument, which I found very compelling, is that "we can know nothing about the world of experience with complete certainty, and that any attempt to prove the superiority of one abstract, universal doctrine over its rivals is a product of human presumptuousness" (196). Although Toulmin's somewhat remiss in his homework, and in placing himself in the tradition, he is a pragmatist, and one of the aims of his book is to put "theorizing on a par with all other practical activities" (172). Toulmin did such a good job of tracing the (end!) of the history of the mistaken search for a "Universal Language which had Meaning and Rationality built in from the start"(70) that I found myself considerably taken aback in David Berger's excellent presentation, during Wednesday morning's meeting of the Emergence Group, of the New Kind of Science being laid out in Stephen Wolfram's attempt to pin down the universe into a fairly limited number of universal computations .

So: The Two Cultures, Reinstated. Redux. Where to go from here???

Well, Al, I still want applicability. See some reflections arising from/in the current Summer Institute on Emergence.

William Safire's On Language column in yesterday's NYT Magazine is on the term tipping point. Of course he brings ip Malcolm Gladwell's book. He also mentions Thomas Schelling's segregation model, best known to us in its NetLogo implementation. Schelling got the idea from U of Chicago political scientist Morton Grodzins, whom Safire quotes:

gliderglasmia??

This week and next, I'm teaching, w/ Doug Blank and Kim Cassidy, the Summer Institute on Exploration and Emergence; this means we're thick in the process of finding out how this stuff "plays" in the world beyond our cozy circle. Speaking of which....

I've also been engaged in a couple of parallel-processing, out-of-forum conversations, via e-mail w/ Lucy Kerman and in the flesh w/ Andrea Friedman. Their comments have kept me chewing on this notion of closed systems: both practically (in terms of how this forum operates) and theoretically (in terms of how emergence works "best").

What I'm learning is that the language of our public on-line conversation can/does sound, to those who haven't participated in the Wednesday morning discussions, as theoretical, tight, self-referential. This isn't something that either Lucy or Andrea particularly objects to; but it bothers me a LOT. I very much want this to be an open forum; "a fundamental part of Serendip's development" involves encouraging continuing conversation among lots of different folks (see Serendip Forums.) If our forum area seems to use a "closed" language, or a shorthand that others can't translate, then it's not serving its most important purpose, however well it may be serving the purpose of archiving our conversations.

This seems to me to be a question not only about the accessibility of conversations on Serendip, but also another example of the old "closed/open system" debate that has been running here for some time, w/ Doug and Ted arguing that limited resources are essential to emergence, and Paul insisting, contrari-wise, that "evolution in fact rarely proceeds under conditions of 'limited resources' and is not generally best thought of as a 'zero sum game.' Instead, much of evolution involves cooperative adaptations which themselves increase the total possible payoff and the total available resources."

I think, to increase payoff and available resources, these forums need to be "fed" with case studies from "outside." I think we need to hear (for instance) more of Andrea's thoughts about (ecological) "sustainability," and the critique it offers to the notions we've been playing with, about conceptualizing systems of unlimited resources. I think we need to hear more of Lucy's musings about what happens when you enter a place of scarce resources and increase funding; how ethical is it to raise expectations?

"Say you have kids who can't read (our issue here all the time). What do you make of that? what kind of goals do they have? what are your goals for them? do you reach kids where they are or find ways to move them somewhere else? what if their expectations (and the expectations of their culture) are 'different'-- do you accept their low expectations or help them see that there are others? do you do that according to your middle class values or do you try to see another value that is 'theirs'?"

As Lucy goes on to say,

"overly theoretical discussions, even those that come from serious initial consideration of case studies...force a pattern for the sake of the theory, rather than responding to what is actually happening....there is inevitably less room for innovation when there is an effort to reify it all and fit everything into clear theoretical categories. "

What I'm fretting/worrying about:

the degree to which "emergence" has become, in our group, a "theory" that is guiding/limiting the explorations it was designed to...provoke in the first place. Some good case-study-observations might help...

unsettle that/open things up....

Is this intended to be a public discussion?
Do we want/need it to be?

Edgily, A.

But emergence is all about theory! We're all interested in different real-life phenomena; what brings us all together is this abstract, mathematical thing. Mark is interested in how inflation, unemployment, etc, result from individual consumer choices. I'm interested in how optimal leaf placement results from individual meristematic growth decisions. What do we have in common is basically a math problem.

Lucy (via Anne) contrasts theory and "what is actually happening." But if the abstract stuff is what is common among all our disciplinary interests, then maybe the theory is what's really happening. Plants growing, interest rates dropping, communities changing -- it's all just shadows on the cave wall. Or, our group is a place to see whether imagining them as shadows is a fruitful way to go.

In general I want to be cautious about applying concepts from biology or physics to social issues, cultural phenomena, personal situations. The reason is that one (the?) reason we do science is to be careful about defining and evaluating mechanisms. To transfer explanations as metaphors too easily can defeat that scientific purpose, as the metaphors carry the authority of science but not always the content. Classic example: social Darwinism. It's a misapplication of a good theory because the mechanisms underlying economic stratification are different from those underlying speciation.

Emergence, though, probably survives exporting just fine in many cases. Emergence as a concept is vague enough -- and its conditions are commonly enough met (we haven't agreed on a definition, but the conditions would likely include things like many similar agents, simple rules, different levels) -- that social and cultural systems probably often have emergent phenomena. But then it's no longer about metaphors, but rather it's the real thing. (Or, the math is always metaphor, the way all models are metaphors.)

I like case studies. I think they help us sharpen our theoretical tools. For them to be useful, though, we have to recognize some cases as not about emergence. And we have keep our intentions selfish -- we use cases to test and better articulate our ideas, not to provide insights into real-world issues. Two reasons: (1) we don't know enough yet about emergence, and (2) assuming that emergence is too generally useful tempts us into seeing emergent mechanisms when they may not be there.

You know the saying: When you're holding a hammer, every problem looks like a nail.

Did everyone see this? It's from today's NYT, and it's about how the Defense Department was considering creating a futures market with anonymous investors in order to forecast assassinations, coups, and terrorist attacks. It was a DARPA project, and the idea was to harness a kind of emergent analysis of diffuse political, economic, and secret information. The story says that "the Pentagon, in initially defending the program, said such futures trading had proven effective in predicting other events like oil prices, elections and movie ticket sales."

According to descriptions given to Congress, available at the Web site and provided by the two senators, traders who register would have deposited money into an account similar to a stock account and win or lose money based on predicting events.

"For instance," Mr. Wyden said, "you may think early on that Prime Minister X is going to be assassinated. So you buy the futures contracts for 5 cents each. As more people begin to think the person's going to be assassinated, the cost of the contract could go up, to 50 cents. "The payoff if he's assassinated is $1 per future. So if it comes to pass, and those who bought at 5 cents make 95 cents. Those who bought at 50 cents make 50 cents." The senators also suggested that terrorists could participate because the traders' identities will be unknown.

Senators, of course, were horrified at the macabre nature of the "commodity" being traded. They also pointed out that the financial benefits of insider information could give incentive to people to carry out attacks.

These online bazaars allow punters to plunk down money, real or imagined, on the potential of films, ideas, or the U.S. military's success in snagging Saddam Hussein. It may sound like nothing more than glorified sports gambling, but many economists believe that such markets can suss out vital, hidden information about future events-much in the same way that a soaring stock on Wall Street can indicate that good things are afoot for the company in question.

There were expressions of willingness by a number of people to take salary reducations themselves (or forgo other things they have) in order to avoid layoffs of others. The idea of people sacrificing in order to avoid/prevent immediate suffering of others is simple and emotionally satisfying. It also fits with a larger notion of the College as fundamentally an "emergent" system, in which the larger patterns result (or should result) from the collection of individual actions. In fact, lots of people felt about the layoffs that "how it was done" was the most serious problem: that more people should have been involved in the discussions of what to do rather than having those decisions made in a way that highlighted the top-down control aspects of a hierarchical system.

Is the well-being of a community actually best assured by having all individuals try to minimize what they know of local suffering of other individuals? My guess is no, that that one readily imagine circumstances in which that particular reliance on emergence will result in longer term problems for both the community and most of its members. More generally, is the well-being of a community actually best assured simply by local interactions among individuals? Here too my guess is no; that some simple models would verify the existence of circumstances where some kind of hierarchy improves outcomes both individually and collectively.

So, the theoretical questions, motivated by and relevant for a particular case: under what circumstances does one want/need, not want/need to rely on particular forms of interactive emergence? on emergence itself? What are the circumstances in which some "top down" organization/action is needed? And, of course, the practical questions which in turn can help the theoretical understanding: was THIS a case where it was necessary to go beyond emergence? And, whether yes or no, what are the next actions most likely to sustain a healthy community?

Emergence group ready to take on that kind of challenge?

Thank you, Paul: for exactly the kind of case study I was asking for, above (though of course now I'm wishing it hadn't been provided so readily to hand/close to home). Because I was "teaching about emergence" this morning, I missed the campus meeting about staff layoffs. But I'm wondering, from what I've heard, if a better outcome might not have emerged if more folks had been involved in earlier decision-making re: cost implications of enhanced sabbaticals, pay scales, etc. in a time of economic downturn? I don't see the choice as being one between "simple local interactions among individuals" and the need for "some kind of hierarchy," but rather as indicating the need for what we've been calling, in the Institute,"emergent pedagogy": one that involves "editing," evoking individual inputs/thoughts/reactions, then having senior administrators act on that information. More use might have been made of local insights/interactions much earlier in this process. Calls for them now seem...

belated.

Anyone who thinks markets are good at predicting the future should read John Maynard Keynes' Chapter 12 in the The Geneeral Theory. Well, I am off on my unplugged vacation.

Home | Calendar | About | Getting Involved | Groups | Initiatives | Bryn Mawr Home | Serendip Home

Director: Liz McCormack - emccorma@brynmawr.edu | Faculty Steering Committee | Secretary: Lisa Kolonay
© 1994- , by Center for Science in Society, Bryn Mawr College and Serendip
Last Modified: Wednesday, 02-May-2018 11:57:03 CDT