Making Sense of Information II

Beyond Reversibility and Computability and Consistency and Skepticism: Information?

Paul Grobstein
Center for Science Information Working Group
15, 22 July 2004

THREE WEEK'S WORK:
DISCOVERING THE LIMITS OF THE BOX

Physics/physicists are not in fact committed to time irreversibility - 22 June

They have and may continue to find it a useful tool in many situations but acknowledge problems with it in thermodynamics and possibly in quantum mechanics as well.

Hence, investigators of other phenomena ought not to presume that time reversibility is necessarily applicable in, or should be taken as a model for, their own inquiries.

There are important non-computable numbers - 1,8 July "Non-computable" = not "Turing-computable", ie not derivable using formal axiomatic systems as a basis assumption for exploration

"Important" in the sense that such numbers are a consequence of any formal axiomatic system adequate to support arithmetic. Perhaps also important in that such numbers can be generated in other ways and so may play a role in "reality".

Arithmetic is not "complete". Hence, investigators of other phenomena ought not to presume that either arithmetic (and, by inference, formal mathematics generally) or "Turing-computability" is necessarily applicable in, or should be taken as a model for, their own inquiries.

Formal axiomatic systems sufficiently rich to support arithmetic (and anything more sophisticated) cannot be both consistent and complete - 1 July If they are consistent, they will always be capable of producing outputs that follow from the starting conditions but cannot be inferred from the starting conditions using any given set of inference rules.

Hence, investigators of other phenomena ought not to presume that formal axiomatic systems are necessarily applicable, or should be taken as a model for, their own inquiries.

"Reality" may or may not be the playing out of a formal axiomatic system but, regardless, trying to understand it in terms of formal axiomatic systems will inevitably fail in the long run.

If "reality" were a formal axiomatic system, no set of inference rules would suffice to account for all observations even if the starting conditions were known. (Additional conjecture: no finite set of observations of the outputs of a formal axiomatic system can yield its starting conditions, and there is no assurance whatsoever of "convergence" to the starting conditions)

If "reality" is not a formal axiomatic system, then the conclusion follows directly.

Ergo ..... it is time to stop trying to "ground" inquiry/exploration on some foundation of "certainty", ie on any principle or set of principles which are not themselves subject to inquiry and change

Known tools that work are fine when and where they work but it should not be presumed they are the only tools that one will ever need.

The best known tools derive from one or another form of the presumption that there is a solid starting point or set of starting points from which everything else derives/can be understood.

The best known tools clearly generate things they are unable to handle, at least in some cases because of self-referencing.

"Maybe its time to seriously entertain the possibility that looking for a single solid starting point just isn't the right way to go, that one has to find another, different way to proceed." ... Letter to René

Computability beyond Turing-computability?

Logics/mathematics beyond formal axiomatic systems?

POSSIBLE WAYS OUT OF THE BOX:
IRREVERSIBILITY, INCONSISTENCY, INDETERMINACY, SELF-REFERENTIALITY (RECURSION) AND THE BRAIN

One Trick: Look at your starting points and see what you can change ...

"A foolish consistency is the hobgoblin ..."
(Ralph Waldo Emerson, Self-Reliance, 1841) Could one relax the demand for consistency? Is it useful? Would it yield greater completeness?
"To be, or not to be: that is the question"
Shakespeare, Hamlet
Another Trick: Take a fresh look at what evolution has created as an effective inquirer into/explorer of "reality" ....

Irreversibility already established (eg lateral inhibition network) (reservation: local vs global irreversibility)

Inconsistency already established, is in fact a "neural uncertainty principle", comparable to the uncertainty inherent in quantum mechanics? and with same implications re reversibility? advantages re "computing"?

Indeterminacy .... likely (VERY likely)

Improved "completeness"?
Roger Penrose, The Emperor's New Mind, 1990
Four color problem
"provable"? no ... "likely"? yep (product of Chaitin's "experimental math")

Self-referentiality, recursion (more on the brain trick)

Human brain is both iterative and recursive/self-referential (others?)
Iteration (Wolfram) - deterministic but not compressible (Chaitin)
Iteration plus - indeterminate and not compressible
Iteration plus model model-building
Iteration plus etc. and recursion/self-referentiality?

Has both multiple "model building" and compressive/self-reflective ("story telling") capabilities

Has distinctive "agency" capabilites vis a vis outside world: to conceive/bring into existence what would not result from "model building" (iterative change) - "free will"

Has distinctive "agency" capabilities vis a vis itself: to conceive/bring into existence what would not result from "model building"

"I am, and I can think, therefore I can change who I am"

22 July
Click to go back to beginning on 15 July
How to proceed in face of "profound skepticism"?, ie with no fixed starting point that one can treat as unquestionable ...

The universe happily iterated/evolved without formal axiomatic systems (recursion/self-referentiality?) for billions of years
Humans have the capability to do so without formal axiomatic systems ("treeness")
Humans also have the capacity to conceive/make use of AND modify formal axiomatic systems
The combination gives us unusual capabilities IF we make use of both
Whether "reality" a formal axiomatic system or not, create/use/modify formal axiomatic systems, but don't "believe" in them.
Trying it out in practice
Three Doors Problem can be addressed using several different "logics" (unexamined starting presumptions used to derive answers)
"Logics" used can be different in different parts of brain
Which logic to use is an experimental question (which "works"?); there is no absolute or foundational logic to appeal to
Noticing conflicts essential (Three Doors a way to notice one), may yield new candidate "logics", new "understanding", ie a "logic" that connects things previously treated as separate (not "description" or just "compression" in local realm but greater connectedness)
Introduction to ....

BAYESIAN INFERENCE:
WORKING WITHOUT A FOUNDATION?

Interpret probabilities not as frequencies/distributions but as "degree of uncertainty", may/may not be based on prior observations (can be "subjective" to varying degrees)
No true (1.0) nor false (0.0) but only numbers (continuously(?) varying) in between
Bayes theorem for altering degree of uncertainty based on new observations

The new degree of uncertainty about an hypothesis when a new observation has been made ("posterior probability") is the old degree of uncertainty ("prior probability") multiplied by a "likelihood factor". The "likelihood factor" is the degree of uncertainty of the observation given the hypothesis divided by the degree of uncertainty of the observation by itself.
A simple case:
hypothesis = the sun comes up every day
prior probability high (uncertainty low)
new observation = the sun came up today
likelihood factor marginally greater than 1
posterior probability slightly increased (uncertainty margically reduced)
hypothesis = the sun comes up every day
prior probability high (uncertainty low)
new observation = the sun didn't come up today
likelihood factor less than 1
posterior probability decreased (uncertainty increased)
Application to Three Doors
- actually MATTERS ( in practical, real life terms)
- is "surprising" to "story teller"/consciousness, not? to unconscious/"treeness" (see Piatelli-Palmarini, Inevitable Illusions, 1994)
- can become a part of "story telling"
Other examples; problems of false positives and the like
Point is not whether Bayes' Theorem is "true", point is it works in these situations and others, is a new "logic"/understanding, useful for "singular" cases. Furthermore, it provides a basis for inquiry which relies solely on present hypotheses and degrees of uncertainty about them. There need in principle be no other foundation, no presumption that there is a "reality" much less that it is one which itself derives from a small number of fixed principles
The Monty Hall Problem - Bayesian calculation
Monty Hall Solution - a variety of them
Cause, Chance and Bayesian Statistics
Bayesian Inference for the Physical Sciences
Introduction to Bayesian Statistics
International Society for Bayesian Analysis
Bayesian Networks
Bayesian Inference

AN UPDATED THEORY OF INFORMATION ?
(From the top down, hoping to meet from the bottom up in the middle)

What IS "information"?

At the top ("story teller") level

"I have two children. one of whom is a girl. What sex is my other child?"

contrast with
"I have two children, one of whom is a girl. She is the taller of the two. What sex is my other child?"

von Bayer, Information, Chapter 9

Information, to a human story-teller, is something that changes in a story the degree of uncertainty about something (alters Batesian "probability", critically dependent on ability to appreciate counter-factuals). There is no "information" unless there is such a change (information is not an intrinsic property of anything; it is fundamentally relational)

At the bottom level

Information "may perhaps be defined as that which is is transformed by a decoder and used by a model to make predictions; a "decoder" is any organized matter that associates with some predictability elements of one set of organized matter with elements of another set of organized matter .... (where organized=not randomly distributed)".

"Information" can exist without humans, without counter-factuals. It is any organization of matter/energy that affects, in an at least somewhat non-random way, the organization of other organized matter/energy.

Between the "active inanimate" and "story tellers"

models =? non random distributions of decoders that to some degree "represent" and "predict" their surroundings (without any counter-factuals, representations of alternatives to their surroundings), ie are "adapted" - have "shapes" that fit other "shapes" (antibodies, DNA) - can be self-referential

model makers =? as above except with added capability to become further modified by their own activity as decoders

humans (other model makers?) =? as above except with added capability to represent/use counterfactuals (Batesian inference can exist in model makers but doesn't there represent to the entity "uncertainty"). Implies (working from bottom up) that "information" is something that "fits", hence that in human thought there is something analogous to "shape", something that defines "spaces". Can be origin of "exploration"?

To continue:

Where/how does "indeterminacy" play into this series of related stories?
Is "information" usefully quantifiable in this series of related stories?
Are there, in this series of related stories, about whether and how information is created/destroyed?
Can the series of related stories be further "squeezed" to yield useful "laws of information"?

Beyond Reversibility and Computability and Consistency and Skepticism: Information?

Paul GrobsteinCenter for Science Information Working Group15, 22 July 2004

THREE WEEK'S WORK:DISCOVERING THE LIMITS OF THE BOX

POSSIBLE WAYS OUT OF THE BOX: IRREVERSIBILITY, INCONSISTENCY, INDETERMINACY, SELF-REFERENTIALITY (RECURSION) AND THE BRAIN

"To be, or not to be: that is the question"Shakespeare, Hamlet

BAYESIAN INFERENCE: WORKING WITHOUT A FOUNDATION?

AN UPDATED THEORY OF INFORMATION ?(From the top down, hoping to meet from the bottom up in the middle)

Paul Grobstein
Center for Science Information Working Group
15, 22 July 2004

THREE WEEK'S WORK:
DISCOVERING THE LIMITS OF THE BOX

POSSIBLE WAYS OUT OF THE BOX:
IRREVERSIBILITY, INCONSISTENCY, INDETERMINACY, SELF-REFERENTIALITY (RECURSION) AND THE BRAIN

"To be, or not to be: that is the question"
Shakespeare, Hamlet

BAYESIAN INFERENCE:
WORKING WITHOUT A FOUNDATION?

AN UPDATED THEORY OF INFORMATION ?
(From the top down, hoping to meet from the bottom up in the middle)