Information

Claude Shannon, an engineer at Bell Laboratories, proposed to measure a surprise value in communication and named it "information entropy" - a measure of disorder. (More surprise - more disorder.) That quickly degenerated into "information" followed by a complete reversal of the meaning (and the sign in the equation). Decades of confusion followed this magical transformation of a measure of disorder into a measure of information/order (like a frog turning into a beautiful princess when kissed by Norbert Wiener and Leon Brillouin). By now nobody really knew what information really is and concepts of complexity and logical depth were put forward in an attempt to resolve the issue.

The difference could be explained in terms of misplaced or broken glass. While a misplaced glass may surprise us, just like a broken glass does, we can put the misplaced glass back to its place. However, the broken glass has only one destination - a rubbish bin. Like all other rubbish - it has lost its meaning.

There is a fundamental difference between information entropy and information that questions some of our assumptions/expectations about “world and ourselves within it.” Information entropy is a noise (with no meaning) that erodes patterns, upheld by redundancy, we receive as information that has a meaning to us. Regardless of what kind of turmoil information may cause in our “description of the world and ourselves within it” (surprise) it always has a meaning.

The distinction has been lost by a theoretical assumption that all of the noise has a potential meaning. And indeed - big strides have been made in physics and other sciences by discovering regular patterns in an apparent noise. But, what we were always looking for, and what we will continue to look for, are regular patterns to which a meaning could be attached. (Etruscan inscriptions, for example, are surely regular patterns to which we still did not manage to attach meanings.) Only when we attach a meaning to a perceived pattern - it becomes information. And this is a far cry from (information) entropy...

Recently, a friend of mine was very surprised to learn that there was a hardware shop in the shopping centre she frequented for years. This could be considered as simple addition of an “item” to what to expect in that shopping centre. However, when she later found herself at home, pondering about shopping trips, she needed to make, a number of things started to “click together” in a different way – like why she did not see neighbours in another shop or how inconvenient it was. A number of oddities (exceptions, contradictions) were gradually removed significantly simplifying “her description of the world and herself within it.” And this is where, I will suggest, we should look at what information really is and how to estimate its value.

Each time an “aha” connects two dots (see Model) one of the active chains is simplified. A single inserted “cultural sketch” could therefore have a potential to simplify a great number of cultural chains in time. The value of information is not, however, in insertion - but in the resulting “aha” that simplifies a chain. (An “aha” may happen upon insertion, but it also may happen years later.) An insertion of an “item” only has a potential value (theoretically infinite) that may, in fact, never result in an “aha”. Only when an “aha” happens is the value of the insertion realised and a cultural chain within our landscape of chains is simplified.

I will also suggest that the breakthrough thinking is a gradual process that lifts a large number of "chains" in the unresolved state (unlearning) until a resolution is found that initiates series of "fallings".

An "aha" that enables describing "more" with "less" - is a realised value of an inserted cultural sketch. If we quantify this "more with less" before and after an "aha" we might be able to produce a figure between 0 and 1 - as in classical "information" theory. However, things look rather complex when we consider that alternation of a single active cultural chain is, as a rule, accompanied with alternations within other active cultural chains. And when we add to this immense complexity of "our description of the world and ourselves within it" - the math also becomes rather complex. Great Russian mathematician - Andrey Nikolaevich Kolmogorov devised a math that is now called Kolmogorov Complexity along similar lines of thought and, I guess, a mathematical synthesis might give us better means in calculating (assessing) a value of information.

The complexity of our “description of the world and ourselves within it” - is hard even to contemplate. However, when we consider our culture and language only (cultural imprint), we might be in a somewhat better position. A culture of an individual and the vocabulary this individual commands could be assessed. This could provide a base (but only a base) in the “reconstruction” of an “individual description of the world and himself within it." (I think that we actually do this as we get to know somebody -- but mostly in a non-conscious way.)

Damir Ibrisimovic's concept of chains, with insertions being new perceptions with emotive content, deletions being forgetting, and the wholesale rearranging of chains of thought being intellectual "aha"'s resulting from past insertions provides an interesting, informal semantics for nonmonotonic logic. It does need some emendations, though, to quite qualify.

Joseph S. Fulda, C.S.E., Ph.D.

Describing “more” with “less” seems to be our eternal struggle. Although the landscape of “our description of the world and ourselves within it” is both vast and very complex - I think that we could start looking into it and start modelling. The suggested variant of nonmonotonic logic does offer a significant simplification in comparison to neural networks and weighted fuzzy logic could be easily emulated using existing hardware. When developed, such a model could be left to “evolve." However, I suggest that it would need constant human supervision, especially in the form of reconciliation with our human, “subjective” experiences. Sufficiently refined and reconciled with our “subjective” experiences, it could then provide a platform for numerous applications as well as a good inkling into workings of our minds.