Andrey Nikolayevich’s Rule

There is another, even deeper reason for our inclination to narrate, and it is not psychological. It has to do with the effect of order on information storage and retrieval in any system, and it’s worth explaining because of what is considered the central problem of probability and information theory.

The first problem is that information is costly to obtain.

The second problem is that information is also  costly to store–like real state in NYC. The more orderly, less random, patterned, and narratized a series of words and symbols, the easier to store that series in one’s mind or jot it down in a book so your grandchildren can read it someday.

Finally, information is costly to manipulate and retrieve.

With so many brain cells–one hundred billion–the attic is quite large, so the difficulties probably do not arise from storage-capacity limitations, but maybe just indexing problems. Your conscious, or working memory, the one you’re using to read this line and make sense of their meaning, is considerably smaller than the attic. Consider that your memory  has difficulty holding a mere seven digit long phone number.

Consider a collection of words glued together  to constitute a 500-page book. If the words are purely random, picked up from the dictionary in an unpredictable way, you’ll not be able to summarize, transfer, or reduce the dimensions of that book without loosing something significant from it. You need a 100.000 words to carry the exact message of a random 100.000 words with you on your next trip to Siberia.

Now consider the opposite: a book filled with the repetition of the following sentence: “The chairman of [insert here your company name] is a lucky fellow who happened to be in the right place at the right time and claims credit for the company’s success, without making a single allowance for luck”. The entire book can be accurately compressed, as I just did, in 34 words, out of 100.000; you could accurately reproduce with total fidelity out of such a kernel.

By finding the pattern, the logic of the series, you no longer need to memorize it all. You just store the pattern. And, as we can see here, the pattern is obviously more compact than then raw information. You looked to the book and found a rule. It is along these lines that the great probabilist Andrey Nikolayevich Kolmorogov defined the  degree of randomness; it is called “Kolmorogov complexity“.

— Nassim N. Taleb, The Black Swan