What is information?
A short introduction to Information Theory
“What we mean by information—the elementary unit of information—is a difference which makes a difference …” — Gregory Bateson
We are fully immersed in the ‘information age’, we are practically drowning in data; yet ask the average executive to define what information actually is and you will probably receive a blank stare. Let’s fix that.
In 1948, a juggling unicyclist, eccentric toymaker (1), and notable mathematician by the name of Claude Shannon published a seminal paper — A Mathematical Theory of Communication (2) — which now forms the basis of Information Theory. It is widely used in complexity research and provides an invaluable framework for understanding adaptive organisations.
What it is
Shannon was trying to determine how to transmit signals in the most efficient and effective means across telephone wires. His model consists of an information source, a transmitter, a channel, a receiver, and a destination. It also includes a potential source of noise that can change the received signal. The diagram from his paper is reproduced below.
This model can be used to describe any type of information transmission: an email, a face-to-face conversation, DNA transmitting instructions to a protein, a market transmitting signals to a business, etcetera.
In addition to the transmission model, Shannon described the content of a message in terms of probabilities. This is often expressed as the ’surprisal’ of the message. It is best explained by example.
Imagine if someone’s vocabulary consisted of only one word: “blah”, then the probability of receiving the word “blah” in their messages is 100%, which is perfectly unsurprising and yields no information (3). Some of us may have experienced this after a friend has spent too much time at a bar.
In Shannon’s formulation, the less probable a message is the more surprising it is and therefore the more information it yields. The key to communication is uncertainty.
Why it matters
Organisations are giant information processors. They extract information from the environment and use that information to modify their behaviour. If a business detects threats in the market, it uses that information to adjust its strategy. Likewise, when it senses opportunities. The better an organisation interprets and processes information, the better it reduces uncertainty and the better it performs.
The quality of information, and how well it moves around the organisation, is enormously important. Adaptive information — information that can help an organisation to adapt to ever-changing conditions — needs to get to those parts of the organisation that can do something with it, with minimal noise or delay.
But organisations are generally terrible at basic communication. Most of the information moving through them is unsurprising and full of noise.
Reflect on the information content of a meeting that you recently attended. What portion of information discussed was predictable (low-information) compared to what was surprising (high-information)?
Also, consider the various sources of noise (see illustration above) active in the meeting:
were knowledgeable people holding back or sugar-coating information because they were afraid to speak up?
were they using technical language that others didn’t understand? (If a message can’t be understood by the receiver then by definition it is encrypted)
were the destination minds open or closed to new information?
was cognitive bias filtering any information?
were people actually listening (receiving) or were they being distracted by their devices, or off mind-wandering somewhere?
That’s just meetings. Consider all the communication issues across other channels: the deluge of low-information emails, the bloated PowerPoints and reports, the unnecessary layers of bureaucracy, the plethora of low-information analytics, all that compulsive multitasking, etcetera.
Employing Shannon’s conceptual framework, executives can set about improving the adaptiveness of the organisations by eliminating unnecessary noise, re-designing communication channels for better flow, and improving the propagation of high-value information.
Learn more
In future transmissions, we will explore many techniques and design patterns that can dramatically improve information flow. Please subscribe here:
This a great little introductory tutorial by Professor Melanie Mitchell based on her book Complexity: A Guided Tour.
This very accessible 25 minute lecture by David Krakauer on The Transformative Power of Information describes the fundamental relationship between energy and information.
This a good article about Claude Shannon and the history of his theory: How Claude Shannon Invented the Future
For deeper immersion into information-theoretic approaches across various disciplines, I highly recommend Walker, S., Davies, P., & Ellis, G. (Eds.). (2017). From Matter to Life: Information and Causality. Cambridge: Cambridge University Press.
Notes and References
1. Shannon also designed flame-throwing trumpets, rocket-powered Frisbees, and plastic foam shoes that he used for ‘walking on water’ — much to the astonishment of lakeside observers. See the Time article Claude Shannon: The Juggling Unicyclist Who Pedaled Us Into the Digital Age for more about this eccentric genius.
2. Shannon, Claude Elwood (1948). "A Mathematical Theory of Communication". Bell System Technical Journal. 27
3. For simplicity, I am ignoring the very important paralinguistic and non-verbal communication modes. We’ll discuss these in detail in future transmissions.