The code paradigm

1. A mathematical theory of communication, 1948 The work of Turing’s team in the UK breaking the German code was also an important early contribution to IT. His vast experience with both long distance communications and war cryptography granted Shannona unique insight about the potential of maths to optimize information transmission. Information & uncertainty are reverse opposites. The brilliant idea Characteristics of a coding system to fit the requirements of information theory Uncertainty A practical case All options are equiprobable. The binary code is an example of compact code –every string of digits corresponds to one of the possible options. When all options are equiprobable, the most efficient way to reduce uncertainty is to code signals that will discard each half the options. He worked in Bell Co. for many years. 02 A compact code is one in which any combination of signals means something, it does not ‘waste’ a single signal. Information Measure of uncertainty downsizing. Two golden rules An efficient code is one in which less informative components are either omitted or assigned shorter or simpler signals. Information should become measurable to ensure that most informative signals are the ones selected for transmission. Compressing information The more frequent or likely a thing is, the less informative. 01 The problem increasing popularity of the phone pushes the capacity of the physical medium of transmission to its limit. Because The Morse code is an example of efficient code –shorter signals are assigned to more frequent letters in English texts. A conceptual summaryof information theory Are natural languages efficient? Frequent and predictable combinations are not very informative The communication process Natural languages are quite redundant, full of structures and combinations that are particularly likely to happen, and thus are easy to guess or predict. Human communication never happens in optimal conditions. Noiseblends in with signals, making them difficult to decode. Channel Message Signal Signal Message Source Transmitter Receiver Destination Noise need not be acoustic. It is any kind of irrelevant datathat distort the signal. But redundancy strengthens the signaland helps thus fight the problem of noise. Noise Thus verbal language is realistically adapted to non-optimal communication conditions.