Introduction to Coding Theory

1. Introduction to Coding Theory Rong-Jaye Chen

2. Outline [1] Introduction [2] Basic assumptions [3] Correcting and detecting error patterns [4] Information rate [5] The effects of error correction and detection [6] Finding the most likely codeword transmitted [7] Some basic algebra [8] Weight and distance [9] Maximum likelihood decoding [10] Reliability of MLD [11] Error-detecting codes [12] Error-correcting codes

3. Introduction to Coding Theory [1] Introduction Coding theory The study of methods for efficient and accurate transfer of information Detecting and correcting transmission errors Information transmission system

4. Introduction to Coding Theory [2] Basic assumptions Definitions Digit:0 or 1(binary digit) Word:a sequence of digits Example:0110101 Binary code:a set of words Example:1. {00,01,10,11} , 2. {0,01,001} Block code :a code having all its words of the same length Example: {00,01,10,11}, 2 is its length Codewords :words belonging to a given code |C| : Size of a code C(#codewords in C)

5. Introduction to Coding Theory Assumptions about channel

6. Introduction to Coding Theory Binary symmetric channel

7. Introduction to Coding Theory [3] Correcting and detecting error patterns

8. Introduction to Coding Theory [4] Information rate Definition: information rate of code C Examples

9. Introduction to Coding Theory [5] The effects of error correction and detection 1. No error detection and correction

10. Introduction to Coding Theory 2. parity-check digit added(Code length becomes 12 ) Any single error can be detected ! (3, 5, 7, ..errors can be detected too !) Pr(at least 2 errors in a word)=1-p12-12 x p11(1-p)?66x10-16 So 66x10-16 x 107/12 ? 5.5 x 10-9 wrong words/sec � �

11. Introduction to Coding Theory

12. Introduction to Coding Theory [6] finding the most likely codeword transmitted Example:

13. Theorem 1.6.3 Suppose we have a BSC with � < p < 1. Let and be codewords and a word, each of length . Suppose that and disagree in positions and and disagree in positions. Then Introduction to Coding Theory

14. Introduction to Coding Theory Example

15. Introduction to Coding Theory [7] Some basic algebra

16. Introduction to Coding Theory Kn is a vector space

17. Introduction to Coding Theory [8] Weight and distance Hamming weight: the number of times the digit 1 occurs in Example: Hamming distance: the number of positions in which and disagree Example:

18. Introduction to Coding Theory Some facts:

19. Introduction to Coding Theory CMLD:Complete Maximum Likelihood Decoding If only one word v in C closer to w , decode it to v If several words closest to w, select arbitrarily one of them IMLD:Incomplete Maximum Likelihood Decoding If only one word v in C closer to w, decode it to v If several words closest to w, ask for retransmission

20. Introduction to Coding Theory

21. Introduction to Coding Theory [10] Reliability of MLD The probability that if v is sent over a BSC of probability p then IMLD correctly concludes that v was sent

22. Introduction to Coding Theory [11] Error-detecting codes

23. Introduction to Coding Theory the distance of the code C : the smallest of d(v,w) in C Theorem 1.11.14 A code C of distance d will at least detect all non-zero error patterns of weight less than or equal to d-1. Moreover, there is at least one error pattern of weight d which C will not detect. t error-detecting code It detects all error patterns of weight at most t and does not detect at least one error pattern of weight t+1 A code with distance d is a d-1 error-detecting code.

24. Introduction to Coding Theory [12] Error-correcting codes Theorem 1.12.9 A code of distance d will correct all error patterns of weight less than or equal to . Moreover, there is at least one error pattern of weight 1+ which C will not correct. t error-correcting code It corrects all error patterns of weight at most t and does not correct at least one error pattern of weight t+1 A code of distance d is a error-correcting code.

25. Introduction to Coding Theory