Lots of Bits

1. Lots of Bits Representations, Exponential Growth, Moore’s Law

2. http://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpghttp://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpg

3. How many bit strings of length n? Adding one bit doubles the number

4. Text • 8 bits per character • “A” = 01000001 • “(” = 00101000 • How many combinations of 8 bits? 2· 2· 2· 2· 2· 2· 2· 2 = 28 = 256

5. Hexadecimal Digits

6. ASCIIAmerican Standard Code for Information InterchangeCharacter represented by Hex xy, e.g. 4B is “K”

7. ASCII Underneath • Emails • Web pages

8. http://farm4.static.flickr.com/3021/2494096946_2bf86f8571.jpg?v=0http://farm4.static.flickr.com/3021/2494096946_2bf86f8571.jpg?v=0

9. What if you need more than 256 characters? • Unicode • 32 bits per character

14. = 32 2 4, 294, 967, 296 How many Unicode characters?32 bits each, so there are in all †

16. Binary counting1+1=10, or 1+1=0 and carry 1

17. Positive and Negative Numbers • Signed and unsigned numbers • Unsigned: 28=256 bit patterns represent 0 … 255 • Signed: 28 bit patterns represent -128 … +127 • Leftmost bit = sign bit: 0 => 0 or pos, 1 => neg • Largest 8-bit positive number = 01111111 = 127 • 0 = 00000000 • Most negative negative number = 10000000 = -128

18. Negative numbers -1 = 11111111so addition works the same for positive and negative numbers

19. Biggest Numbers • Biggest positive number = 01111111 (like 999999 on a car odometer) • Most negative negative number = 10000000

20. Biggest Positive Number + 1 “=”Most Negative Negative Number OVERFLOW!

21. The Comair Christmas “Glitch” • 16 bits for monthly count of crew changes • Biggest positive 16-bit number =32,767 • December was a bad month, lots of snowstorms, lots of flights rescheduled • As Christmas approached the count went from 32,767 to -32,768 by adding 1

22. The Y2010 “Glitch” • Binary representation of decimal 10 = 00001010 • Binary Coded Decimal = write decimal 10 as sequence of 4-bit binary codes for digits • Decimal 10 = BCD 0001 0000 • What if you write decimal 10 in BCD but some other program reads it as decimal? • Binary 0001 0000 = decimal 16

23. Bytes • 1 byte = 8 bits = 2 hex digits = 1 character • 210 =1024 bytes = 1 kilobyte = 1KB • 220 =1,048,576 bytes = 1 megabyte = 1MB • 230 bytes = 1 gigabyte = 1GB = “a billion” • 240 bytes = 1 terabyte = 1TB = “a trillion” • 250 bytes = 1 petabyte = 1PB = “a quadrillion” • 260 bytes = 1 exabyte = 1EB = a quintillion bytes • 270 = zetta • 280 = yotta

24. K • All this terminology based on the accident that • Which is 1K? • There are new standard names: • 1 kibibyte = 1000 bytes • vs. 1 kilobyte = 1024 bytes • But almost no one uses “kibi-”, “mebi-”, etc. = = 10 3 1024 2 10 1000 ~ †

27. Moore’s Law (1965) • The number of transistors on a silicon chip doubles every 18 [or 12, or 24] months • 1965: 64 = 26 • 2008 = 2 billion ~ 231 • 25 doublings in 43 years = one doubling every 20+ months

28. Example of linear increase

29. Example of exponential increase • Now for the y axis use instead lg(y) = the exponent e such that 2e=y

30. Same plot, using lg(y) instead of y

31. One of the Greatest Engineering Achievements • An increase by a factor of 225 is about 30 millionfold • If human speed had increased that much over the past 43 years, we would now be traveling faster than the speed of light

32. Probabilities • Fair coin: P(heads) = 1/2 • Fair die: P(rolling 3) = 1/6 • Fair card deck: P(hearts) = 1/4 • P(ace) = 1/13

33. Probabilities of Independent Events Multiply • P(heads and then heads) = 1/2 · 1/2 = 1/4 • P(3 and then 4) = 1/6 · 1/6 = 1/36 • P(ace and ace) = 1/13·1/13 = 1/169 ≈ .0059 but only if the first card drawn is replaced and the deck is completely reshuffled, otherwise the events are not independent • P(ace and ace without reshuffling) = 1/13 · 3/51 ≈ .0045

34. Unlikely Events • How likely are 100 heads in a row? • (1/2)100 ≈ 10-32 = .00000000000000000000000000000001

35. How Small is 2-100 ≈ 10-30? • Age of universe ≈ 1018 sec = 1027 nanoseconds (1 nanosecond = 1 ns = 1 billionth of a second = 10-9 sec) • If you do all 100 coin flips in a billionth of a second, you will get the 100-heads event about once every thousand lifetimes of the universe 1030 = 103 ·1027 • This is “never” for all practical purposes

36. Morse’s telegraph1844 1848

37. Morse Code (1838)

38. Morse Code (1838)

39. How Long are Morse Codes on Average? • Not the average of the lengths of the letters: (2+4+4+3+…)/26 = 82/26 ≈ 3.2 • We want the average a to be such that in a typicalreal sequence of say 1,000,000 letters, the number of dots and dashes should be about a·1,000,000 • The weightedaverage: (freq of A)·(length of code for A) + (freq of B)·(length of code for B) + … = .08·2 + .01·4 + .03·4 + .04·3+… ≈ 2.4

40. Data vs. Information • Message sequence: “yea,” “nay,” “yea,” “yea,” “nay,” “nay” … • In ASCII, 3·8 = 24 bits of data per message • But if the only possible answers are “yea” and “nay,” there is only 1 bit of information per message • Entropy is a measure of the information content of a message, as opposed to its size • Entropy of this message sequence = 1 bit/msg

41. Squeezing out the “Air” • Suppose you want to ship pillows in boxes and are charged by the size of the box • Lossless data compression • Entropy = lower limit of compressibility

42. Claude Shannon (1916-2001)A Mathematical Theory of Communication (1948)

43. Communication over a Channel Source Coded Bits Received Bits Decoded Message S X Y T Channel symbols bits bits symbols Encode bits before putting them in the channel Decode bits when they come out of the channel E.g. the transformation from S into X changes “yea” --> 1 “nay” --> 0 Changing Y into T does the reverse For now, assume no noise in the channel, i.e. X=Y