Introduction to Time Memory Tradeoffs

1. Introduction toTime Memory Tradeoffs Jin Hong SNU

2. Today, we hope to learn ... • Birthday paradox • Hellman tradeoff on blockciphers • Babbage and Golic birthday paradox based tradeoff on streamciphers • Biryukov-Shamir tradeoff on streamciphers • Recent developments 2006 SNU-KMS Winter Workshop on Cryptography

3. Birthday Paradox

4. Birthday paradox– layman’s version • If you have 23 people in one room, it’s a good idea to bet on finding two of them having the same birthday than not. 2006 SNU-KMS Winter Workshop on Cryptography

5. Birthday paradox - most cryptographers’ version • Consider a box containing N numbered balls. If you take out N½ balls, one at a time, with replacements, then there’s a large chance of seeing the same ball twice. 2006 SNU-KMS Winter Workshop on Cryptography

6. Birthday paradox - a more general version • Consider a set of size N, and two subsets of size A and B. If AB=N, there is a large chance that the two subsets intersect non-trivially. 2006 SNU-KMS Winter Workshop on Cryptography

7. Hellman

8. Hellman tradeoff • Martin E. Hellman, A cryptanalytic time-memory trade-off. IEEE Trans. on Infor. Theory, 26 (1980). • A chosen-plaintext attack on blockcipher DES 2006 SNU-KMS Winter Workshop on Cryptography

9. n-bit plaintext block cipher k-bit key n-bit ciphertext Blockcipher • Blockcipher is a parametrized family of permutations. • Each k-bit key specifies a permutation on the set of n-bit strings. • Without knowledge of key, it is not possible to obtain plaintext from ciphertext. 2006 SNU-KMS Winter Workshop on Cryptography

10. share through secure channel key key block cipher block cipher block cipher block cipher block cipher block cipher block cipher block cipher ciphertext ciphertext plaintext ciphertext plaintext ciphertext plaintext plaintext plaintext plaintext plaintext ciphertext ciphertext plaintext ciphertext ciphertext transmit over insecure channel Using a blockcipher • The communicating parties share a common key through some other secure channel. • The long plaintext to be sent is broken into small blocks. • Each block is encrypted though the blockcipher using the common key. • Generated short ciphertext blocks are transmitted over insecure channel. • Receiving party decrypts each ciphertext block using the common key to recover each plaintext block. • The plaintext blocks are concatenated to bring back the whole plaintext. 2006 SNU-KMS Winter Workshop on Cryptography

11. n-bit plaintext block cipher k-bit key n-bit ciphertext Attacking a blockcipher • The number of possible keys is much smaller than the number of possible permutations on the space of plaintext blocks. • The keys size is usually comparable to plaintext size and the number of permutations being used in any blockcipher is comparable to the number of ciphertext blocks. • Hence, in principle, a small number of plaintext-ciphertext pair determines the key uniquely. • But, blockciphers are (or should be) designed so that it is computationally infeasible to find key from plaintext-ciphertext pairs. • If an adversary is successful in obtaining the key from a few plaintext-ciphertext pairs, it may be used to decrypt all other ciphertext blocks encrypted under the same key. 2006 SNU-KMS Winter Workshop on Cryptography

12. fixed plaintext DES key ciphertext Chosen-plaintext attack on DES • DES: 56-bit key, 64-bit block • Attacker is given the ciphertext corresponding to a plaintext of his choice. • Objective of the attacker is to find key from the given ciphertext. • Note that the expected ratio of random mapping image points is (1-1/e)~0.632. 2006 SNU-KMS Winter Workshop on Cryptography

13. Two extreme attacks • Exhaustive search • Try all keys until correct one is found. • This takes quite a long time. • Table lookup • Pre-compute all (key, ciphertext) pairs. • Sort the list according to the ciphertexts. • Read off answer from the dictionary, as soon as ciphertext is given. • This requires quite a large amount of storage. 2006 SNU-KMS Winter Workshop on Cryptography

14. Tradeoff • We could come somewhere in the middle of the two extreme solutions through a tradeoff between online time and storage space. • Offline phase • Pre-compute all (key,ciphertext) pairs, and • store a digest of the computation in a table smaller than the complete dictionary. • Online phase • Given a target, using the incomplete table, find answer in time shorter than require for exhaustive search. 2006 SNU-KMS Winter Workshop on Cryptography

15. Notation • Denote DES encryption by C = EK(P) • Define reduction functionR: (Z/2Z)64 (Z/2Z)56to be any fixed “choosing” of 56 bits from 64 bits. • Fix plaintext P0 and definef: (Z/2Z)56 (Z/2Z)56 by f(K) = R◦EK(P0). • Attacker’s objective translates to that of finding K, given f(K)=R(C). 2006 SNU-KMS Winter Workshop on Cryptography

16. . . . . . . . . . . . . . . . . . . . . . . . . . . . . sp2 spm sp1 sp3 ep3 ep1 ep2 epm ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ f f f f f f f f f f f f f f f f f f f f ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ . . . . . . . . . . . . . . t Hellman table 2006 SNU-KMS Winter Workshop on Cryptography

17. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hellman tradeoff • HT = {(spi,epi)}i, sorted according to the second component. • For j=0…t-1, successively check if the correct key belongs to the (t-j)th column by applying f to R(C) j-many times, and checking for existence of the result among the epi’s. • If key belongs to column t-j, it can be recovered from spi by applying f to it appropriately many times. 2006 SNU-KMS Winter Workshop on Cryptography

18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . sp2 spm sp1 sp3 ep3 ep1 ep2 epm ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ f f f f f f f f f f f f f f f f f f f f ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ . . . . . . . . . . . . . . t Questions? 2006 SNU-KMS Winter Workshop on Cryptography

19. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . False alarm • Due to f being not injective, existence of fj(R(C)) among the epi’s do not guarantee that the correct key belongs to the (t-j)th column. • These false alarms cost t applications of f and its frequency is hard to analyze. 2006 SNU-KMS Winter Workshop on Cryptography

20. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Success probability • Let N=256 be the number of all keys. • Birthday paradox gives the matrix stopping rule: t2m = N. • Success probability= (# of distinct keys in HT)/N~ 0.8 tm/N (when t2m = N) • Success probability of t tables, that use different reduction functions= 1-(1-tm/N)t ~ 1-exp(-t2m/N) = 1-1/e 2006 SNU-KMS Winter Workshop on Cryptography

21. Hellman tradeoff curve • Pre-computation time: P=t2m=N • Online time: T=t2 (applications of f) • Storage: M = tm (sp-ep pairs) • Tradeoff curve: TM2=N2 • Conversely, given T and M satisfying TM2=N2, setting t = T½ and m = M/t results in a tradeoff algorithm requiring time T and storage M. • If cost is measured as T+M, the optimal tradeoff point is T=M=N2/3. • What we have discussed so far does not depend on structure of DES. It is applicable to any one-way function. 2006 SNU-KMS Winter Workshop on Cryptography

22. Inversion Problem Given a one-way function f: XY and a target point y∈Y, find any x∈X such that f(x)=y. Exhaustive Search Try out each x∈X until we see an x with f(x)=y. Table Lookup Pre-compute and store all (x,f(x)) pairs in a table (dictionary), sorted according to the second component. Read off answer when target point y∈Y is given. Inversion problem 2006 SNU-KMS Winter Workshop on Cryptography

23. Time-memory tradeoff Tradeoff • Offline phase • Pre-compute all (x,f(x)) pairs, and • store a digest of the computation in a table smaller than the complete dictionary. • Online phase • Given a target, using the incomplete table, find answer in time shorter than require for exhaustive search. 2006 SNU-KMS Winter Workshop on Cryptography

24. Hellman tradeoff summary • If the keyspace is of size N (DES: 256), for any set of values P, T, and M, satisfyingone may find the key in • online time T using • offline pre-computation time P and • storage of size M for table. • Hellman’s algorithm may be used on arbitrary one-way functions. TM2 = N2, P = N T = M = N2/3 2006 SNU-KMS Winter Workshop on Cryptography

25. Tweaksto Hellman’s Methods

26. Distinguished points • Rivest, before 1982 (according to a book by Denning) • Distinguished point example: a binary string starting with 10 zeros. • To create each row of the Hellman table, function f is iterated until a pre-defined distinguished point is reached. • The length of rows is variable. • This removes much of the table lookup time during the online phase. 2006 SNU-KMS Winter Workshop on Cryptography

27. f1 f1 f1 f1 f2 f2 f2 f2 f3 f3 f3 f3 ft-1 ft-1 ft-1 ft-1 ft ft ft ft . . . . . . . . . . . . . . . . . . . . . . . . . . . . sp1 ep1 ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ sp2 ep2 sp3 ep3 . . . . . . . . . . . . . . spm epm Rainbow tables • PhilippeOechslin, Making a Faster Cryptanalytic Time-Memory Trade-Off. Crypto 2003. 2006 SNU-KMS Winter Workshop on Cryptography

28. Rainbow tables • In a way, t Hellman tables corresponds to one rainbow table. • Compared to the original Hellman method, rainbow tables use half the online time for the same storage. • Using 1.4GB of data (two CD-ROMs) rainbow table method cracks 99.9% of all alphanumerical MS-Windows password hashes in 13.6 seconds. 2006 SNU-KMS Winter Workshop on Cryptography

29. . . . . . . . . . . . . . . . . . . . . . . . . . . . . sp1 spm sp2 sp3 epm ep2 ep1 ep3 ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ f f f f f f f f f f f f f f f f f f f f ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ . . . . . . . . . . . . . . t Checkpoints • G. Avoine, P. Junod, P. Oechslin, Time-memory trade-offs: False alarms detection using checkpoints. Indocrypt 2005. 2006 SNU-KMS Winter Workshop on Cryptography

30. Other neat tricks • Starting points need not be random. For the original Hellman method, they could be small counters concatenated with table numbers. This results in storage savings. (This is an argument against the usefulness of rainbow tables.) • After sorting, the endpoints that are close together have common significant bits. This also leads to storage savings. 2006 SNU-KMS Winter Workshop on Cryptography

31. Digression

32. Are tradeoffs meaningful? • Tradeoff algorithms require exhaustive search. How can such a thing be a meaningful attack? • In constrained environments, systems of marginal security are used. With tradeoff attacks, security level is meaningfully reduced. • Low (short-term) security may be all one wanted. With tradeoff attacks, the security of these systems may turn out to what was expected. • Your neighbor may be incapable of exhaustive search, but a network of hackers may have gotten together and published the needed table. Your adversary may have had such help from a third party. • As soon as exhaustive search is possible by someone, one cannot be sure of the security level provided by the affected system. 2006 SNU-KMS Winter Workshop on Cryptography

33. Affordable tradeoffs • (www.rainbowcrack-online.com) • They have huge tables that implement Oechslin’s tradeoff algorithm and will recover passwords on a subscription basis. • Password hashing schemes based on MD5, LanManage, SHA1, Cisco PIX, NTLM, MySQL-323, MySQL-SHA1, and MD4 are served and they also sell these tables. • LanManager case details: 2006 SNU-KMS Winter Workshop on Cryptography

34. Babbage, Golic

35. Babbage Golic tradeoff • S. H. Babbage, Improved exhaustive search attacks on stream ciphers. European Convention on Security and Detection, 1995. • J. Dj. Golić, Cryptanalysis of alleged A5 stream cipher. Eurocrypt’97. • Attack on streamciphers. 2006 SNU-KMS Winter Workshop on Cryptography

36. Streamcipher is a pseudo-random bit stream generator. The following two steps are repeated. Filter function is applied to internal state to produce a short bit sequence. The internal state is updated. Each initial internal state, i.e., an element of (Z/2Z)s, specifies a long bit sequence (keystream). state update function filter function few bits internal state few bits internal state few bits internal state few bits internal state few bits Streamcipher internal state keystream 2006 SNU-KMS Winter Workshop on Cryptography

37. plaintext long keystream ciphertext ciphertext long keystream plaintext = =   Using a streamcipher share through secure channel internal state internal state • The communicating parties share a common initial internal state through some other secure channel. • A long keystream is generated from the common internal state. • Plaintext is added onto the carrier keystream. • Generated ciphertext is transmitted over insecure channel. • Receiving party generates the same keystream from shared initial state. • Plaintext is recovered from ciphertext by “subtracting” the keystream from ciphertext. transmit over insecure channel 2006 SNU-KMS Winter Workshop on Cryptography

38. Anything that allows recovery of whole keystream from a partial keystream segment is a successful attack. An appropriate length of keystream segment determines the starting internal state uniquely. But, streamciphers are designed so that it is computationally infeasible to recover the starting internal state from a finite keystream segment. Attacking a streamcipher internal state keystream keystream segment 2006 SNU-KMS Winter Workshop on Cryptography

39. The crucial discovery • Given a long keystream, it suffices to find the internal state corresponding to any one of the keystream segments. • Once state is recovered, the cipher may be run forward to obatina future keystream. internal state keystream segment internal state keystream segment 2006 SNU-KMS Winter Workshop on Cryptography

40. Two extreme solutions • Exhaustive search • Try all possible internal states until a known keystream segment is produced. • With N possible states and D keystream segments, N/D tries are expected until an answer is found. • Table lookup • Pre-compute enough (state, keystream seg) pairs. • Sort the list according to the keystream segments. • When D keystream segments are given, look for them in the table and read off answer. • N/D pairs should be pre-computed and stored. 2006 SNU-KMS Winter Workshop on Cryptography

41. Babbage Golic tradeoff • If the number of possible states is N, and the online target data set will be of size D, for any set of values P, T, M, and D, satisfyingone may find the key in • online time T using • offline pre-computation time P, • storage of size M for table, and • online data of size D. • This birthday paradox based method does not depend on the structure of streamciphers, and hence may be used to invert arbitrary one-way functions. TM = N, P = M ≥ N/D T = M = D = N1/2 2006 SNU-KMS Winter Workshop on Cryptography

42. Multi-target Inversion Given a one-way function f: XY and a target set S⊂Y, find at least one x∈X such that f(x)∈S. Attack restatement in terms of one-way functions • Let there be N possible internal states. • Define function one-way function byf: internal state  (ln N) bits of keystream. • Attacker’s objective translates to that of finding any one of the internal states, corresponding to any one of the keystream segments. 2006 SNU-KMS Winter Workshop on Cryptography

43. Biryukov, Shamir

44. Hellman review • Go back to pages 24 and 16. 2006 SNU-KMS Winter Workshop on Cryptography

45. Birthday + Hellman • There’s no reason we can’t apply Hellman table method to the streamcipher situation. • This time, we have the advantage of not having to cover the whole search space. • During the offline phase, it suffices to deal with only N/D internal states. 2006 SNU-KMS Winter Workshop on Cryptography

46. (single target) Offline coverageP = N t tables Online timeT = t•t = t2 StorageM = m•t = mt Tradeoff curveTM2 = N2 (multiple targets) Offline coverageP = N/D t/D tables Online timeT = t•(t/D)•D = t2 StorageM = m•(t/D) = mt/D Tradeoff curveTM2D2 = N2 Birthday + Hellman 2006 SNU-KMS Winter Workshop on Cryptography

47. A. Biryukov and A. Shamir, Cryptanalytic time/memory/data tradeoffs for stream ciphers. Asiacrypt 2000. Combination of Hellman tradeoff and birthday paradox based tradeoff. keystream internal state keystream segment state update internal state keystream segment BS-tradeoff 2006 SNU-KMS Winter Workshop on Cryptography

48. BS-tradeoff • If the state size is N, and the online target data set will be of size D, for any set of values P, T, M, and D, satisfyingone may find the key in • online time T using • offline pre-computation time P, • storage of size M for table, and • online data of size D. • Biryukov-Shamir’s tradeoff algorithm does not depend on the structure of streamciphers, and hence may be used to invert arbitrary one-way functions. TM2D2 = N2, P = N/D, D2 ≤ T T = M = N1/2, D = N1/4 2006 SNU-KMS Winter Workshop on Cryptography

49. TMD-tradeoff theory summary • Even though not made explicit in the original works, the tradeoff algorithms can be applied to arbitrary one-way functions. • Assume a one-way function to be inverted acting on a search space of size N. • For situations where single target inversion problem is applicable, there is a tradeoff algorithm of online complexity N2/3. • For situations where multiple target inversion problem is applicable, there is a tradeoff algorithm of online complexity N1/2. 2006 SNU-KMS Winter Workshop on Cryptography

50. Tradeoff on StreamciphersRevisited