National Institute of Information and Communications Technology, JAPAN Song-Ju Kim and Ken Umeno

1. Revisions to the Spectral Test and the Lempel-Ziv Compression Test in the NIST Statistical Test Suite National Institute of Information and Communications Technology, JAPAN Song-Ju Kim andKen Umeno （ChaosWare Inc.）

2. It is well known that the NIST Statistical Test Suite was used in the evaluation of the AES candidate algorithms. • It is also world-widely used by external audiences in the evaluation of their Pseudo Random Number Generators.

3. The NIST Statistical Test Suite “A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications” National Institute of Standards and Technology (2001) http://csrc.nist.gov/rng/

4. OUTLINE • On the NIST Statistical Test Suite • Test Results (AES, SHA-1, and MUGI) • Checking of the Uniformity of P-values • Corrections to the Spectral (DFT) Test • Corrections to the LZC Test • Summary

5. The test procedure • A set of sequences, each of length n, is produced from the selected generator. • Each statistical test evaluates the sequence and returns one or more P-values. • If the P-value ≥ α(=0.01), then we call the sequence “success”. • 1. Checking of the success rate. • 2. Checking of the uniformity of the distribution of P-values.

6. What is p-value? P-value: the probability that a perfect random number generator would have produced a sequence less random than the sequence that are tested.

7. 1. The checking of the success rate • The range of acceptable proportions: ※(μ±3σ)/m : 99.73% range of binomial distribution, where μ= m (1 – α) and σ= m α(1- α). α=0.01: significance level

8. Success Rate (Example) Key 1 Key 4

9. 2. The checking of the uniformity of the P-values distribution The interval [0,1] is divided into 10 sub intervals, and the p-values that lie within each sub-intervals are counted (F i). p-value of p-values: IGMC( 9/2,χ2/2 ) where IGMC(n, x) = and The test passes if p-value of p-values ≥ 0.0001

10. Uniformity of p-values (Example) Key 1 (fail) Key 4 (pass)

11. The parameters we used 6 n=10, α=0.01, 1000 samples 10 keysх1000 samplesх10^6 (sequence length) total 10^10 bit

12. Test Results AES (OFB)

13. Test Results SHA-1

14. Test Results MUGI

15. If we focus on the uniformity of P-values, only the DFT test and LZC test are failed frequently. • If we choose the sample size m greater than 10000, we cannot find any PRNG that pass these two test.

16. P-value of P-values (SHA-1)

17. These distributions of P-values indicates a apparent deviation from randomness although we use a well-known good PRBG (SHA-1) • This observation suggests that the test settings in these two tests are not accurate.

18. The DFT testtest description (NIST document) • The zeros and ones of the input sequence are converted to values of -1 and +1. • Apply a DFT on X to produce: S=DFT(X). • Calculate M=modulus(S’), where S’ is the substring consisting of the first n/2 elements in S. • Compute T= : the 95% peak height threshold value. • Compute N0 = 0.95n/2. • Compute N1 = the actual observed number of peaks in M that are less than T. • Compute P-value =

19. The probability distribution (SHA-1) 300,000 samples

21. The LZC test test description (NIST document) • Parse the sequence into consecutive, disjoint and distinct words that will form a “dictionary” of words in the sequence. ex. 0|1|00|01|000|11|011| • Compute P-value =

22. The probability distribution (SHA-1)

23. Despite the best fitting of the distribution, the uniformity of P-values cannot be improved. • This is because the distribution of the number of words is too narrow. In other words, a variety of the appeared P-values is limited.

24. The effect of discreteness

25. Because the variety of appeared P-values is too scarce in centered bins, we never get the uniformity of P-values in this situation. • The histogram of P-values always has some biases even if we use good PRNG. • However, these biases are always the same if we use good PRNG.

26. Checking of Uniformity (LZ)

27. P-value of P-values (before)

28. P-value of P-values (after)

29. Summary • We corrected two points for DFT test. (1) the threshold T (2) the variance of the theoretical distribution • We corrected two points for LZ test. (1) setting of standard distribution (asymmetric) which has no algorithm dependence. (2) re-definition of the uniformity of P-values.