Class Discussion prepared by Bumyong Choi

1. Locality Phase PredictionXipeng Shen, Yutao Zhong, Chen DingComputer Science Department, University of Rochester Class Discussion prepared byBumyong Choi University of California San Diego

2. Memory Adaptation • Programs exhibit dynamic locality • Several studies have been done, but require manual analysis to find program phases • Locality-based phase prediction can solve the problem University of California San Diego

3. Previous Analysis • Interval Based • Unclear what the best interval length is • Code-based • The program structure may not reveal its locality pattern • In-lined function, intertwined functions calls University of California San Diego

4. The new technique • Locality analysis • No fixed-size windows • Phase marking • All instructions in the program binary University of California San Diego

5. Locality Phase • A period of a program execution that has stable or slow changing data locality. • We are interested in phases that are repeatedly executed with similar locality for optimization purpose. • Phase Prediction: knowing a phase and its locality whenever the execution enters the phase. University of California San Diego

6. Examples of Recurring Locality Phases • The aging of airplane model • Structural/mechanical/molecular • Other scientific and commercial simulations • GREAT DEMAND FOR COMPUTING RESOURCES!! • Exhibit dynamic but stable phases • Good candidates for adaptation, if we can predict locality phases University of California San Diego

7. Program Phase Source: Phase Tracking and Prediction (Sherwood et al) University of California San Diego

8. Downside University of California San Diego

9. Motivation for the use of locality analysis • Recent studies found that reuse-distance histograms change in predictable patterns in many programs • Reuse distance reveals patterns in program locality University of California San Diego

10. Reuse Distance • The number of distinctive data elements accessed between two consecutive uses of the same element University of California San Diego

11. Reuse Distance Example a b c a a c b rd=2 University of California San Diego

12. Reuse Distance Example a b c a a c b rd=0 University of California San Diego

13. Reuse Distance Example a b ca ac b rd=1 University of California San Diego

14. Reuse Distance Example a bca acb rd=2 University of California San Diego

15. Reuse Distance Example a b c a a c b rd=0 University of California San Diego

16. The reuse-distance trace of Tomcatv University of California San Diego

17. What the example confirms.. • Major shifts in program locality are marked by radical changes • Locality phases have different length • The size changes greatly with program inputs • A phase is a unit of repeating behavior rather than a unit of uniform behavior University of California San Diego

18. New Locality Prediction Method • Analyzes the data locality in profiling runs • Variable-distance sampling • Wavelet filtering • Optimal Phase Partitioning • Analyzes the instruction trace and identifies the phase boundaries in the code • Uses grammar compression to identify phase hierarchies and then inserts program markers through binary rewriting. University of California San Diego

19. Off-line Analysis Variable-distance sampling Optimal Phase Partitioning Filtering(Wavelet) University of California San Diego

20. Variable-distance sampling A small number of representative data Only long-distance reuses Uses dynamic feedback to find suitable thresholds University of California San Diego

21. Wavelet Filtering • Used as a filter to expose abrupt changes in the reuse pattern – removes temporal redundancy • Common Technique in signal an image processing • Shows the change of frequency over time. Further Reading on Wavelet: I.Daubechies. Ten Lectures on Wavelets. Capital City Press, Montpelier, Vermont, 1992 University of California San Diego

22. Wavelet Filtering • The wavelet filtering removes reuses of the same data within a phase University of California San Diego

23. Optimal Phase Partitioning • Removes the spatial redundancy. • Conditions for a good phase partition • A phase should include accesses to as many data samples as possible. • A phase should not include multiple accesses of the same data sample. University of California San Diego

24. Optimal Phase Partitioning • Filtered trace -> a directed acyclic graph • Each edge has a weight . . • More details : in the paper. University of California San Diego

25. New Prediction Method • Analyzes the data locality in profiling runs • Variable-distance sampling • Wavelet filtering • Optimal Phase Partitioning • Analyzes the instruction trace and identifies the phase boundaries in the code • Uses grammar compression to identify phase hierarchies and then inserts program markers through binary rewriting. University of California San Diego

26. Phase Marker Selection • This step finds the basic blocks in the code that uniquely mark detected phases. • Examines all instruction blocks • Possible that the high level program structure may be lost due to compiler optimizations University of California San Diego

27. Phase Marker Selection • Phase detection finds the number of phases but cannot locate the precise time of phase transitions. • Hundreds of memory access vs a few memory references in basic block • What about gradual transition? University of California San Diego

28. Phase Marker Selection • Solution? • Using the frequency of the phases instead of the time of their transition • Marker Block: a basic block that is always executed at the beginning of phase based on the frequency found • If blank region (removed blocks) is larger than threshold, it is considered as a phase execution. University of California San Diego

29. New Prediction Method • Analyzes the data locality in profiling runs • Variable-distance sampling • Wavelet filtering • Optimal Phase Partitioning • Analyzes the instruction trace and identifies the phase boundaries in the code • Uses grammar compression to identify phase hierarchies and then inserts program markers through binary rewriting. University of California San Diego

30. Hierarchical Construction • SEQUITUR • Compresses a string of symbols into a Context Free Grammar • By constructing the phase hierarchy, we find phases of the largest granularity. University of California San Diego

31. Phase Marker Insertion • ATOM- binary rewriting tool • The basic phases (the leaves of the phase hierarchy) have unique markers in the program, so their prediction is trivial. • Based on the phase hierarchy, we make prediction. • Finite automaton to recognize the current phase in the phase hierarchy. University of California San Diego

32. Evaluation • Measure the granularity and accuracy of phase prediction • Cache resizing • Memory remapping • Test the result against manual phase marking University of California San Diego

33. Phase Prediction University of California San Diego

34. Phase Prediction University of California San Diego

35. Adaptive Cache-resizing University of California San Diego

36. Memory-remapping • Assume: the support of Impluse controller • Key requirement: identify when remapping is profitable University of California San Diego

37. Manual vs Phase University of California San Diego

38. Conclusions • General method for predicting hierarchical memory phases in programs with input-dependent but consistent phase-behavior • Predicts the length and locality with near perfect accuracy • It reduces cache size by 40% without increasing the number of cache misses • It improves program performance by 35% when used for memory remappings University of California San Diego

39. Conclusion (cont.) • Locality phase detection should benefit • modern adaptation techniques for increasing performance • reducing energy • other improvements University of California San Diego

40. Questions? University of California San Diego