Spiral: an empirical search system for program generation and optimization

1. Spiral: an empirical search system for program generation and optimization David Padua Department of Computer Science University of Illinois at Urbana-Champaign

2. Program optimization today • The optimization phase of a compiler applies a series of transformations to achieve its objectives. • The compiler uses the outcome of program analysis to determine which transformations are correctness-preserving. • Compiler transformation and analysis techniques are reasonably well-understood. • Since many of the compiler optimization problems have “exponential complexity”, heuristics are needed to drive the application of transformations.

3. Optimization drivers • Developing driving heuristics is laborious. • One reason for this is the lack of methodologies and tools to build optimization drivers. • As a result, although there is much in common among compilers, their optimization phases are usually re-implemented from scratch.

4. Optimization drivers (Cont.) • A consequence: Machines and languages not widely popular usually lack good compilers. (some popular systems too) • DSP, network processor, and embedded system programming is often done in assembly language. • Evaluation of new architectural features requiring compiler involvement is not always meaningful. • Languages such as APL, MATLAB, LISP, … suffer from chronic low performance. • New languages difficult to introduce (although compilers are only a part of the problem).

5. A methodology based on the notion of search space • Program transformations often have several possible target versions. • Loop unrolling: How many times • Loop tiling: size of the tile. • Loop interchanging: order of loop headers • Register allocation: which registers are stored in memory to give room for new values. • The process of optimization can be seen as a search in the space of possible program versions.

6. Empirical searchIterative compilation • Perhaps the simplest application of the search space model is empirical search where several versions are generated and executed on the target machine. The fastest version is selected. T. Kisuki, P.M.W. Knijnenburg, M.F.P. O'Boyle, and H.A.G. Wijshoff . Iterative compilation in program optimization. In Proc. CPC2000, pages 35-44, 2000

7. Empirical search and traditional compilers • Searching is not a new approach and compilers have applied it in the past, but using architectural prediction models instead of actual runs: • KAP searched for best loop header order • SGI’s MIPS-pro and IBM PowerPC compilers select the best degree of unrolling.

8. Limitations of empirical search • Empirical search is conceptually simple and portable. • However, • the search space tends to be too large specially when several transformations are combined. • It is not clear how to apply this method when program behavior is a function of the input data set. • Need heuristics/search strategies. • Availability of performance “formulas” could help evaluate transformations across input data sets and facilitate search.

9. Compilers and Library Generators Algorithm Program Generation Internal representation Program Transformation Source Program

10. Empirical search in program/library generators • Examples: • FFTW [M. Frigo, S. Johnson] • Spiral (FFT/signal processing) [J. Moura (CMU), M. Veloso (CMU), J. Johnson (Drexel), …] • ATLAS (linear algebra)(R. Whaley, A. Petitet, J. Dongarra) • PHiPAC[J. Demmel et al]

12. SPIRAL • The approach: • Mathematical formulation of signal processing algorithms • Automatically generate algorithm versions • A generalization of the well-known FFTW • Use compiler technique to translate formulas into implementations • Adapt to the target platform by searching for the optimal version

14. Fast DSP Algorithms As Matrix Factorizations • Computing y = F4 x is carried out as: t1 = A4 x( permutation ) t2 = A3 t1( two F2’s ) t3 = A2 t2( diagonal scaling ) y = A1 t3( two F2’s ) • The cost is reduced because A1, A2, A3 and A4 are structured sparse matrices.

15. Tensor Product Formulation of Cooley-Tuckey Theorem Example is a diagonal matrix is a stride permutation

16. Formulas for Matrix Factorizations R1 R2 where n = n1…nk, ni- = n1…ni-1, ni+= ni+1…nk

17. F8 : R1 F8 : R1 F2 F4 : R1 F4 : R1 F2 F2 F2 F2 F2 F8 : R2 F2 F2 F2 Factorization Trees Different computation order Different data access pattern Different performance

18. Walsh-Hadamard Transform

19. Optimal Factorization Trees • Depend on the platform • Difficult to deduct • Can be found by empirical search • The search space is very large • Different search algorithms • Random, DP, GA, hill-climbing, exhaustive

22. Size of Search Space

25. More Search Choices • Programming: • Loop unrolling • Memory allocation • In-lining • Platform choices: • Compiler optimization options

26. The SPIRAL System DSP Transform Formula Generator SPL Program Search Engine SPL Compiler C/FORTRAN Programs PerformanceEvaluation DSP Library Target machine

27. Spiral • Spiral does the factorization at installationtime and generates one library routine for each size. • FFTW only generates codelets (input size  64) and at run time performs the factorization.

28. A Simple SPL Program Definition Formula Directive Comment ; This is a simple SPL program (define A (matrix(1 2)(2 1))) (define B (diagonal(3 3)) #subname simple (tensor (I 2)(compose A B)) ;; This is an invisiblecomment

29. Templates (template (F n)[n >= 1] ( do i=0,n-1 y(i)=0 do j=0,n-1 y(i)=y(i)+W(n,i*j)*x(j) end end )) Pattern Condition I-code

30. SPL Compiler SPL Formula Template Definition Parsing Template Table Abstract Syntax Tree Intermediate Code Generation I-Code Intermediate Code Restructuring I-Code Optimization I-Code Target Code Generation FORTRAN, C

31. Intermediate Code Restructuring • Loop unrolling • Degree of unrolling can be controlled globally or case by case • Scalar function evaluation • Replace scalar functions with constant value or array access • Type conversion • Type of input data: real or complex • Type of arithmetic: real or complex • Same SPL formula, different C/Fortran programs

33. Optimizations * High-level scheduling * Loop transformation Formula Generator * High-level optimizations - Constant folding - Copy propagation - CSE - Dead code elimination SPL Compiler C/Fortran Compiler * Low-level optimizations - Instruction scheduling - Register allocation

34. Basic Optimizations (FFT, N=25, SPARC, f77 –fast –O5)

35. Basic Optimizations(FFT, N=25, MIPS, f77 –O3)

36. Basic Optimizations(FFT, N=25, PII, g77 –O6 –malign-double)

37. Performance Evaluation • Evaluation the performance of the code generated by the SPL compiler • Platforms: SPARC, MIPS, PII • Search strategy: dynamic programming

38. Pseudo MFlops • Estimation of the # of FP operations: • FFT (radix-2): 5nlog2n – 10 + 16

39. FFT Performance (N=21 to 26) SPARC MIPS PII

40. FFT Performance (N=27 to 220) SPARC MIPS PII

41. Important Questions • What lessons can be learned from this work? • Can this approach be used in other domains ?