Just-In-Time Compiler

1. Just-In-Time Compiler 김 성 무 MASSLAB

2. Contents • Java Virtual Machine • Runtime data areas • Java instruction set • pros & cons • Just-In-Time Compiler(JITC) • Architecture System Overview • Multi-level recompilation • Cost/Benefit model • Profiling heuristics • performance • Conclusion

3. Java Virtual Machine • An abstract computing machine • Own instruction set (Java ISA) • Memory manipulation at runtime class loader The Java API’s class file user class files bytecodes execution engine JVM native method invocation Host OS

4. Runtime Data Areas • PC register • each JVM thread has its own program counter • the index in bytecode of the instruction • JVM stack • each JVM thread has its own stacks • invoke method -> new frame is allocated on stack • store operand stack, local variables, state variables • Operand Stack • used to pass arguments • receive return results • interpreter is responsible for transferring return result to the caller operand stack

5. Runtime Data Areas • Local variable • number of local variables in stack is determined at compile time • local variables are numbered • Heap • one heap per one JVM • created on VM start-up • all class instance and arrays are allocated in heap • Constant pool • analogous to ‘symbol table’ • a collection of all symbolic data need by a class • each constant pool is private to its class

6. Java Instruction Set • opcode is just one byte (256 combination possible) • additional index bytes • additional data bytes opcode opcode index1 index2 opcode data

7. Java Instruction Set • Data movement instr. • Stack manipulation instructions • ex) bipush, pop, dup, Iconst_1, swap • Type conversion • convert one type of item on the stack to another • ex) i2f, i2d, i2l, f2i, f2l… • Functional instr. • operands are always taken from the stack and results are placed on the stack • ex) iadd, iand • Control flow instr. • conditional branches ex) ifeq, if_icmpeq • unconditional branches ex) jsr, jsr_w

8. Pros & cons of JVM • Pros • Write once, run everywhere! • Cons • Bytecode interpret at runtime • Low performance • Solution: JITC & AOTC

9. JITC vs AOTC • Just-In-Time Compiler (JITC) • Compile bytecode to machine code at runtime • Compile time are included in execute time • make the maximum use of program execution information • require some CPU and Memory resources • ex) Jikes RVM, DSVM • Ahead-Of-Time Compiler (AOTC) • Compile bytecode to machine code before program execution • Able to do heavy optimizations • Can’t know program execution information • ex) LaTTe

10. Jikes RVM – an example of JITC • developed at IBM T.J. Watson Research Center • targeting server applications • written in Java • employs 2 strategies • baseline compiler • optimizing compiler (O0, O1, O2) • A flexible online adaptive compilation infrastructure.

11. Jikes RVM

12. Architecture System Overview • 3 components of Adaptive Optimization System (AOS) • Runtime measurements subsystem • Controller • Recompilation subsystem

13. Runtime Measurements Subsystem • Subsystem which produce raw profiling data • Monitor the performance of individual methods from a hardware performance monitor • Organizers periodically process and analyze the raw data • the processes data is used by the Controller

14. Controller • Orchestrates and conducts the other components of AOS system • build an compilation plan using profiling data to improve their performance • Determines whether it is profitable to recompile the method • To estimate cost of recompilation, linear model is used • this model is calibrated offline • use Cost/Benefit model to make this calculation • analytic model representing the cost and benefits of these tasks

15. Recompilation Subsystem • Recompilation occur • in separate threads from the application • 3 components of compilation plan • Profiling data • Optimization plan • Instrumentation plan

16. Multi-Level Recompilation • Level 0 • optimizations performed on-the-fly during the translation from bytecodes to the HIR • Register renaming for local variable, dead-code elimination • Level 1 • addition local optimizations • CSE, array bound check elimination, redundant load elimination • flow-insensitive optimizations • copy and constant propagation, dead-assignment elimination • Scalar replacement of aggregates and short arrays • Level 2 • SSA-based flow sensitive optimization • SSA-PRE, SSA-CSE

17. Cost/Benefit model • Assumptions • Sample data determines how long a method has executed • Method will execute as much in the future as it has in the past • Compilation cost and speedup are offline average

18. Cost/Benefit model • Definitions • cur, current optimization level for method m • T(j), expected future execution time at optimization level j • C(j), compilation cost at optimization level j • Choose j > cur that minimizes T(j) + C(j) • If T(j) + C(j) < T(cur) • recompile at optimization level j.

19. Estimated Cost/Benefit

20. Profiling heuristics • How to find candidates for optimization • counters • call stack sampling

21. Counters • Insert method-specific counter on method entry and loop back edge. • Optimize method that surpasses threshold. • Very popular approach : Self, Hotspot, DSVM • Issues • Overhead for incrementing counter can be significant • simple, but hard to tune

22. Call stack sampling • Periodically record which method(s) are on the top of call stack. • Approximates amount of time spent in each method • Use Cost/Benefit model • ex) Jikes RVM, JRocket • Issues • low-overhead, but non-deterministic.

23. Performances

24. Conclusion • Java Virtual Machine has good features, but its performance is low. • We can overcome low performance of JVM with Just-In-Time Compiler.