1. Trace Caches Michele Co CS 451

2. Motivation • High performance superscalar processors • High instruction throughput • Exploit ILP • Wider dispatch and issue paths • Execution units designed for high parallelism • Many functional units • Large issue buffers • Many physical registers • Fetch bandwidth becomes performance bottleneck

3. Fetch Performance Limiters • Cache hit rate • Branch prediction accuracy • Branch throughput • Need to predict more than one branch per cycle • Non-contiguous instruction alignment • Fetch unit latency

4. Problems with Traditional Instruction Cache • Contain instructions in compiled order • Works well for sequential code with little branching, or code with large basic blocks

5. Suggested Solutions • Multiple branch target address prediction • Branch address cache (1993, Yeh, Marr, Patt) • Provides quick access to multiple target addresses • Disadvantages • Complex alignment network, additional latency

6. Suggested Solutions (cont’d) • Collapsing buffer • Multiple accesses to btb (1995, Conte, Mills, Menezes, Patel) • Allows fetching non-adjacent cache lines • Disadvantages • Bank conflicts • Poor scalability for interblock branches • Significant logic added before and after instruction cache • Fill unit • Caches RISC-like instructions derived from CISC instruction stream • (1988, Melvin, Shebanow, Patt)

7. Problems with Prior Approaches • Need to generate pointers for all noncontiguous instruction blocks BEFORE fetching can begin • Extra stages, additional latency • Complex alignment network necessary • Multiple simultaneous access to instruction cache • Multiporting is expensive • Sequencing • Additional stages, additional latency

8. Potential Solution – Trace Cache • Rotenberg, Bennett, Smith (1996) • Advantages • Caches dynamic instruction sequences • Fetches past multiple branches • No additional fetch unit latency • Disadvantages • Redundant instruction storage • Between trace cache and instruction cache • Within trace cache

9. Trace Cache Details • Trace • Sequence of instructions potentially containing branches and their targets • Terminate on branches with indeterminate number of targets • Returns, indirect jumps, traps • Trace identifier • Start address + branch outcomes • Trace cache line • Valid bit • Tag • Branch flags • Branch mask • Trace fall-through address • Trace target address

11. Next Trace Prediction (NTP) • History register • Correlating table • Complex history indexing • Secondary Table • Indexed by most recently committed trace ID • Index generating function

12. NTP Index Generation

13. Return History Stack

14. Trace Cache vs. Existing Techniques

15. Trace Cache Optimizations • Performance • Partial matching [Friendly, Patel, Patt (1997)] • Inactive issue [Friendly, Patel, Patt (1997)] • Trace preconstruction [Jacobson, Smith (2000)] • Power • Sequential access trace cache [Hu, et al., (2002)] • Dynamic direction prediction based trace cache [Hu, et al., (2003)] • Micro-operation cache [Solomon, et al., 2003]

16. Trace Processors • Trace Processor Architecture • Processing elements (PE) • Trace-sized instruction buffer • Multiple dedicated functional units • Local register file • Copy of global register file • Use hierarchy to distribute execution resources • Addresses superscalar processor issues • Complexity • Simplified multiple branch prediction (next trace prediction) • Elimination of local dependence checking (local register file) • Decentralized instruction issue and result bypass logic • Architectural limitations • Reduced bandwidth pressure on global register file (local register files)

17. Trace Processor

18. Trace Cache Variations • Block-based trace cache (BBTC) • Black, Rychlik, Shen (1999) • Less storage capacity needed

19. Trace Table: BBTC Trace Prediction

20. Block Cache

21. Rename Table

22. BBTC Optimization • Completion time multiple branch prediction (Rakvic, et al., 2000) • Improvement over trace table predictions

23. Tree-based Multiple Branch Prediction

24. Tree-PHT

25. Tree-PHT Update

26. Trace Cache Variations (cont’d) • Software trace cache • Ramirez, Larriba-Pey, Navarro, Torrellas (1999) • Profile-directed code reordering to maximize sequentiality • Convert taken branches to not-taken • Move unused basic blocks out of execution path • Inline frequent basic blocks • Map most popular traces to reserved area of i-cache