Register Renaming through Tomasulo’s Algorithm and Remap Tables

1. Register Renaming through Tomasulo’s Algorithm and Remap Tables April 28, 2005 Prof. Nancy Warter-Perez Note: These lecture notes are modified from Profs. Dave Patterson and John Kubiatowicz.

2. Review: Dynamic hardware techniques for out-of-order execution • HW exploitation of ILP • Works when can’t know dependence at compile time. • Code for one machine runs well on another • Scoreboard (ala CDC 6600 in 1963) • Centralized control structure • No register renaming, no forwarding • Pipeline stalls for WAR and WAW hazards. • Are these fundamental limitations??? (No) • Reservation stations (ala IBM 360/91 in 1966) • Distributed control structures • Implicit renaming of registers (dispatched pointers) • WAR and WAW hazards eliminated by register renaming • Results broadcast to all reservation stations for RAW

3. Tomasulo Organization FP Registers From Mem FP Op Queue Load Buffers Load1 Load2 Load3 Load4 Load5 Load6 Store Buffers Add1 Add2 Add3 Mult1 Mult2 Reservation Stations To Mem FP adders FP multipliers Common Data Bus (CDB)

4. Three Stages of Tomasulo Algorithm 1. Issue—get instruction from FP Op Queue If reservation station free (no structural hazard), control issues instr & sends operands (renames registers). 2. Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch Common Data Bus for result 3. Write result—finish execution (WB) Write on Common Data Bus to all awaiting units; mark reservation station available • Normal data bus: data + destination (“go to” bus) • Common data bus: data + source (“come from” bus) • 64 bits of data + 4 bits of Functional Unit source address • Write if matches expected Functional Unit (produces result) • Does the broadcast

5. Tomasulo Loop Example Loop: LD F0 0 R1 MULTD F4 F0 F2 SD F4 0 R1 SUBI R1 R1 #8 BNEZ R1 Loop • Assume Multiply takes 4 clocks • Assume first load takes 8 clocks (cache miss), second load takes 1 clock (hit) • To be clear, will show clocks for SUBI, BNEZ • Reality: integer instructions ahead

6. Loop Example

7. Loop Example Cycle 1

8. Loop Example Cycle 2

9. Loop Example Cycle 3 • Implicit renaming sets up “DataFlow” graph

10. Loop Example Cycle 4 • Dispatching SUBI Instruction

11. Loop Example Cycle 5 • And, BNEZ instruction

12. Loop Example Cycle 6 • Notice that F0 never sees Load from location 80

13. Loop Example Cycle 7 • Register file completely detached from computation • First and Second iteration completely overlapped

14. Loop Example Cycle 8

15. Loop Example Cycle 9 • Load1 completing: who is waiting? • Note: Dispatching SUBI

16. Loop Example Cycle 10 • Load2 completing: who is waiting? • Note: Dispatching BNEZ

17. Loop Example Cycle 11 • Next load in sequence

18. Loop Example Cycle 12 • Why not issue third multiply?

19. Loop Example Cycle 13

20. Loop Example Cycle 14 • Mult1 completing. Who is waiting?

21. Loop Example Cycle 15 • Mult2 completing. Who is waiting?

22. Loop Example Cycle 16

23. Loop Example Cycle 17

24. Loop Example Cycle 18

25. Loop Example Cycle 19

26. Loop Example Cycle 20

27. Why can Tomasulo overlap iterations of loops? • Register renaming • Multiple iterations use different physical destinations for registers (dynamic loop unrolling). • Reservation stations • Permit instruction issue to advance past integer control flow operations • Other idea: Tomasulo building “DataFlow” graph on the fly.

28. What about Precise Interrupts? • Both Scoreboard and Tomasulo have:In-order issue, out-of-order execution, and out-of-order completion • Need to “fix” the out-of-order completion aspect so that we can find precise breakpoint in instruction stream.

29. Relationship between precise interrupts and specultation: • Speculation is a form of guessing. • Important for branch prediction: • Need to “take our best shot” at predicting branch direction. • If we issue multiple instructions per cycle, lose lots of potential instructions otherwise: • Consider 4 instructions per cycle • If take single cycle to decide on branch, waste from 4 - 7 instruction slots! • If we speculate and are wrong, need to back up and restart execution to point at which we predicted incorrectly: • This is exactly same as precise exceptions! • Technique for both precise interrupts/exceptions and speculation: in-order completion or commit

30. Reorder Buffer FP Op Queue FP Regs Res Stations Res Stations FP Adder FP Adder HW support for precise interrupts • Need HW buffer for results of uncommitted instructions: reorder buffer • 3 fields: instr, destination, value • Reorder buffer can be operand source => more registers like RS • Use reorder buffer number instead of reservation station when execution completes • Supplies operands between execution complete & commit • Once operand commits, result is put into register • Instructionscommit • As a result, its easy to undo speculated instructions on mispredicted branches or on exceptions

31. Four Steps of Speculative Tomasulo Algorithm 1. Issue—get instruction from FP Op Queue If reservation station and reorder buffer slot free, issue instr & send operands &reorder buffer no. for destination (this stage sometimes called “dispatch”) 2. Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute; checks RAW (sometimes called “issue”) 3. Write result—finish execution (WB) Write on Common Data Bus to all awaiting FUs & reorder buffer; mark reservation station available. 4. Commit—update register with reorder result When instr. at head of reorder buffer & result present, update register with result (or store to memory) and remove instr from reorder buffer. Mispredicted branch flushes reorder buffer (sometimes called “graduation”)

32. Reorder Buffer FP Op Queue Compar network Program Counter Exceptions? Dest Reg FP Regs Result Valid Reorder Table Res Stations Res Stations FP Adder FP Adder What are the hardware complexities with reorder buffer? • Need fully-associative comparison with all reorder entries on instruction dispatch to find latest version of register. • Need as many ports as register file

33. 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest F0 LD F0,10(R2) N Registers To Memory Dest from Memory Dest Dest Reservation Stations FP adders FP multipliers

34. F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers

35. F0 ADDD F0,F4,F6 N F4 LD F4,0(R3) N -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 6 ADDD ROB5, R(F6) Dest Reservation Stations 1 10+R2 5 0+R3 FP adders FP multipliers

36. -- ROB5 ST 0(R3),F4 N F0 ADDD F0,F4,F6 N F4 LD F4,0(R3) N -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 6 ADDD ROB5, R(F6) Dest Reservation Stations FP adders FP multipliers

37. -- M[10] ST 0(R3),F4 Y F0 <val2> ADDD F0,F4,F6 Ex F4 M[10] LD F4,0(R3) Y -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers

38. -- M[10] ST 0(R3),F4 Y F0 <val2> ADDD F0,F4,F6 Ex F4 M[10] LD F4,0(R3) Y -- BNE F2,<…> N What about memory hazards??? 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N Oldest F0 LD F0,10(R2) N Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers

39. Memory Disambiguation:Sorting out RAW Hazards in memory • Question: Given a load that follows a store in program order, are the two related? • (Alternatively: is there a RAW hazard between the store and the load)? Eg: st 0(R2),R5 ld R6,0(R3) • Can we go ahead and start the load early? • Store address could be delayed for a long time by some calculation that leads to R2 (divide?). • We might want to issue/begin execution of both operations in same cycle. • Today: Answer is that we are not allowed to start load until we know that address 0(R2)  0(R3) • Next Week: We might guess at whether or not they are dependent (called “dependence speculation”) and use reorder buffer to fixup if we are wrong.

40. Hardware Support for Memory Disambiguation • Need buffer to keep track of all outstanding stores to memory, in program order. • Keep track of address (when becomes available) and value (when becomes available) • FIFO ordering: will retire stores from this buffer in program order • When issuing a load, record current head of store queue (know which stores are ahead of you). • When have address for load, check store queue: • If any store prior to load is waiting for its address, stall load. • If load address matches earlier store address (associative lookup), then we have a memory-induced RAW hazard: • store value available  return value • store value not available  return ROB number of source • Otherwise, send out request to memory • Actual stores commit in order, so no worry about WAR/WAW hazards through memory.

41. Memory Disambiguation: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer F4 R[F5] LD F4, 10(R3) N -- R[F5] ST 10(R3), F5 N F0 LD F0,32(R2) N Oldest -- <val 1> ST 0(R3), F4 Y Registers To Memory Dest from Memory Dest Dest Reservation Stations 2 32+R2 FP adders FP multipliers

42. Explicit Register Renaming • Make use of a physical register file that is larger than number of registers specified by ISA • Keep a translation table: • ISA register => physical register mapping • When register is written, replace table entry with new register from freelist. • Physical register becomes free when not being used by any instructions in progress. • Pipeline can be exactly like “standard” DLX pipeline • IF, ID, EX, etc…. • Advantages: • Removes all WAR and WAW hazards • Like Tomasulo, good for allowing full out-of-order completion • Allows data to be fetched from a single register file • Makes speculative execution/precise interrupts easier: • All that needs to be “undone” for precise break pointis to undo the table mappings

43. P0 P2 P4 P6 P8 P10 P12 P14 P16 P18 P20 P22 P24 P26 P28 P30 Newest Done? Oldest  P32 P34 P36 P38 P60 P62 Explicit register renaming:Hardware equivalent of static, single-assignment (SSA) compiler form • Physical register file larger than ISA register file • On issue, each instruction that modifies a register is allocated new physical register from freelist • Used on: R10000, Alpha 21264, HP PA8000 Current Map Table Freelist

44. P32 P2 P4 P6 P8 P10 P12 P14 P16 P18 P20 P22 P24 P26 P28 P30 Newest Oldest  P34 P36 P38 P40 P60 P62 Explicit register renaming:Hardware equivalent of static, single-assignment (SSA) compiler form • Note that physical register P0 is “dead” (or not “live”) past the point of this load. • When we go to commit the load, we free up Done? Current Map Table Freelist F0 P0 LD P32,10(R2) N

45. P32 P2 P4 P6 P8 P34 P12 P14 P16 P18 P20 P22 P24 P26 P28 P30 Newest Oldest  P36 P38 P40 P42 P60 P62 Explicit register renaming:Hardware equivalent of static, single-assignment (SSA) compiler form Done? Current Map Table F10 P10 ADDD P34,P4,P32 N Freelist F0 P0 LD P32,10(R2) N

46. P32 P32 P36 P36 P4 P4 P6 P6 P8 P8 P34 P34 P12 P12 P14 P14 P16 P16 P18 P18 P20 P20 P22 P22 P24 P24 P26 P26 P28 P28 P30 P30 Newest Done? -- Oldest -- BNE P36,<…> N F2 P2 DIVD P36,P34,P6 N F10 P10 ADDD P34,P4,P32 N  P38 P38 P40 P40 P44 P44 P48 P48 P60 P62 F0 P0 LD P32,10(R2) N Explicit register renaming:Hardware equivalent of static, single-assignment (SSA) compiler form Current Map Table Freelist  Checkpoint at BNE instruction P60 P62

47. P40 P32 P36 P36 P38 P4 P6 P6 P8 P8 P34 P34 P12 P12 P14 P14 P16 P16 P18 P18 P20 P20 P22 P22 P24 P24 P26 P26 P28 P28 P30 P30 Newest Oldest  P42 P38 P40 P44 P44 P48 P50 P48 P0 P10 Explicit register renaming:Hardware equivalent of static, single-assignment (SSA) compiler form Done? Current Map Table -- ST 0(R3),P40 Y F0 P32 ADDD P40,P38,P6 Y F4 P4 LD P38,0(R3) Y -- BNE P36,<…> N F2 P2 DIVD P36,P34,P6 N F10 P10 ADDD P34,P4,P32 y Freelist F0 P0 LD P32,10(R2) y  Checkpoint at BNE instruction P60 P62

48. P32 P32 P36 P36 P4 P4 P6 P6 P8 P8 P34 P34 P12 P12 P14 P14 P16 P16 P18 P18 P20 P20 P22 P22 P24 P24 P26 P26 P28 P28 P30 P30 Newest Oldest  P38 P38 P40 P40 P44 P44 P48 P48 P60 P62 Explicit register renaming:Hardware equivalent of static, single-assignment (SSA) compiler form Done? Current Map Table F2 P2 DIVD P36,P34,P6 N F10 P10 ADDD P34,P4,P32 y Freelist F0 P0 LD P32,10(R2) y Speculation error fixed by restoring map table and freelist  Checkpoint at BNE instruction P60 P62

49. FP Mult FP Mult FP Divide Registers Functional Units FP Add Integer SCOREBOARD Memory Question:Can we use explicit register renaming with scoreboard? Rename Table

50. Scoreboard Example • Initialized Rename Table