John Kubiatowicz Electrical Engineering and Computer Sciences University of California, Berkeley

1. CS252Graduate Computer ArchitectureLecture 7 ILP 1:Loop-Level parallelism extraction,Data Flow, Explicit Register Renaming. John Kubiatowicz Electrical Engineering and Computer Sciences University of California, Berkeley http://www.eecs.berkeley.edu/~kubitron/cs252 http://www-inst.eecs.berkeley.edu/~cs252

2. Review: Hardware techniques for out-of-order execution • HW exploitation of ILP • Works when can’t know dependence at compile time. • Code for one machine runs well on another • Scoreboard (ala CDC 6600 in 1963) • Centralized control structure • No register renaming, no forwarding • Pipeline stalls for WAR and WAW hazards. • Are these fundamental limitations??? (No) • Reservation stations (ala IBM 360/91 in 1966) • Distributed control structures • Implicit renaming of registers (dispatched pointers) • WAR and WAW hazards eliminated by register renaming • Results broadcast to all reservation stations for RAW CS252-s07, lecture 7

3. FP Mult FP Mult FP Divide FP Add Integer Review: Scoreboard Architecture(CDC 6600) Registers Functional Units SCOREBOARD Memory CS252-s07, lecture 7

4. Review: Four Stages of Scoreboard Control • Issue—decode instructions & check for structural hazards (ID1) • Instructions issued in program order (for hazard checking) • Don’t issue if structural hazard • Don’t issue if instruction is output dependent on any previously issued but uncompleted instruction (no WAW hazards) • Read operands—wait until no data hazards, then read ops (ID2) • All real dependencies (RAW hazards) resolved in this stage, since we wait for instructions to write back data. • No forwarding of data in this model! • Execution—operate on operands (EX) • The functional unit begins execution upon receiving operands. When the result is ready, it notifies the scoreboard that it has completed execution. • Write result—finish execution (WB) • Stall until no WAR hazards with previous instructions: CS252-s07, lecture 7

5. Review: Tomasulo Organization FP Registers From Mem FP Op Queue Load Buffers Load1 Load2 Load3 Load4 Load5 Load6 Store Buffers Add1 Add2 Add3 Mult1 Mult2 Reservation Stations To Mem FP adders FP multipliers Common Data Bus (CDB) CS252-s07, lecture 7

6. Review: Three Stages of Tomasulo Algorithm 1. Issue—get instruction from FP Op Queue If reservation station free (no structural hazard), control issues instr & sends operands (renames registers). 2. Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch Common Data Bus for result 3. Write result—finish execution (WB) Write on Common Data Bus to all awaiting units; mark reservation station available • Normal data bus: data + destination (“go to” bus) • Common data bus: data + source (“come from” bus) • 64 bits of data + 4 bits of Functional Unit source address • Write if matches expected Functional Unit (produces result) • Does the broadcast CS252-s07, lecture 7

7. Review: Comparison Cycle 62 • Why take longer on scoreboard/6600? • Structural Hazards • Lack of forwarding CS252-s07, lecture 7

8. Tomasulo Loop Example Loop: LD F0 0 R1 MULTD F4 F0 F2 SD F4 0 R1 SUBI R1 R1 #8 BNEZ R1 Loop • Assume Multiply takes 4 clocks • Assume first load takes 8 clocks (cache miss), second load takes 1 clock (hit) • To be clear, will show clocks for SUBI, BNEZ • Reality: integer instructions ahead CS252-s07, lecture 7

9. Loop Example CS252-s07, lecture 7

10. Loop Example Cycle 1 CS252-s07, lecture 7

11. Loop Example Cycle 2 CS252-s07, lecture 7

12. Loop Example Cycle 3 • Implicit renaming sets up “DataFlow” graph CS252-s07, lecture 7

13. Loop Example Cycle 4 • Dispatching SUBI Instruction CS252-s07, lecture 7

14. Loop Example Cycle 5 • And, BNEZ instruction CS252-s07, lecture 7

15. Loop Example Cycle 6 • Notice that F0 never sees Load from location 80 CS252-s07, lecture 7

16. Loop Example Cycle 7 • Register file completely detached from computation • First and Second iteration completely overlapped CS252-s07, lecture 7

17. Loop Example Cycle 8 CS252-s07, lecture 7

18. Loop Example Cycle 9 • Load1 completing: who is waiting? • Note: Dispatching SUBI CS252-s07, lecture 7

19. Loop Example Cycle 10 • Load2 completing: who is waiting? • Note: Dispatching BNEZ CS252-s07, lecture 7

20. Loop Example Cycle 11 • Next load in sequence CS252-s07, lecture 7

21. Loop Example Cycle 12 • Why not issue third multiply? CS252-s07, lecture 7

22. Loop Example Cycle 13 CS252-s07, lecture 7

23. Loop Example Cycle 14 • Mult1 completing. Who is waiting? CS252-s07, lecture 7

24. Loop Example Cycle 15 • Mult2 completing. Who is waiting? CS252-s07, lecture 7

25. Loop Example Cycle 16 CS252-s07, lecture 7

26. Loop Example Cycle 17 CS252-s07, lecture 7

27. Loop Example Cycle 18 CS252-s07, lecture 7

28. Loop Example Cycle 19 CS252-s07, lecture 7

29. Loop Example Cycle 20 CS252-s07, lecture 7

30. Why can Tomasulo overlap iterations of loops? • Register renaming • Multiple iterations use different physical destinations for registers (dynamic loop unrolling). • Reservation stations • Permit instruction issue to advance past integer control flow operations • Other idea: Tomasulo building dynamic “DataFlow” graph from instructions. CS252-s07, lecture 7

31. B A + * / + X(0) Y X Data-Flow Architectures • Basic Idea: Hardware respresents direct encoding of compiler dataflow graphs: • Data flows along arcs in“Tokens”. • When two tokens arrive atcompute box, box “fires” andproduces new token. • Split operations produce copiesof tokens Input: a,b y:= (a+b)/x x:= (a*(a+b))+b output: y,x CS252-s07, lecture 7

32. Operation Unit 0 Instruction Cell Operation Unit m-1 Instruction Instruction Cell 0 Operation Packet Operand 1 Data Packets Instruction Cell 1 Operand 2 Memory Instruction Cell n-1 Paper by Dennis and Misunas “Reservation Station?” CS252-s07, lecture 7

33. Explicit Register Renaming • Make use of a physical register file that is larger than number of registers specified by ISA • Keep a translation table: • ISA register => physical register mapping • When register is written, replace table entry with new register from freelist. • Physical register becomes free when not being used by any instructions in progress. • Pipeline can be exactly like “standard” DLX pipeline • IF, ID, EX, etc…. • Advantages: • Removes all WAR and WAW hazards • Like Tomasulo, good for allowing full out-of-order completion • Allows data to be fetched from a single register file • Makes speculative execution/precise interrupts easier: • All that needs to be “undone” for precise break pointis to undo the table mappings CS252-s07, lecture 7

34. FP Mult FP Mult FP Divide Registers Functional Units FP Add Integer SCOREBOARD Memory Question: Can we use explicit register renaming with scoreboard? Rename Table CS252-s07, lecture 7

35. Scoreboard Example • Initialized Rename Table CS252-s07, lecture 7

36. Renamed Scoreboard 1 • Each instruction allocates free register • Similar to single-assignment compiler transformation CS252-s07, lecture 7

37. Renamed Scoreboard 2 CS252-s07, lecture 7

38. Renamed Scoreboard 3 CS252-s07, lecture 7

39. Renamed Scoreboard 4 CS252-s07, lecture 7

40. Renamed Scoreboard 5 CS252-s07, lecture 7

41. Renamed Scoreboard 6 CS252-s07, lecture 7

42. Renamed Scoreboard 7 CS252-s07, lecture 7

43. Renamed Scoreboard 8 CS252-s07, lecture 7

44. Renamed Scoreboard 9 CS252-s07, lecture 7

45. WAR Hazard gone! Renamed Scoreboard 10 • Notice that P32 not listed in Rename Table • Still live. Must not be reallocated by accident CS252-s07, lecture 7

46. Renamed Scoreboard 11 CS252-s07, lecture 7

47. Renamed Scoreboard 12 CS252-s07, lecture 7

48. Renamed Scoreboard 13 CS252-s07, lecture 7

49. Renamed Scoreboard 14 CS252-s07, lecture 7

50. Renamed Scoreboard 15 CS252-s07, lecture 7