1 / 52

Recap: Summary of Pipelining Basics

CS 152: Computer Architecture and Engineering Lecture 13 Advanced Pipelining Randy H. Katz, Instructor Satrajit Chatterjee, Teaching Assistant George Porter, Teaching Assistant. Recap: Summary of Pipelining Basics. Five Stages: Fetch: Fetch instruction from memory

galya
Télécharger la présentation

Recap: Summary of Pipelining Basics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 152: Computer Architectureand EngineeringLecture 13Advanced PipeliningRandy H. Katz, InstructorSatrajit Chatterjee, Teaching AssistantGeorge Porter, Teaching Assistant

  2. Recap: Summary of Pipelining Basics • Five Stages: • Fetch: Fetch instruction from memory • Decode: get register values and decode control information • Execute: Execute arithmetic operations/calculate addresses • Memory: Do memory ops (load or store) • Writeback: Write results back to registers (I.e. COMMIT) • Pipelines pass control information down the pipe just as data moves down pipe • Forwarding/Stalls handled by local control • Balancing length of instructions makes pipelining much smoother • Increasing length of pipe increases impact of hazards; pipelining helps instruction bandwidth, not latency

  3. Recap: Can Pipelining Get Us into Trouble? • Yes:Pipeline Hazards • Structural hazards: attempt to use the same resource two different ways at the same time • E.g., combined washer/dryer would be a structural hazard or folder busy doing something else (watching TV) • Data hazards: attempt to use item before it is ready • E.g., one sock of pair in dryer and one in washer; can’t fold until get sock from washer through dryer • Instruction depends on result of prior instruction still in the pipeline • Control hazards: attempt to make a decision before condition is evaulated • E.g., washing football uniforms and need to get proper detergent level; need to see after dryer before next load in • Branch instructions • Can always resolve hazards by waiting • Pipeline control must detect the hazard • Take action (or delay action) to resolve hazards

  4. 1st lw Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Pipelining the Load Instruction Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Clock • The five independent functional units in the pipeline datapath are: • Instruction Memory for the Ifetch stage • Register File’s Read ports (bus A and busB) for the Reg/Dec stage • ALU for the Exec stage • Data Memory for the Mem stage • Register File’s Write port (bus W) for the Wr stage 2nd lw 3rd lw

  5. Ifetch Reg/Dec Exec Wr The Four Stages of R-type Cycle 1 Cycle 2 Cycle 3 Cycle 4 • Ifetch: Instruction Fetch • Fetch the instruction from the Instruction Memory • Reg/Dec: Registers Fetch and Instruction Decode • Exec: • ALU operates on the two register operands • Update PC • Wr: Write the ALU output back to the register file R-type

  6. Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Wr Pipelining the R-type and Load Instruction Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Clock • We have pipeline conflict or structural hazard: • Two instructions try to write to the register file at the same time! • Only one write port Ops! We have a problem! R-type R-type Load R-type R-type

  7. 1 2 3 4 5 Load Ifetch Reg/Dec Exec Mem Wr 1 2 3 4 R-type Ifetch Reg/Dec Exec Wr Important Observation • Each functional unit can only be used once per instruction • Each functional unit must be used at the same stage for all instructions: • Load uses Register File’s Write Port during its 5th stage • R-type uses Register File’s Write Port during its 4th stage • 2 ways to solve this pipeline hazard.

  8. Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Wr Ifetch Reg/Dec Exec Solution 1: Insert “Bubble” into the Pipeline Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Clock • Insert a “bubble” into the pipeline to prevent 2 writes at the same cycle • The control logic can be complex. • Lose instruction fetch and issue opportunity. • No instruction is started in Cycle 6! Load R-type Pipeline R-type R-type Bubble

  9. Ifetch Reg/Dec Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Ifetch Reg/Dec Exec Mem Wr Solution 2: Delay R-type’s Write by One Cycle • Delay R-type’s register write by one cycle: • Now R-type instructions also use Reg File’s write port at Stage 5 • Mem stage is a NOOP stage: nothing is being done. 4 1 2 3 5 Exec Mem R-type Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Clock R-type R-type Load R-type R-type

  10. A M B D Modified Control & Datapath IR <- Mem[PC]; PC <– PC+4; A <- R[rs]; B<– R[rt] S <– A + B; S <– A or ZX; S <– A + SX; S <– A + SX; if Cond PC < PC+SX; M <– S M <– Mem[S] Mem[S] <- B M <– S R[rd] <– M; R[rt] <– M; R[rd] <– M; Equal Reg. File Reg File S Exec IR PC Inst. Mem Next PC Mem Access Data Mem

  11. Ifetch Reg/Dec Exec Mem The Four Stages of Store Cycle 1 Cycle 2 Cycle 3 Cycle 4 • Ifetch: Instruction Fetch • Fetch the instruction from the Instruction Memory • Reg/Dec: Registers Fetch and Instruction Decode • Exec: Calculate the memory address • Mem: Write the data into the Data Memory Store Wr

  12. Ifetch Reg/Dec Exec The Three Stages of Beq Cycle 1 Cycle 2 Cycle 3 Cycle 4 • Ifetch: Instruction Fetch • Fetch the instruction from the Instruction Memory • Reg/Dec: • Registers Fetch and Instruction Decode • Exec: • compares the two register operand, • select correct branch target address • latch into PC Beq Mem Wr

  13. A M S B D Control Diagram IR <- Mem[PC]; PC < PC+4; A <- R[rs]; B<– R[rt] S <– A + B; S <– A or ZX; S <– A + SX; S <– A + SX; If Cond PC < PC+SX; M <– S M <– Mem[S] Mem[S] <- B M <– S R[rd] <– S; R[rt] <– S; R[rd] <– M; Equal Reg. File Reg File Exec IR PC Inst. Mem Next PC Mem Access Data Mem

  14. Administrivia • Get started on Lab 5: Pipelining is difficult to get right! Be sure that we will test “gotcha” cases in our mystery programs… • Next Lecture: “state-of-the-art” pipelining • Out-of-order execution/register renaming • Reorder buffers

  15. Processor Input Control Memory Datapath Output The Big Picture: Where are We Now? • The Five Classic Components of a Computer • Today’s Topics: • Recap last lecture • Review MIPS R3000 pipeline • Administrivia • Advanced Pipelining • SuperScalar, VLIW/EPIC

  16. ALU PC Clk Recall: Single Cycle Control! Control Ideal Instruction Memory Control Signals Conditions Instruction Rd Rs Rt 5 5 5 Instruction Address A Data Address Data Out 32 Rw Ra Rb 32 Ideal Data Memory 32 32 32-bit Registers Next Address Data In B Clk Clk 32 Datapath

  17. Data Stationary Control • The Main Control generates the control signals during Reg/Dec • Control signals for Exec (ExtOp, ALUSrc, ...) are used 1 cycle later • Control signals for Mem (MemWr Branch) are used 2 cycles later • Control signals for Wr (MemtoReg MemWr) are used 3 cycles later Reg/Dec Exec Mem Wr ExtOp ExtOp ALUSrc ALUSrc ALUOp ALUOp Main Control RegDst RegDst Ex/Mem Register IF/ID Register Mem/Wr Register ID/Ex Register MemWr MemWr MemWr Branch Branch Branch MemtoReg MemtoReg MemtoReg MemtoReg RegWr RegWr RegWr RegWr

  18. A M S B D PC Datapath + Data Stationary Control IR v v v fun rw rw rw wb wb wb Inst. Mem Decode me me WB Ctrl rt Mem Ctrl rs ex op im rs rt Reg. File Reg File Exec Mem Access Data Mem Next PC

  19. Let’s Try it Out 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 these addresses are octal

  20. A M S B = IF D Next PC 10 PC Start: Fetch 10 n n n n Inst. Mem Decode WB Ctrl Mem Ctrl IR im rs rt Reg. File Reg File Exec Mem Access Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15

  21. A M S B = ID D IF Next PC 14 PC Fetch 14, Decode 10 n n n lw r1, r2(35) Inst. Mem Decode WB Ctrl Mem Ctrl IR im 2 rt Reg. File Reg File Exec Mem Access 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 Data Mem

  22. M S B = D ID IF Next PC 20 PC Fetch 20, Decode 14, Exec 10 n n addI r2, r2, 3 Inst. Mem Decode WB Ctrl lw r1 Mem Ctrl IR 35 2 rt Reg. File Reg File r2 Exec Mem Access EX Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15

  23. M B = D EX ID Next PC 24 IF PC Fetch 24, Decode 20, Exec 14, Mem 10 n addI r2, r2, 3 sub r3, r4, r5 Inst. Mem Decode WB Ctrl lw r1 Mem Ctrl IR 3 4 5 Reg. File Reg File r2 r2+35 Exec Mem Access Data Mem M 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15

  24. r5 = WB D M EX Next PC ID 30 IF PC Fetch 30, Dcd 24, Ex 20, Mem 14, WB 10 beq r6, r7 100 Inst. Mem Decode addI r2 WB Ctrl sub r3 lw r1 Mem Ctrl IR 6 7 Reg. File M[r2+35] Reg File r4 r2+3 Exec Mem Access Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 Note Delayed Branch: always execute ori after beq

  25. r7 = D Next PC EX 100 ID PC IF Fetch 100, Dcd 30, Ex 24, Mem 20, WB 14 ori r8, r9 17 Inst. Mem Decode addI r2 WB Ctrl sub r3 Mem Ctrl beq IR 9 xx 100 r1=M[r2+35] Reg. File Reg File r6 r2+3 r4-r5 Exec Mem Access Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 WB M

  26. = D WB Next PC M ___ EX PC ID Fetch 104, Dcd 100, Ex 30, Mem 24, WB 20 ? Inst. Mem Decode WB Ctrl Mem Ctrl IR Reg. File Reg File Exec Mem Access Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 Fill it in yourself!

  27. = D Next PC WB ___ PC EX Fetch 110, Dcd 104, Ex 100, Mem 30, WB 24 ? ? Inst. Mem Decode WB Ctrl Mem Ctrl IR ? Reg. File Reg File ? Exec ? Mem Access Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 M Fill it in yourself!

  28. = D Next PC ___ WB PC Fetch 114, Dcd 110, Ex 104, Mem 100, WB 30 ? ? ? Inst. Mem Decode WB Ctrl Mem Ctrl IR ? Reg. File Reg File ? ? Exec Mem Access Data Mem 10 lw r1, r2(35) 14 addI r2, r2, 3 20 sub r3, r4, r5 24 beq r6, r7, 100 30 ori r8, r9, 17 34 add r10, r11, r12 100 and r13, r14, 15 Fill it in yourself! M

  29. Bubbles Stalls Valid IRex IR IRwb Inst. Mem WB Ctrl IRmem Dcd Ctrl Ex Ctrl Mem Ctrl Equal Reg. File Reg File A D S Exec PC Next PC B Mem Access M Data Mem Pipelined Processor • Separate control at each stage • Stalls propagate backwards to freeze previous stages • Bubbles in pipeline introduced by placing “Noops” into local stage, stall previous stages.

  30. RAW Data Hazard IF DCD EX Mem WB IF DCD EX Mem WB WAW Data Hazard IF DCD EX Mem WB IF DCD OF Ex Mem IF DCD OF Ex RS WAR Data Hazard Recap: Data Hazards • Avoid some “by design” • Eliminate WAR by always fetching operands early (DCD) in pipe • Eliminate WAW by doing all WBs in order (last stage, static) • Detect and resolve remaining ones • Stall or forward (if possible)

  31. New Inst Inst I Window on execution: Only pending instructions can cause exceptions Instruction Movement: Inst J Hazard Detection • Suppose instruction i is about to be issued and a predecessor instruction j is in the instruction pipeline. • A RAW hazard exists on register if Rregs( i ) Wregs( j ) • Keep a record of pending writes (for inst's in the pipe) and compare with operand regs of current instruction. • When instruction issues, reserve its result register. • When on operation completes, remove its write reservation. • A WAW hazard exists on register if Wregs( i ) Wregs( j ) • A WAR hazard exists on register if Wregs( i ) Rregs( j )

  32. Record of Pending Writes In Pipeline Registers IAU • Current operand registers • Pending writes • hazard <= ((rs == rwex) & regWex) OR ((rs == rwmem) & regWme) OR ((rs == rwwb) & regWwb) OR ((rt == rwex) & regWex) OR ((rt == rwmem) & regWme) OR ((rt == rwwb) & regWwb) npc I mem op rw rs rt Regs PC im n op rw B A alu n op rw S D mem m n op rw Regs

  33. Resolve RAW by “forwarding” (or bypassing) IAU • Detect nearest valid write op operand register and forward into op latches, bypassing remainder of the pipe • Increase muxes to add paths from pipeline registers • Data Forwarding = Data Bypassing npc I mem Regs op rw rs rt PC Forward mux im n op rw B A alu n op rw S D mem m n op rw Regs

  34. Forwarding

  35. R D T What about Memory Operations? op Rd Ra Rb • If instructions are initiated in order and operations always occur in the same stage, there can be no hazards between memory operations! • What does delaying WB on arithmetic operations cost? – cycles ? – hardware ? • What about data dependence on loads? R1 <- R4 + R5 R2 <- Mem[ R2 + I ] R3 <- R2 + R1 “Delayed Loads” • Can recognize this in decode stage and introduce bubble while stalling fetch stage (hint for lab 5!) • Tricky situation: R1 <- Mem[ R2 + I ] Mem[R3+34] <- R1 Handle with bypass in memory stage! op Rd Ra Rb A B Rd Mem Rd to reg file

  36. Compiler Avoiding Load Stalls:

  37. What about Interrupts, Traps, Faults? • External Interrupts: • Allow pipeline to drain, Fill with NOPs • Load PC with interrupt address • Faults (within instruction, restartable) • Force trap instruction into IF • disable writes till trap hits WB • must save multiple PCs or PC + state • Recall: Precise Exceptions  State of the machine is preserved as if program executed up to the offending instruction • All previous instructions completed • Offending instruction and all following instructions act as if they have not even started • Same system code will work on different implementations

  38. Exception/Interrupts: Implementation Questions 5 instructions, executing in 5 different pipeline stages! • Who caused the interrupt? Stage Problem interrupts occurring IF Page fault on instruction fetch; misaligned memory access; memory-protection violation ID Undefined or illegal opcode EX Arithmetic exception MEM Page fault on data fetch; misaligned memory access; memory-protection violation; memory error • How do we stop the pipeline? How do we restart it? • Do we interrupt immediately or wait? • How do we sort all of this out to maintain preciseness?

  39. Exception Handling IAU npc I mem detect bad instruction address Regs lw $2,20($5) Excp PC detect bad instruction im n op rw Excp B A detect overflow alu Excp S D mem detect bad data address m Excp Allow exception to take effect Regs

  40. IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB Another Look at the Exception Problem Time Data TLB • Use pipeline to sort this out! • Pass exception status along with instruction. • Keep track of PCs for every instruction in pipeline. • Don’t act on exception until it reache WB stage • Handle interrupts through “faulting noop” in IF stage • When instruction reaches end of MEM stage: • Save PC  EPC, Interrupt vector addr  PC • Turn all instructions in earlier stages into noops! Bad Inst Inst TLB fault Program Flow Overflow

  41. Resolution: Freeze Above & Bubble Below IAU • Flush accomplished by setting “invalid” bit in pipeline npc I mem freeze op rw rs rt Regs PC bubble im n op rw B A alu n op rw S D mem m n op rw Regs

  42. FYI: MIPS R3000 clocking discipline phi1 • 2-phase non-overlapping clocks • Pipeline stage is two (level sensitive) latches phi2 phi1 phi2 phi1 Edge-triggered

  43. Resource Usage TLB TLB I-cache RF WB ALUALU D-Cache MIPS R3000 Instruction Pipeline Decode Reg. Read Inst Fetch ALU / E.A Memory Write Reg TLB I-Cache RF Operation WB E.A. TLB D-Cache Write in phase 1, read in phase 2 => eliminates bypass from WB

  44. Im ALU Im ALU Im Dm Reg Reg ALU Recall: Data Hazard on r1 Time (clock cycles) IF ID/RF EX MEM WB add r1,r2,r3 Reg Reg ALU Im Dm I n s t r. O r d e r sub r4,r1,r3 Dm Reg Reg Dm Reg Reg and r6,r1,r7 Im Dm Reg Reg or r8,r1,r9 ALU xor r10,r1,r11 With MIPS R3000 pipeline, no need to forward from WB stage

  45. op Rd Ra Rb mul Rd Ra Rb A B Rd R Rd T to reg file MIPS R3000 Multicycle Operations Use control word of local stage to step through multicycle operation Stall all stages above multicycle operation in the pipeline Drain (bubble) stages below it Alternatively, launch multiply/divide to autonomous unit, only stall pipe if attempt to get result before ready - This means stall mflo/mfhi in decode stage if multiply/divide still executing - Extra credit in Lab 5 does this Ex: Multiply, Divide, Cache Miss

  46. IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB IFetch Dcd Exec Mem WB Is CPI = 1 for Our Pipeline? • Remember that CPI is an “Average # cycles/inst • CPI here is 1, since the average throughput is 1 instruction every cycle. • What if there are stalls or multi-cycle execution? • Usually CPI > 1. How close can we get to 1??

  47. Case Study: MIPS R4000 (200 MHz) • 8 Stage Pipeline: • IF–first half of fetching of instruction; PC selection happens here as well as initiation of instruction cache access. • IS–second half of access to instruction cache. • RF–instruction decode and register fetch, hazard checking and also instruction cache hit detection. • EX–execution, which includes effective address calculation, ALU operation, and branch target computation and condition evaluation. • DF–data fetch, first half of access to data cache. • DS–second half of access to data cache. • TC–tag check, determine whether the data cache access hit. • WB–write back for loads and register-register operations. • 8 Stages: What is impact on Load delay? Branch delay? Why?

  48. Case Study: MIPS R4000 IF IS IF RF IS IF EX RF IS IF DF EX RF IS IF DS DF EX RF IS IF TC DS DF EX RF IS IF WB TC DS DF EX RF IS IF TWO Cycle Load Latency IF IS IF RF IS IF EX RF IS IF DF EX RF IS IF DS DF EX RF IS IF TC DS DF EX RF IS IF WB TC DS DF EX RF IS IF THREE Cycle Branch Latency (conditions evaluated during EX phase) Delay slot plus two stalls Branch likely cancels delay slot if not taken

  49. MIPS R4000 Floating Point • FP Adder, FP Multiplier, FP Divider • Last step of FP Multiplier/Divider uses FP Adder HW • 8 kinds of stages in FP units: Stage Functional unit Description A FP adder Mantissa ADD stage D FP divider Divide pipeline stage E FP multiplier Exception test stage M FP multiplier First stage of multiplier N FP multiplier Second stage of multiplier R FP adder Rounding stage S FP adder Operand shift stage U Unpack FP numbers

  50. MIPS FP Pipe Stages FP Instr 1 2 3 4 5 6 7 8 … Add, Subtract U S+A A+R R+S Multiply U E+M M M M N N+A R Divide U A R D28 … D+A D+R, D+R, D+A, D+R, A, R Square root U E (A+R)108 … A R Negate U S Absolute value U S FP compare U A R Stages: M First stage of multiplier N Second stage of multiplier R Rounding stage S Operand shift stage U Unpack FP numbers • A Mantissa ADD stage • D Divide pipeline stage • E Exception test stage

More Related