1. Custom Code Generationfor Soft Processors Martin Labrecque Peter Yiannacouras Gregory Steffan ECE Dept. University of Toronto Presented at RAAW 2006, Orlando, FL

2. Soft Processor: Processor in FPGA FPGA Processor • Compelling solution: software programmable • Soft processors are end-user customizable • Different application realm than hard ASIC processors • Can add more features: trade area for performance • Well known approach: add custom instructions (ex. A*B+C) Techniques orthogonal to custom instructions Programmable Logic

3. Application-Specific Code Generation  Use default gcc, ISA? Interested in app-specific optimizations  Application Compiler Processor Customized for: • Area • Power • Wallclock time • Freq. requirements

4. Infrastructure

5. SPREE System(Soft Processor Rapid Exploration Environment) ISA Datapath • Verify ISA against datapath • Datapath Instantiation • Control Generation • Multi-cycle/variable-cycle FUs • Multiplexer select signals • Interlocking • Branch handling SPREE RTL • Output: Synthesizable Verilog [CASES 05, FPGA 06] • Input: Processor description • Made of hand-coded components Processor Description • SPREE System

6. Back-End Infrastructure RTL 20Benchmarks (MiBench, Dhrystone 2.1, RATES, XiRisc) Stratix 1S40C5 Cycle Count 2. Area 3. Clock Frequency 4. Power We can measure area/performance/energy accurately Modelsim RTL Simulator Quartus II 5.0 CAD Software

7. Area efficiency #Million Instr. x Frequency  # Cycles x Area • A combined metric: MIPS #Million Instr.  1000 LEs WallclockTime x Area • 4 criteria trading-off (power not included) • Want app-specific ( average) improvement

8. Representative Processors <900 LEs, <70 MHz >1500 LEs, >100 MHz F: Fetch D: Decode R: Register EX: Execute M: Memory WB: Writeback Serial F/D/R/EX/WB Pipe3 F/D R/EX/M WB Pipe5 F D R/EX1 EX2/M WB EX1 WB2 F D R EX2/M EX3/WB1 Pipe7

9. SPREE vs Nios II Serial faster Pipe7 Pipe5 Pipe3 smaller

10. Code Generation Options Studied( Outline ) Low-level hardware-software tradeoffs Reducing hardware shift support Removing hazard detection logic Impact of unique ISA features Removing delay slots Hi/Lo registers vs 3-operand multiplies Using unaligned memory load and stores Application-specific register management Operand scheduling and forwarding lines Limiting the use of architected registers Combining these into app-specific optimizations    

11. Reducing Hardware Shift Support Best performance per area: Using hard multiplier for shifting Multiplications and shifts: both in software? Software shifting using additions & subtractions Impact of removing the dedicated LUT-based shifter? Costs ~250LEs, 30% of smallest soft processor Can we have partial hardware support for shifting? 

12. 343 LEs 48 LEs 2 fixed-amount shifters is cheap! Area for Various Shift Strategies (Pipe3)

13. Dynamic Instructions Containing Shifts less than 2% of shift amounts are variable some benchmarks have very few shifts Percentage

14. How to get rid of the shifter Software-only shifts require an order of magnitude more cycles to compute Measure the cost in cycles for each shift operation Replace shifts by hard shifts and/or software shifts: Srl 8 Srl 8 Srl 4 Srl 4 Srl 4 Srl 4 Srl 3 Srl 3 Srl 3 Srl 3 Srl 3 Shift_left(1) ... Srl 16 or or or Evaluate cost in cycles for all combinations of shifters available

15. Impact of up to 2 Fixed-Amount Shifters (pipe3) Can improve area efficiency by up to 65% Beneficial for certain applications only Area efficiency (MIPS/1000LE)

16. PC hazard avoided Instr. in delay slot Branch/ Jump F/D R/EX/M WB Time Time  Removing Delay Slots load hazard avoided • Default MIPS has branch and load delay slots • Under what conditions are they worth it? • Load delay slots need no additional hardware support • Because of hazard detection in the processor • Branch delay slots require hardware support • We only have predict-not-taken so far • Are working on better branch prediction Instr. in delay slot Load F/D R/EX/M WB

17. Removing Load Delay Slots (serial) 3% better performance for Serial, 2% for Pipe3 Normalized Wall-Clock Time

18. Removing Branch Delay Slots pipe3: 7% performance hit pipe7 improvements: 13% freq, 8% performance

19.  3-Operand Multiplies vs Hi/Lo Registers Default MIPS has Hi/Lo registers Motivated by multi-cycle multiplies Hold multiplication results (Hi and Lo each 32 bits) Two special instructions to access Hi/Lo Which to choose? • 3-operand multiplies (NIOS2 and Microblaze) • Two instructions compute high and low parts • Result is stored in register file Hi/Lo Register file Multiplier MUX

20. Impact of 3-Operand Multiplies 8% slower clock Saves area, reduces frequency, increases power Normalized Value

21. Impact of 3-Operand Multiplies Only pipe3 benefits from cycle savings

22. Forwarding Lines and Code Generation Necessary to forward both operands (A and B)? Simultaneous dependences Non commutative operations  r3 = r1 + r2 r4 = r3 + r3 r3 = r1 + r2 r4 = r5 - r3 r3 = r1 + r2 r4 = r3 – r5 • Compiler can reorder commutative operands of instrs • Can compiler compensate when only one forwarding line? • Save ~30 LEs for fwding line and incur more stall cycles? Added 1-2% cycle improvement with 1 fwding line 3-4% short of 2 fwding lines’ performance for 30% of apps, 1 fwding line more area efficient

23. Soft Processor Customization Techniques • Best overall (general purpose) processor • Best per application (application-tuned) • Reduce processor by reducing the ISA (Subset) SPREE automatically removes • Unused connections • Unused components • Unused parts of the ISA • Apply optimization techniques (Opt)

24. Average Combined Improvements (pipe3) Subsetting & Opts +25% 36% Opts +12% Efficiency (MIPS/1000LEs) App-Specific: +11% Opt: 2 fixed shifts, no dly slots, 3-op mult, op sched overall 36% improvement in efficiency! Subsetting +8%

25. Summary Software-only and custom shifters Load delay slots Branch delay slots 3-operand multiply Operand scheduling to save a forwarding line App-specific Useless with hazard detection Useful with poor branch prediction Processor-specific App-specific  12% area efficiency over app-specific processor 17% area eff. over subsetted app-specific proc.  without adding complexity! Conclusion

26. Future Research Integrating branch prediction in SPREE Research on memory hierarchy Automatic selection of app-specific SP features

27. Thank you

28. Architectural Parameters Used in SPREE Multiplication Support Hardware FU or software routine Shifter implementation Flipflops, multiplier, or LUTs Pipelining Depth (2-7 stages) Forwarding lines We focus on core microarchitecture (for now)

29. No specific evaluation of studied features in SP Related work • Custard [Dimond, Mencer, Luk] • Customizable forwarding lines • Optional delay slots • NIOS II [Altera] • 3-operand multiply • No delay slots • Microblaze [Xilinx] • 3-operand multiply • Branches with and without delay slots

30. Removing some/all hazard detection logic • Can the compiler compensate with scheduling? • E.g., worst case use no-ops to ensure correctness • Challenge: variable, multi-cycle instructions • What is the cost/benefit of doing so? F/D R/EX/M WB Pipe3 Potential hazard F/D stall R/EX/M WB Time

31. Up to 10% area and 6% frequency gains Impact of Removing Hazard Detection Logic