Access Regulation to Hot-Modules in Wormhole NoCs

1. Access Regulation toHot-Modules in Wormhole NoCs Or: Hot-Modules, Cool NoCs Isask’har (Zigi) Walter Supervised by: Israel Cidon, Ran Ginosar and Avinoam Kolodny Technion – Israel Institute of Technology

2. Hot-Modules • NoC is designed and dimensioned to meet QoS requirements • Buffer sizing, routing, router arbitration, link capacities, … • NoC designers cannot tune everything • Modules typically have limited capacity • High-demanded, bandwidth limited modulescreate edge bottlenecks • In SoC, often known in advance • Off-chip DRAM, on-chip special purpose processor • System performance is strongly affected • Even if the NoC has infinite bandwidth Hot-Modules in Wormhole NoCs

3. Hot Module (HM) in NoC • Wormhole, BE NoC • At high Hot Moduleutilization, multiple worms “get stuck” in the network • Two problems arise: • System Performance • Source Fairness IP(HM) Interface Hot-Modules in Wormhole NoCs

4. Hot Module Affects the System IP1(HM) IP2 Problem#1 Interface Interface Interface IP3 • HM is not a local problem. Traffic not destined at the HM suffers too! Hot-Modules in Wormhole NoCs

5. Source Fairness Problem Problem#2 HM Interface Multiple locally fair decisions Global fairness • The limited, expensive HMresource isn’t fairly shared Hot-Modules in Wormhole NoCs

6. Our Approach • Problem is not caused by the NoC • But rather by a congested end-point • Solution should address the root cause • Not the symptoms • Utilize existing NoC infrastructure • Solve both problems • Simple and efficient Hot-Modules in Wormhole NoCs

7. Hot Module Congestion During congested periods, sources should not inject packets towards the HM Will experience increased delay anyway Better wait at the source, not in the network Keep routers unmodified! Hot-Modules in Wormhole NoCs

8. HM Allocation Control Basics Control IP3 Interface IP1 AllocationController IP2(HM) Interface NoC Interface Interface IP4 Hot-Modules in Wormhole NoCs

9. HM Allocation Control Basics IP3 Interface IP1 Control AllocationController IP2(HM) Interface NoC Interface Interface IP4 Hot-Modules in Wormhole NoCs

10. HM Allocation Control Basics IP3 Interface IP1 Control AllocationController IP2(HM) Interface NoC Interface Interface IP4 Hot-Modules in Wormhole NoCs

11. HM Control Packets Dest. Source Req. Credit Dest. Source Credit Credit request packet Credit reply packet • The HM Controller receives all requests and can employ any scheduling policy • Control packets are sent using a high service level • Bypassing (blocked) data packets! Hot-Modules in Wormhole NoCs

12. Multiple Priority Router Control packets Hot-Modules in Wormhole NoCs

13. Enhanced Request packet Dest. Source Req. Credit Priority Expiration Deadline … … • The request may include additional data as needed • payload’s priority, deadline, expiration time, etc. Optional fields Credit request packet Hot-Modules in Wormhole NoCs

14. HM Allocation Controller Optional • The HM Allocation Controller is customized according to system’s requirements CreditRequests CreditReplies Requests Decoder Reply Encoder PendingRequestsTable LocalArbiter HM Access Controller Hot-Modules in Wormhole NoCs

15. Further Enhancements • Short packets are not negotiated • Source’s quota is slowly self-refreshing • The mechanism is turned-off when the network is not congested • Crediting modules ahead of time hides request-grant latency • For light-load periods Hot-Modules in Wormhole NoCs

16. Not Classic Flow-Control • Flow-control protects destination’s buffer • A pair-wise protocol • HM access regulation protects the system • Many-to-one protocol Hot-Modules in Wormhole NoCs

17. Results – Synthetic scenario R R R R R R R R R R R R HM Module Module Module R R R R Module Module Module Module Module Module Module Module Module Module Module Module • Hotspot traffic • All-to-one traffic with all-to-all background traffic • High network capacity • Limited hot module bandwidth • HM controller arbitration: Round-robin Hot-Modules in Wormhole NoCs

18. System Performance Without regulation WithRegulation Average Packet Latency X30 X10 Hot-Modules in Wormhole NoCs

19. Hot vs. non-Hot Module Traffic Background TrafficWithout regulation HM Trafficwithout regulation HM Trafficwith regulation Background TrafficWith regulation Average Packet Latency X40 Using regulation, non-HM traffic latency is drastically reduced Hot-Modules in Wormhole NoCs

20. Source Fairness Source#16no regulation Source#16with regulation Source#5with regulation Source#5no regulation R R R R 1 2 3 4 R R R R 5 6 7 8 R R R R 9 10 11 12 R R R R 13 14 15 16 Hot-Modules in Wormhole NoCs

21. Fairness in Saturated Network No allocation control With allocation control Simulation results for a 4-by-4 system,Data packet length: 200 flitsControl packet length: 2 flits • Hot-Module Utilization: 99.99% • Regulated Hot-Module Utilization: 98.32% Hot-Modules in Wormhole NoCs

22. MPEG-4 Decoder SRAM2 SDRAM 22% of all traffic MED CPU VU RAST AU SRAM2 SDRAM IDCT SRAM1 UP SAMP BAB RISC ADSP 25% of all traffic • Real SoC • Over provisioned NoC • Two hot-modules Hot-Modules in Wormhole NoCs

23. Results – MPEG-4 Decoder All traffic HM/non-HM traffic breakdown X8 X2 @80% load: X8 reduction @80% load: X2 reduction Hot-Modules in Wormhole NoCs

24. The HMs are better utilized Significant differences in BW! No allocation control With allocation control Flows destined at HM1 Flows destined at HM2 Total 1HM1 2HM1 3HM1 4HM1 9HM1 10HM1 11HM1 8HM2 10HM2 11HM2 12HM2 • Without regulation, the hot-modules are only 60% utilized • Traffic to one HM blocks the traffic to the other! Hot-Modules in Wormhole NoCs

25. Hot-Module Placement Hot-Modules in Wormhole NoCs

26. Summary • Hot-modules are common in real SoCs • Hot-modules ruin system performance and are not fairly shared • Even in NoCs with infinite capacity • The network intensifies the problem • But can also provide tools for resolving it • Simple mechanism achieves dramatic improvement • Completely eliminating the HM effects Hot-Modules, Cool NoCs! Hot-Modules in Wormhole NoCs

27. Hot-Modules, Cool NoCs! QNoC Research Group Thank you! Questions? zigi@tx.technion.ac.il QNoC Research Group Hot-Modules in Wormhole NoCs

28. Backup Slides Hot-Modules in Wormhole NoCs

29. Wormhole Routing • Suits well on chip interconnect • Small number of buffers • Low latency IP2 Interface Interface IP1 Hot-Modules in Wormhole NoCs

30. Router-Based Approach? Input Buffer Output Buffer • Virtual circuit • Fair queuing • Dedicated queues • Deflective routing • Packet combining • Packet dropping • Backpressure (credit/rate based) • and more… X-Bar • Can router detect congested periods? Hot-Modules in Wormhole NoCs

31. Router-Based Solutions? Input Buffer Output Buffer • Routers must be fast, power and area efficient • A few buffers • Efficient routing • Simple arbitration policy • No state/flow memory • Problem caused by end-points • Address the root-cause X-Bar Hot-Modules in Wormhole NoCs

32. Future Work • Dynamically set hot-modules • Other scheduling policies at hot-module controller • Single/Multiple control modules for multiple HMs • Placement Hot-Modules in Wormhole NoCs

33. References [1] E. Bolotin, I. Cidon, R. Ginosar, A. Kolodny, “QoS Architecture and Design Process for Cost-Effective Network on Chip”, Journal of Systems Architecture, 2004 [2] G. F. Pfister and V. A. Norton, "Hot Spot contention and combining in multistage interconnection networks," IEEE Trans. Comp., Oct. 1985 [3] D. Bertozzi, A. Jalabert, S. Murali R. Tamhankar, S. Stergiou, L. Benini, and G. De Micheli , "NoC Synthesis Flow for Customized Domain Specific Multiprocessor Systems-on-Chip", IEEE Transactions on Parallel and Distributed Systems, 2005 Hot-Modules in Wormhole NoCs

34. Is that new problem? Seem to draw most attention • Hotspots were comprehensively studied in the past • Classically, solutions are categorized by the mechanism policy • Avoidance-based (frequently impossible) • Detection-based (requires threshold tuning) • Prevention-based (overhead during light load) • And by the mechanism implementation • Central arbitration • Router-based • End-to-end flow-control Hot-Modules in Wormhole NoCs

35. Saturation (Un)Fairness Less than 1% of HM BW! • A saturated router divides available BW equally between inputs R R R R HM IP IP IP R R R R IP IP IP IP R R R R IP IP IP IP Hot-Modules in Wormhole NoCs