Gaussian Interconnections for On-Chip Networks

1. Gaussian Interconnections for On-Chip Networks Ramón Beivide and Enrique Vallejo University of Cantabria, Spain enrique@atc.unican.es

2. Gaussian Interconnections for On-chip Networks Index • Introduction: Why a Topology? • Dense Gaussian Networks and other topologies • Different layouts • Routing: • ideas: Adaptive routing, deadlock avoidance, fault tolerance • Unicast routing • Broadcast Routing • Perfect placement of resources • Expansibility: • Increasing number of nodes in a Gaussian network • Hierarchical Gaussian networks • Some ideas about cache coherence R. Beivide, E. Vallejo

3. Gaussian Interconnections for On-chip Networks Introduction • Future trends: many PE on a chip • Possible interconections: bus, MIN, direct network • Bus-based interconnections do not scale – they do not provide a sufficient bandwith when there are many PEs. MIN hard to implement in a chip. • Direct networks with a given topology: The way to connect different routers in the chip R. Beivide, E. Vallejo

4. b -2+2i 2i 2+2i 1+2i -1+2i -2+i -1+i i 1+i 2+i b -2 -1 0 1 2 -1-i -2-i -i 1-i 2-i -2i 2-2i -2-2i -1-2i 1-2i Gaussian Interconnections for On-chip Networks Mesh Network Number of Nodes N: N = b x b = b2 Diameter k: k = (b-1) + (b-1) = 2b-2 R. Beivide, E. Vallejo

5. 3i -1+2i 2i 1+2i 2 b+1 -2+i -1+i i 1+i 2+i -3 -2 -1 0 1 2 3 -2-i -1-i -i 1-i 2-i -1-2i -2i 1-2i -3i Gaussian Interconnections for On-chip Networks Diamond Network Number of Nodes N: N = (b+1)2 + b2 Diameter k= b + b = 2b R. Beivide, E. Vallejo

6. Number of Nodes N: N = b x b = b2 Diameter k = b -1 -2+2i 2i 2+2i 1+2i -1+2i -2+i -1+i i 1+i 2+i -2 -1 0 1 2 -1-i -2-i -i 1-i 2-i -2i 2-2i -2-2i -1-2i 1-2i Gaussian Interconnections for On-chip Networks Torus Network b b R. Beivide, E. Vallejo

7. 2b+1 0 1 2 -1 -2 3 -3 2+i 2+i -2i 2i -3i 3i 2+i 1+2i 2+i 2+i 2+i 2+i -i i 1+i 2+i 2+i 2+i Gaussian Interconnections for On-chip Networks Dense Gaussian Network Number of Nodes N: N = (b+1)2 + b2 Diameter k = b • Same # links as torus, with • peripheral links. • Lower mean distance and • Diameter. R. Beivide, E. Vallejo

8. Gaussian Interconnections for On-chip Networks Topological properties comparative Lower average distance and diameter R. Beivide, E. Vallejo

9. Gaussian Interconnections for On-chip Networks Area comparative R. Beivide, E. Vallejo

10. 11 12 10 9 13 16 15 14 8 7 5 6 17 18 19 20 2 4 1 3 3 -1+2i -3i -2+i -1+i 2+i 1+i i 1-i -2i 2-i 1-2i 2i 3i -2 2 -3 1-2i -i -1 1+2i -1-i -2-i 0 1 24 23 21 22 0 Gaussian Interconnections for On-chip Networks Different Layouts • Different layouts for the same network: • Mesh-like layout • Without peripheral links, bounded link length R. Beivide, E. Vallejo

11. Gaussian Interconnections for On-chip Networks Routing ideas • Adaptive routing: in-fligh packets can choose their (minimal) path from info in the Routing Record (jumps in each direction), depending on congestion or other parameters. • Deadlock avoidance: Bubble routing proposed as a cost-effective deadlock avoidance mechanism (used in IBM’s Blue Gene). Only 2 virtual channels needed per link. • Fault-tolerant routing: Inmunet proposed as a fast, efficient mechanism to detect link failures and restore network performance. R. Beivide, E. Vallejo

12. Gaussian Interconnections for On-chip Networks Unicast Routing 3i -1+2i 2i 1+2i Route from a to b: Routing record generated From the difference: dest-source (x, y) -2+i -1+i i 1+i 2+i -3 -2 -1 0 1 2 3 -2-i -1-i -i 1-i 2-i Example: i – (-1-i) = 1+2i (x=1, y=2) 1 jump to the right, 2 jumps up Movement from source node to the origin (node 0) generates routing record. -1-2i -2i 1-2i Example 2: The movement makes the arrow fall outside the original network  Peripheral links used Translations from surrounding replicas of the network are considered, to obtain an optimal RR -3i R. Beivide, E. Vallejo

13. 3i 2i NW NE 1+2i -1+2i i -2+i -1+i 1+i 2+i P -3 -2 2 3 -1 1 -i 1-i 2-i -2-i -1-i -2i 1-2i -1-2i SW SE -3i Gaussian Interconnections for On-chip Networks Broadcast Routing • Triangle-based broadcast • Minimum number of steps • The same for any node (due to peripheral links) • Balanced use of resources • Simple routing based on labels (see abstract) R. Beivide, E. Vallejo

14. Gaussian Interconnections for On-chip Networks Perfect placement of resources Resource distribution over the network. All nodes have resources within a given distance (example: distance 1) Resource example: I/O ports Processing elements Memory banks ... R. Beivide, E. Vallejo

15. Gaussian Interconnections for On-chip Networks Expansibility: Increasing # nodes • Increasing Gaussian network: Network can be expanded with the number of nodes necessary to increase diameter in 1 unit: 4k +4. • Alternatively, hierarchical Gaussiannetworks have been proposed, joining several Gaussian networks with another gaussian pattern. Useful for CMPs sceneries, for example (different latency, link length, etc. in each hierarchical level): • Lower level: interconnection between different cores inside a chip. Fast and reliable, with low latency • Higher level: interconnection between different chips. Slower and with higher latency. R. Beivide, E. Vallejo

16. Gaussian Interconnections for On-chip Networks Expansibility: Increasing # nodes Lower level (on-chip) with a dense Gaussian pattern. Higher level, with the same pattern. Central routers will have 8 links: 4 internal links 4 external links Route from one node to another: 1) Route to the central router of the same gaussian 2) Route in the higher level to the desired gaussian. 3) Route from the central router of the dest. Gaussian, to the destiny node. R. Beivide, E. Vallejo

17. Gaussian Interconnections for On-chip Networks Cache coherence in Gaussian networks • Recent proposals based in broadcast, such as TokenB (M. Hill) can beneficiate from a Gaussian interconnection: • Broadcast block requests (latency optimized with Gaussian interconection). • Unicast response with grants (Tokens) to use memory blocks, between different nodes and main memory. • There is no need to maintain a directory for coherence. • Broadcast network can work as a bus with a snoopy-like protocol. R. Beivide, E. Vallejo

18. R. Beivide, E. Vallejo

19. Additional comments (not presented) • Dense Gaussian Networks are isomorphic to Dense Midimew Networks. However, these two topologies are not isomorphic in the general case (not dense). In this work, related to Dense Gaussian networks, properties studied for both Gaussian and Midimew topologies are presented. • References in the next slide will be thus referred to both Midimew and Gaussian networks R. Beivide, E. Vallejo

20. Commented References (I) • Midimew networks were first introduced in: R. Beivide, E. Herrada, J.L. Balcázar, Agustín Arruabarrena, “Optimal Distance Networks of Low Degree for Parallel Computers”. IEEE Transactions on Computers, Vol. 40, No 10, Oct 1991, pp. 1109-1124.This paper introduces properties, analysis and some rectangular (mesh-like) layouts. Unicast routing is also proposed, but based on labeling nodes with integer labels (instead of Gaussian numbers). • Bounded link-length layouts were introduced in:E. Vallejo, R. Beivide y C. Martínez, “Practicable Layouts for Optimal Circulant Graphs”. Proceedings of the “13th Euromicro Conference on Parallel, Distributed and Network-based Processing”, Lugano, Switzerland, Feb. 2005. A previous work on Midimew folding, which obtained a worse result (maximum link length 4) is the following one:Francis C. M. Lau, Guihai Chen, “Optimal Layouts of Midimew Networks”. IEEE Transactions on Parallel and Distributed Systems, Vol 7, No 9, pp 954-961 R. Beivide, E. Vallejo

21. Commented References (II) • Gaussian Networks will be introduced in:C. Martínez, R. Beivide, J. Gutierrez and E. Gabidulin. "On the Perfect t-Dominating Set Problem in Circulant Graphs and Codes over Gaussian Integers". Accepted for presentation at ISIT’05, September, Australia.This paper also deals with perfect resource placement. • Broadcasting in Dense Gaussian Networks will be introduced in:R. Beivide, C. Martínez, E. Vallejo, J. Gutierrez, C. Izu, “Gaussian Interconnection Networks”. Accepted for presentation at the Spanish Parallelism Conferences, Sept. 05, Granada, Spain.This paper also introduces unicast routing in terms of the Gaussian numbers (instead of integer labels) • Hierarchical Gaussian Networks will be introduced in:Miquel Moretó, Carmen Martínez, Enrique Vallejo, Ramón Beivide, Mateo Valero, “Hierarchical Topologies for Large-scale Two-level Networks”, Accepted for presentation at the Spanish Parallelism Conferences, Sept. 05, Granada, Spain. R. Beivide, E. Vallejo

22. Commented References (III) • Bubble routing is described inV. Puente, C. Izu, R. Beivide, J.A. Gregorio, F. Vallejo and J.M. Prellezo, “The Adaptative Bubble Router”, Journal of Parallel and Distributed Computing. Vol 61 - nº 9, September 2001 • Inmunnet was introduced inV. Puente, J.A. Gregorio, F. Vallejo and R. Beivide. "Immunet: A Cheap and Robust Fault-Tolerant Packet Routing Mechanism". 31th Annual International Symposium on Computer Architecture (ISCA-31), pp. 198-209, 2004. • Token Coherence was presented in:M. M. K. Martin, M. D. Hill, and D. A. Wood. "Token Coherence: Decoupling Performance and Correctness". 30th Annual International Symposium on Computer Architecture (ISCA-30), pp. 182-193, 2003. R. Beivide, E. Vallejo