Interconnection Networks

1. ENGS 116 Lecture 19 Interconnection Networks Vincent H. Berk November 14, 2005 Reading for Friday: 8.1-8.7 Reading for Monday: 8.8-8.13

2. ENGS 116 Lecture 19 Project Reports • Due by Beginning of class next Monday, November 24 • Content • Introduction and description of the topic • Coverage of topic: breadth/depth, appropriate background information • Analysis and discussion • References: correct citations, proper form • Writing • Spelling • Grammar • Style and presentation • Assume that the reader is familiar with basic architecture concepts • Must use appropriate citations. • Argument all your decisions. • Email all source code to <vberk@Dartmouth.EDU>

3. ENGS 116 Lecture 19 Project Presentations • 15 minutes – absolutely no more – Practice your timing! • All group members talk • 24th (4) of November, 1st (4) of December • Present: • research question • approach • results • conclusions • EVERYONE ATTENDS these presentations

4. ENGS 116 Lecture 19 Networks • Common topics of conversation: • direct (point-to-point) vs. indirect (multi-hop) • topology (e.g., bus, ring, directed acyclic graph, star) • routing algorithms • switching (aka multiplexing) • wiring (e.g., choice of media, copper, coax, fiber) • What really matters: • latency • bandwidth • cost • reliability

5. ENGS 116 Lecture 19 ABCs of Networks • Starting point: Send bits between 2 computers • Queue on each end • Can send both ways (“Full Duplex”) • Rules for communication? “protocol” • Inside a computer: • Loads/Stores: Request (Address) & Response (Data) • Need request & response signaling • Name for standard group of bits sent: packet

6. ENGS 116 Lecture 19 A Simple Example • What is the format of a packet? (Protocol) • Fixed? Number of bytes? Request/ Response Address/Data 1 bit 32 bits 0: Please send data from address 1: Packet contains data corresponding to request

7. ENGS 116 Lecture 19 Questions About Simple Example • What if more than 2 computers want to communicate? • Need computer address field (destination) in packet • What if packet is garbled in transit? • Add error detection field in packet (e.g., CRC) • What if packet is lost? • More elaborate protocols to detect loss (e.g., NAK, ARQ, time outs) • What if multiple processes per machine? • Queue per process • Questions such as these lead to more complex protocols and packet formats

8. ENGS 116 Lecture 19 A Simple Example Revisited • What is the format of a packet? • Fixed? Number of bytes? Request/ Response Address/Data CRC 2 bits 32 bits 4 bits 00: Request—Please send data from address 01: Reply—Packet contains data corresponding to request 10: Acknowledge request 11: Acknowledge reply

9. ENGS 116 Lecture 19 Additional Background • Connection of 2 or more networks: Internetworking • 3 cultures for 3 classes of networks • SAN: server (storage) networks, performance • LAN: workstations, cost • WAN: telecommunications, long range • Cost • Performance (BW, latency) • Reliability

10. Node Node Node Node SW Interface SW Interface SW Interface SW Interface Link Link Link Link HW Interface HW Interface HW Interface HW Interface ... ... Interconnect ENGS 116 Lecture 19 Interconnections (Networks) • Examples: • SAN networks (infiniband): 100s nodes; ≤ 100 meters per link • Local Area Networks (Ethernet): 100s nodes; ≤ 1000 meters • Wide Area Network (ATM): 1000s nodes; ≤ 5,000,000 meters

11. ENGS 116 Lecture 19 Software to Send and Receive • SW Send steps 1: Application copies data to OS buffer 2: OS calculates checksum, starts timer 3: OS sends data to network interface HW and says start • SW Receive steps 3: OS copies data from network interface HW to OS buffer 2: OS calculates checksum, if matches send ACK; if not, deletes message (sender resends when timer expires) 1: If OK, OS copies data to user address space and signals application to continue • Sequence of steps for SW: protocol • Example similar to UDP/IP protocol in UNIX

12. Node Node SW Interface SW Interface Link Link HW Interface HW Interface Overhead Overhead Link Bandwidth Link Bandwidth Interconnect Bisection Bandwidth Latency ENGS 116 Lecture 19 Network Performance Measures ... ...

13. ENGS 116 Lecture 19 Universal Performance Metrics Sender Sender Overhead Transmission time (size ÷ bandwidth) (processor busy) Time of Flight Transmission time (size ÷ bandwidth) Receiver Overhead Receiver (processor busy) Transport Latency Total Latency Total Latency = Sender Overhead+ Time of Flight+ Message Size ÷ BW +Receiver Overhead Includes header/trailer in BW calculation?

14. ENGS 116 Lecture 19 Simplified Latency Model • Total Latency ≈ Overhead + Message Size / BW • Overhead = Sender Overhead + Time of Flight + Receiver Overhead • Example: show what happens as we vary the following • Overhead: 1, 25, 500 µsec • BW: 10, 100, 1000 Mbit/sec (factors of 10) • Message Size: 16 Bytes to 4 MB (factors of 4) • If overhead is 500 µsec, how big a message is needed to get > 10 Mb/s of bandwidth?

15. ENGS 116 Lecture 19 Overhead, Bandwidth, Size 1000 bw1000 o1, bw1000 o500, bw1000 o25, bw1000 100 bw100 o1, bw100 o500, bw100 10 bw10 o500, bw10 Effective Bandwidth (Mbits/sec) o1, bw10 o1, bw100 1 o1, bw1000 o25, bw10 o25, bw100 o25, bw1000 0.1 o500, bw10 o500, bw100 o500, bw1000 0.01 16 64 256 1024 4096 16384 65536 262144 1048576 4194304 Message Size (bytes)

16. 100% 90% Msgs 80% 70% Why? Bytes 60% Cumulative % 50% 40% 30% 20% 10% 0% 0 1024 2048 3072 4096 5120 6144 7168 8192 Packet size ENGS 116 Lecture 19 Measurement: Sizes of Message for NFS • 95% messages, 30% bytes for packets ≤ 200 bytes • > 50% data transferred in packets = 8KB

17. I/O I/O Controller Controller ENGS 116 Lecture 19 HW Interface Issues • Where to connect network to computer? • Cache consistent to avoid flushes? ( memory bus) • Latency and bandwidth? ( memory bus) • Standard interface card? ( I/O bus) • MPP  memory bus; LAN, WAN  I/O bus CPU Network Network $ ideal: high bandwidth, low latency, standard interface L2 $ Memory Bus I/O bus Memory Bus Adaptor

18. ENGS 116 Lecture 19 SW Interface Issues • How to connect network to software? • Programmed I/O? (low latency) • DMA? (best for large messages) • Receiver interrupted or received polls? • Things to avoid • Invoking operating system in common case • Operating at uncached memory speed (e.g., check status of network interface)

19. Overhead 100 90 80 70 60 Polling 50 message overhead (µsecs) 40 30 Interrupts 20 10 0 0 10 20 30 40 50 60 70 80 90 100 message interarrival (µsecs) Time between messages ENGS 116 Lecture 19 CM-5 Software Interface • CM-5 example (MPP) • Time per poll 1.6 secs; time per interrupt 19 secs • Minimum time to handle message: 0.5 secs • Enable/disable 4.9/3.8 secs • As rate of messages arriving changes, use polling or interrupt? • Solution: Always enable interrupts, have interrupt routine poll until no messages pending • Low arrival rate  interrupt • High arrival rate  polling

20. ENGS 116 Lecture 19 Network Media Twisted Pair: Copper, 1mm thick, twisted to avoid antenna effect (telephone) Used by cable companies: high BW, good noise immunity Coaxial Cable: Plastic Covering Braided outer conductor Insulator Copper core Fiber Optics Total internal 3 parts are cable, light source, light detector. reflection Transmitter Air Receiver – L.E.D – Photodiode – Laser Diode light Silica source

21. Shared Media (Ethernet) Node Node Node Node Node Node Node Switch Switched Media (CM-5, ATM) ENGS 116 Lecture 19 Connecting Multiple Computers • Shared Media vs. Switched: pairs communicate at same time, “point-to-point” connections • Aggregate BW in switched network is many times that of shared • point-to-point faster since no arbitration, simpler interface • Arbitration in shared network? • Central arbiter for LAN? • Listen to check if being used (“Carrier Sensing”) • Listen to check if collision (“Collision Detection”) • Random resend to avoid repeated collisions; not fair arbitration; • OK if low utilization (a.k.a. data switching interchanges, multistage interconnection networks, interface message processors)

22. ENGS 116 Lecture 19 Switch Topology • Structure of the interconnect • Determines • Degree: number of links from a node • Diameter: max number of links crossed between nodes • Average distance: number of hops to random destination • Bisection: minimum number of links that separate the network into two halves (worst case) • Warning: these three-dimensional drawings must be mapped onto chips and boards which are essentially two-dimensional media • Elegant when sketched on the blackboard may look awkward when constructed from chips, cables, boards, and boxes (largely 2D)

23. ENGS 116 Lecture 19 Figure 8.15 A ring network topology. A Simple Example

24. b) Linear array d) Ring e) Tree f) Near-neighbor mesh h) 3–cube (hypercube) g) Completely connected ENGS 116 Lecture 19 Examples of Static Interconnection Network Topologies a) Bus c) Star

25. ENGS 116 Lecture 19 Figure 8.16: Network topologies that have appeared in commercial MPPs.

26. ENGS 116 Lecture 19 Important Topologies N = 1024 Type Degree Diameter Avg Dist Bisection Diam Avg D 1D mesh ≤ 2 N-1 N/3 1 2D mesh ≤ 4 2(N1/2 - 1) 2N1/2 / 3 N1/2 63 21 3D mesh ≤ 6 3(N1/3 - 1) 3N1/3 / 3 N2/3 ~ 30 ~ 10 Ring 2 N / 2 N/4 2 2D torus 4 N1/2 N1/2 / 2 2N1/2 32 16 Hypercube n n = LogN n/2 N/2 10 5

27. 0 1 0 0 0 1 2 3 ENGS 116 Lecture 19 Figure 8.14 A fat-tree topology for 16 nodes. 3 0 2 0 1 0 0 0 0 1 1 0 1 2 1 0 2 3 1 0 3 4 5 6 8 7 9 10 12 11 13 14 15

28. ENGS 116 Lecture 19 Figure 8.13 Popular switch topologies for eight nodes.

29. = switch = processor ENGS 116 Lecture 19 Examples of dynamic interconnection network topologies a) Crossbar switch b) 8  8 Baseline