GigaSunet

1. GRID Connections in GigaSunet Börje Josefsson <bj@sunet.se>

2. GigaSunet: Nationwide 10 Gbit/s network • Core network (22 cities): • 5.270 km of lambdas. • 27 inter-city segments with 10 Gbit/sec. • 49 routers (Cisco 124xx). • Access network (2,5 Gbit/s): • 25 dark fiber rings. • 3 segments of 2,5 Gbit/sec. lambda. • 1 STM-1 connection. • 70 routers (Cisco 10720).

4. Research networks in Europe

5. Suggested access scenario Uplink 2,5 Gbit/sec (redundant) 8 * Fast Ethernet 4 * Gigabit Ethernet For future ideas… Production network (connected to University LAN). Local peering IPv6-network (via UTI) GRID National research network (via UTI)

6. GRID-connection at the edge • Fiber directly to SUNETs access router. • There are two access routers at each university. • 1 Gbit/sec. [Gigabit Ethernet]. • IP version 4 supported today at this speed. • Avoid the local LAN-equipment if possible! • As ”straight” to Your hosts as possible ! • Avoid (cheap) switches etc.!

7. But it is still slow?!? For a single session, GigaSunet can be felt1/ as slow! Speed of light and buffer sizes limits the speed. Theoretical maximum speed with 32 Kbytes buffer. Bwbest = buffer / RTT 1/ Unless You have tweaked Your computer and LAN.

8. ”Fast-and-far” • You have to think. Twice… • Tweak Your OS parameters. • Sweden is approx. 18 ms ”long”. • 1 Gbit/sec * 36 ms RTT  4,5 megabyte buffer in the end hosts (usually it is 32 kilobyte…). • This has nothing to do with the network itself, this is just how computer communication works. • It doesn’t help to have a “private” network!

9. Test results Luleå-Stockholm (approx. 1.000 km). Hosts connected to Gigabit Ethernet. dino# ttcp -s -t -f m -l 61440 -n 20345 wilma.sunet.se ttcp-t: buflen=61440, nbuf=20345, align=16384/0, port=5001 tcp -> wilma.sunet.se ttcp-t: socket ttcp-t: connect ttcp-t: 1249996800 bytes in 10.35 real seconds = 966181102 bits/sec +++ ttcp-t: 20345 I/O calls, msec/call = 0.52, calls/sec = 1966.33 ttcp-t: 0.0user 1.6sys 0:10real 16% 0i+0d 0maxrss 0+341060pf 41346+4csw 966,2 Mbit/sec. This is ”real” traffic, not counting the framing and packet overhead.

10. Alt. B Alt. A Core switch Core switch Datacenter switch Datacenter switch Host Host Access router Access router Can You spot the difference? • Answer: • 0,001% more cable in ”B” (compared to total distance) . • More (expensive) equipment in ”B”. • 75-80% less performance in ”B”.

11. Conclusion • SUNET and GigaSunet are ready to handle this. Now. • The ”problems” are local at each university. • We can help You with tuning guidelines and act as testing partners. • SUNET test hosts are connected directly to the core network, without LAN equipment in between. That helps us isolate “local” problems.

12. What You should do next.. • Contact Your local network group. • Get a fiber connection from Your site to one of the SUNET access routers. • Have them1 configure the router for Your connection. • Connect the GRID equipment, as ”clean” as possible. • Tune Your system! 1/ Done locally at each university, except Uppsala.

13. Questions? Börje Josefsson <bj@sunet.se>