Pacific NorthWest Gigapop, Pacific Wave & International Peering

1. Pacific NorthWest Gigapop, Pacific Wave & International Peering David Morton, PNWGP RENU Technology Workshop

2. Introduction • Our many jobs: • University of Washington • Pacific Northwest Gigapop • Pacific Wave • Operate regional and state networks • Pacific Northwest Gigapop (PNWGP) - Regional and International interconnection point for: • Pacific Wave (including IEEAF link to Tokyo) • National R&E Networks (National Lamba Rail, Internet2) • Regional and State networks (Northern Tier, K20, other Universities in the region • Enabling advanced research in a wide variety of topics. Pacific Wave & International Peering

3. The Direct Connections Pacific Wave & International Peering

4. Pacific Wave • A joint project between Corporation for Education Network Initiatives in California (CENIC), Pacific Northwest Gigapop (PNWGP), University of Southern California (USC), and University of Washington (UW) • A distributed Internet Exchange Point (IXP) running the length of the entire United States Pacific Coast • Supports high-end networking and protocols, including IPv4 (ucast/mcast), IPv6 (ucast/mcast), Jumbo Frames Pacific Wave & International Peering

5. Pacific Wave Initial Services • Provide an open infrastructure for connecting international IP networks • Facilitate any-to-any connectivity between connectors without the need for involvement of exchange operators • Primary connectivity provided via shared VLANs • Private VLANs used sparingly on as-needed basis Pacific Wave & International Peering

6. Characteristics of the Service Offering • Distributed connectivity using shared broadcast domains • Strict connector policy to ensure high availability and uptime • Single MAC per VLAN • No direct connection of layer-2 switching devices • Spanning tree root guard Pacific Wave & International Peering

7. What is a Distributed IXP? • A switching fabric that extends beyond the confines of a small geographic region which Internet exchange points typically operate • Capable of providing direct connectivity between entities which do not share common, physical presences due to financial, operational, or logistical constraints • An architecture which does not define any central or primary point in the infrastructure Pacific Wave & International Peering

8. Shared VLAN Infrastructure • Local, shared VLANs • Seattle • SF Bay Area • Los Angeles • Intersite, shared VLANs • Seattle - SF Bay Area • SF Bay Area - LA • Seattle - LA • Standard MTU and Jumbo MTU overlay Pacific Wave & International Peering

9. Underlying Infrastructure • Exclusively Cisco 7600 and 6500 class devices with Sup720 • Redundant supervisor modules at critical points • All linecards are 67xx series fabric enabled • Strategic termination of TenGigE links based on fabric utilization • PIM-SM snooping used to contain multicast traffic Pacific Wave & International Peering

10. Layer-1 Infrastructure • Seattle • Nortel HDXc co-operated with Canarie • Cisco IEEAF ONS 15454 MSPP • 4 ML1000-2 line cards (Capable of GFP-F and LEX) • KREONet2 ONS 15454 MSPP • Los Angeles • CENIC ONS 15454 MSPP • AARNet ONS 15454 MSPP Pacific Wave & International Peering

11. Combined Layer-1 and Layer-2 Infrastructure Pacific Wave & International Peering

12. Wide Area Connectivity • TenGigabit Ethernet LAN-PHY circuit provisioned via NLR, Seattle to Sunnyvale • TenGigabit Ethernet LAN-PHY circuit provisioned via CENIC , Sunnyvale to LA Pacific Wave & International Peering

13. Physical Topology • Five 65/7600 layer-2 switches • Deployed over NLR and CENIC 10g lambdas • Seattle • Westin • SF Bay Area • Sunnyvale (Level 3) • Palo Alto (PAIX) • Los Angeles • One Wilshire • Equinix (600 W 7th) • Equinix (818 W 7th) Pacific Wave & International Peering

14. NLR Enables Pacific Wave Pacific Wave & International Peering

15. International Participation Pacific Wave & International Peering

16. Pacific Northwest Participation Pacific Wave & International Peering

17. Examples of Collaborations • WIDE/TLEX-PNWGP (IEEAF Tokyo - Seattle): Data Reservoir; HD Video • AARNet (SX-Transport Sydney to Seattle): Huygens Data Transfer • CA*Net4: Data Reservoir and Huygens Data Transfer • ESNet/UltraScienceNet: SuperComputing 2005 • National LambdaRail: SuperComputing 2005, iGrid 2005 • OptiPuter Chicago - Seattle - San Diego: iGrid2005, HD Video • Abilene: HD Video • KREONet2 (GLORIAD Daejon - Seattle): HD Video Pacific Wave & International Peering

18. GLIF (Global Lambda Integrated Facility) Pacific Wave & International Peering

19. Pacific Wave Optical Lightpath Exchange (GOLE) Diagram Pacific Wave & International Peering

20. The Hybrid Approach • The hybrid exchange operator can provide easy interconnection between layer-1 and layer-2 devices as necessary • Today: manual with physical cross-connect • Future: optically switched manually or via control plane protocols • Technology choices made based on: • Performance requirements • Bandwidth requirements • Timeframe desired for lightpath activation • Availability of wide-area transport Pacific Wave & International Peering

21. A Distributed GOLE? • A term as yet defined in the GLIF community • What properties might a distributed GOLE (GLIG Optical Lightpath Exchange) have? • Multi-transport interconnect between nodes • Layer-1 lambda • Layer-2 frame mapped (i.e. GFP-F) • Layer-2 switched • Able to make establish lightpaths between and through the nodes with the efficiency of a single operating entity • No bandwidth limitations between nodes (within reason…) Pacific Wave & International Peering

22. Future • Today, Pacific Wave layer-2 switching is a series of linear interconnected fabrics • Perhaps more complex wide-area topologies may one day be necessary on the layer-2 network • Bandwidth management • Resiliency • MPLS layer-2 Ethernet VPNs may provide additional opportunities to support future requirements • Optical switching facilitates efficient lightpath establishment, especially during technology translation, i.e. frame-mapped  switched Ethernet Pacific Wave & International Peering

23. Thank You dmorton@u.washington.edu

24. An Amazing Feat at SC05 Pacific Wave & International Peering