Abilene Update Fall Member Meeting ’05 Philadelphia, PA

1. Abilene UpdateFall Member Meeting ’05Philadelphia, PA Steve Cotter Director, Network Services scotter@internet2.edu

2. Welcome • The Abilene Network • Hurricane Katrina • Advanced Services Across Abilene • Network Research Across Abilene • Other Network Services • Network Security • Abilene Network Futures

3. The Abilene Network

4. Abilene Partnerships • Indiana University • Juniper Networks • Nortel Networks • Qwest Communications • ITECs • NC ITEC • Ohio ITEC • San Diego ITEC • Texas ITEC • Internet2 Staff

5. Abilene Network Topology

6. Abilene Network Topology

7. Abilene Network Topology

8. Abilene Network Topology

9. Abilene Network Topology

10. Abilene Network Topology

11. Abilene Network Topology

12. Abilene Community • 38 direct connections (OC-3c  10 Gbps) • 3 10 GE connections (OC-192c SONET also supported) • 7 OC-48c connections & 3 GE connectors • 26 connected at OC-12c (622 Mbps) or higher • 240 Primary Participants – research universities and labs • Claremont Colleges, New World Symphony, Manhattan School of Music, Cleveland Museum of Art, Cleveland Institute of Music, Los Alamos National Lab and Qwest are the most recent additions • 130 Sponsored Participants - Individual institutions, K-12 schools, museums, libraries, research institutes • 34 Sponsored Educational Group Participants - state-based education networks See: http://abilene.internet2.edu/

13. Abilene R&E Peerings

14. Abilene International Peerings September 2005

15. Abilene Connector Fees

16. Abilene Participation Fees Effective January 1, 2006: • Abilene Primary Participation - $21,000 Effective January 1, 2007: • Abilene Primary Participation - $22,000 First increase since Abilene was launched in 1998

17. Hurricane Katrina

18. Hurricane Katrina

19. Hurricane Katrina • Hurricane Katrina strikes the Gulf Coast on August 29th, 2005. • Abilene’s unprotected lambda network link from Houston to Atlanta goes down. The IGP (IS-IS) automatically reroutes around the fault. • On September 1st, 2005 the damage to the carrier network was fully assessed and estimated to take days to repair. • During this time, Abilene was operating with the risk of network isolation if there is a loss of the Chicago to Kansas City link. • A redundancy plan was formulated and approved by Internet2 to have Abilene traffic route over the HOPI wave from Chicago to Seattle in the event that Chicago to Kansas link fails. The Abilene NOC engineers implement the redundancy plan. • Service is restored to the Houston to Atlanta link on September 8th, 2005. No Abilene outages occurred during this period.

20. Hurricane Katrina H 8 8 8 H 8 8 8 H 8 8 8 8 H

21. Hurricane Katrina We would like to thank our partner Qwest for the extraordinary efforts they made to repair the network. Great job! We also appreciate the support we received from the Abilene NOC and NLR. Thanks!

22. Abilene Redundancy • Responding to requests of our members, Internet2 has pursued redundancy options with our partner Qwest Communications. • Qwest has agreed to provide on a per port basis, redundant connections to the Abilene router, at the node, for a cost of $400 per month regardless of speed as long as the redundant circuit speed is equal to or less than the primary circuit. • This option is available to any active Abilene Connector who delivers their redundant circuit to the Abilene node. SONET and Ethernet framing methods would be supported under this option.

23. Abilene Redundancy Most Abilene Connectors Today: What We Can Offer:

24. Redundancy Offering • We can make the following redundant connections available to our members who bring their circuits to an Abilene node: • VLAN connections through an existing exchange point: • 1 GE $50,000.00 • Physical connections to the router: • OC3 $70,000.00 • OC12 $75,000.00 • OC48 $90,000.00 • OC192 $125,000.00 • 1 GE $80,000.00 • 10GE $125,000.00 • A redundant circuit must be equal to or less than the primary circuit in speed and will not carry traffic unless the primary circuit fails. • Each request will be evaluated on a case basis. The above figures are for budgetary purposes and are subject to change.

25. Redundancy Offering • Member’s requests for redundant circuits carried back to an Abilene node over the Qwest network will have their requests evaluated on a case-by-case basis for available capacity and pricing. • These types of connections currently must be SONET.

26. Advanced Services Across Abilene

27. IPv6 Peerings • IPv6 Deployment • Significant number of peers and connectors now have native connections: • Roughly 2/3 of the connectors are IPv6 enabled • Roughly 1/2 of the peers are IPv6 enabled • Connected to Palo Alto PAIX peering fabric at 333 Mbps for IPv6 and IPv4-Multicast experimental, non-production peering • 10 new experimental, non-production IPv6 peerings at the PAIX so far in 2005 • Connected to MCI MAE-West at OC-3 for IPv6-only experimental, non-production peering • Qwest and MCI collaborated in providing the connection

28. IPv6 Addressing • Abilene has /32 that it can distribute to its members • However, a number of connectors and members have or are acquiring their own address space: • 2001:4e0::/32 Wiscnet • 2001:5e8::/32 Pittsburgh Supercomputing Center • 2001:1860::/32 Pacific Northwest Gigapop • 2001:18e8::/32 Indiana University

29. IPv6 Security • Abilene NOC activities: • Limiting the v6 prefixes connectors send us (as we do for IPv4) • Limited filtering for peer networks • v6-security@internet2.edu is a mailing list for v6 security topics

30. Internet2 Involvement with the NAv6TF • Internet2 is active in the North American IPv6 Task Force (NAv6TF). • Rick Summerhill is on NAv6TF advisory committee • Abilene is key network component of the NAv6TF's Moonv6 national test network

31. Internet2 IPv6 Member Activities • North Carolina State University and Centaur Labs -- IPv6 streaming audio feeds from radio stations WCPE and WZYC • IPv6 H.323 at Georgia Tech • Abilene IPv6-enabled hosts • http://ipv6.internet2.edu/ipv6hosts.shtml

32. Internet2 Member Multicast Activities • DVGuide - http://db.arts.usf.edu/dvguide/listings.asp • Several campus radio stations multicasting across Abilene • ConferenceXP, a Microsoft Research initiative, relies on multicast and has been deployed at several schools • Access Grid continues to grow • More activity requiring "bridging" to multicast in challenged environments, using the rcBridge software from ANU • NYSERnet, Abilene and Internet2 deploying native IPv6 multicast • IPv6 Multicast demo live at Fall Member Meeting

33. Multicast Security • Basic measures on Abilene: • Not allowing multicast streams with RFC1918 source addresses • Not allowing multicast streams to "site local" group addresses (239.0.0.0/8) which is a similar idea to RFC1918 addresses, but for group addresses. • Blocking group addresses which are used for application which only have local significance. A good example of this is Norton Ghost. • Other measures are under consideration, such as: • Blocking all IANA reserved multicast group addresses • Place a limit on the number of MSDP SAs each Abilene Connector/Peer can originate

34. Internet2 Hands-on Multicast Workshops Upcoming Workshops: • Hartford, CT – 4-6 October 2005 • Ann Arbor, Michigan - 17-19 October 2005 • Albuquerque, New Mexico - 2-4 February 2006 http://multicast.internet2.edu/workshops/

35. Other Advanced Services • MPLS VPN testing – NC-ITEC experimenting (with ITEC Ohio) with creating a multipoint Layer 2 VPN using inter-domain MPLS tunnels and Virtual Private LAN Service (VPLS). Working in both a lab environment and between the ITECs using Abilene. • The goal is to examine multipoint alternatives for a possible Abilene private network service offering.

36. Network Research Across Abilene

37. Network Research Philosophy • Internet2 today does not do network research per se, but seeks to facilitate and support research projects led by faculty at member institutions • Make accessible network resources readily available to this community • Participate in research collaborations and provide support for proposals • Integrate research findings into the evolution of Internet2 network initiatives and services

38. Network Research Resources • Resources available to researchers: • Abilene Observatory • MAN LAN Exchange Point • HOPI testbed • Collaboration with NLR, Regional Optical Networks and other testbeds

39. Abilene Observatory The Abilene Observatory is a program that supports the collection and dissemination of network data associated with the Abilene Network. Provides researchers: • Operational data associated with a large-scale network • Data associated with the fundamental properties of basic network protocols. Two components of the Observatory: • Data collected by Abilene engineers using equipment located in the router nodes and operated by the Abilene NOC • Data collected by separate research projects using equipment collocated in the Abilene racks

40. Abilene Observatory There are more than 30 research projects currently using Observatory data. Some of the more recent additions are: • Flow Sampling and Anomaly Detection , Paul Barford, University of Wisconsin • Assess the Presence and Incidence of Alpha Flows in Backbone Links, Vincenzo Liberatore, Case Western Reserve University • Traffic Management and QoS Provisioning in IP Networks, Hassan Peyravi, Kent State University • Spatio-Temporal Network Analysis, Mark Crovella and Eric Kolaczyk, Boston University. • MINDS Project, Vipin Kumar, University of Minnesota • Study of the Temporal-spatial Correlations in Network Traffic, Don Towsley, University of Massechusetts For a more comprehensive list, see: http://abilene.internet2.edu/observatory/research-projects.html

41. Project Highlight: PlanetLab • PlanetLab Upgrade • PlanetLab nodes currently located at all Abilene router nodes, connected to the IP network • Upgrade will add connection to an MPLS L2VPN configuration forming a layer2 network where the PlanetLab nodes will provide the routing engines • Abilene becomes the layer2 circuit provider for PlanetLab • Normal users on Abilene don't have direct access to this new "backbone network" • The PlanetLab network can peer with the commodity network • Provides an infrastructure for network research that has national scope

42. Other Network Services: FiberCo & MAN LAN

43. FiberCo Overview • Tool designed to support optical initiatives in the regions or nationally • Spun off from NLR governance discussions • Internet2 took responsibility for forming the LLC • Operates on behalf of U.S. higher education and affiliates – Internet2 and NLR membership • Not an operating entity • Will not light the fiber – only a holding company • Functions • Market maker • Assignment vehicle for both national & regional optical initiatives • Dark fiber provider: Level3 Communications • 3 year pricing agreement ends March 06 • Intercity and metro fiber, new builds, consulting services • Exploring more formal relationships with other providers

44. Alabama* Arizona (CENIC) Arkansas* California (CALREN) Colorado (FRGP/BRAN) Connecticut (Conn. Education Network) Florida (Florida LambdaRail) Georgia (Southern Light Rail) Great Plains Network* (MIDnet) Indiana (I-LIGHT) Illinois (I-WIRE) Louisiana* (LONI) Massachusetts* Maryland, D.C. & northern Virginia (MAX) Michigan (MiLR) Minnesota* (BOREAS) National LambdaRail New England Region (NEREN) New Mexico (NMSU, UNM) New York (NYSERNet*, Cornell) North Carolina (NC LambdaRail) Ohio (Third Frontier Network) Oklahoma (OneNet) Oregon Pacific Northwest (Lariat – NIH BRIN, PNNL) Rhode Island (OSHEAN) SRON* (southeastern U.S.) Tennessee* (OneTN) Texas (LEARN) Virginia (MATP) Wisconsin (WiscNet) Wyoming State and Regional Optical Networks (*RONs with RFx’s issued or in process of acquiring fiber) (RONs in red have made dark fiber acquisitions through FiberCo)

45. States with Regional Optical Networks States with a RON

46. Dark Fiber Placement • Aggregate dark fiber assets acquired by U.S. R&E optical initiatives • CENIC (for CalREN & NLR) 6,200+ • FiberCo (via Level 3 for NLR & RONs) 8,600 • SURA (via AT&T) 6,000 • Plus 2,000 route-miles for research • NLR Phase 2 (WilTel & Level3) 5,000 • OARnet 1,500 • ORNL (via Qwest) 900 • NEREN 670 • Other projects (IN,IL,OR,CT…) 2,200+ Total (conservative estimate) 30,000+ • Over 60% of these assets are now held by RONs • Remainder held by NLR (~11,250 route-miles)

47. MAN LAN Exchange Point • Manhattan Landing in New York City - partnership with NYSERNet, Indiana University, and the IEEAF • Provides a high performance exchange facility for research and education networks • Located at 32 AoA in NYC - easy interconnection to many national and international carriers and other research and education networks • Peering model is open and bilateral • Cost recovery model - minimal connection charges for layer 2 facility, none for layer 1 connections • Working with AtlanticWave on future distributed exchange point along U.S. East Coast (NYC↔Miami)

48. MAN LAN Services • Layer 2 - Ethernet switch for IPv4/v6 peering with 1GigE and 10 GigE interfaces • Layer 1 - TDM based optical equipment (SONET / Ethernet interfaces) • Cisco 15454 • Nortel OME 6500 • Nortel HDXc • Layer 0 – Glimmerglass optical cross connect to facilitate changes

49. Network Security

50. Network Security Basic Premise: Abilene Security Policy is determined by the properties of an IP network • Control is at the edge • Hosts determine when and where to send packets and initiate flows • This control often leads to vulnerabilities • Hosts can become compromised • Hosts may be used to compromise other hosts • Can lead to large amounts of traffic sent to other hosts As a backbone network, we view Abilene as a ‘pipe’ and not a controlling entity