1. The Dynamic Circuit Network Rick Summerhill Chief Technology Officer, Internet2 rrsum@internet2.edu Internet2 Fall Member Meeting 9 October 2007 San Diego, CA

2. Contents • Challenges • The HOPI Project • The DRAGON Investigation • The Internet2 National Infrastructure • The DICE Collaboration • Control Plane Developments

3. Challenges - Applications • Applications in the R&E community are very demanding • Large Data Flows, for example, from LHC • 8 Gbps flows for hours on end • UNL to FERMILAB was a recent example • Flows that demand deterministic capabilities • Very high definition video • eVLBI applications in radio astronomy and the SKA project • Applications that demand very high bandwidth for relatively short periods of time • Backplanes for distributed computers or content distribution schemes • How should the R&E community support these applications?

4. Challenges - Infrastructure • Five years ago it became clear the R&E community would have access to greater infrastructure • The early R&E networks were IP based • With the advent of access to dark fiber, the R&E community now had access to layers 0 and up • Given the application challenges, what is the best way to utilize that infrastructure? • Also, access to additional infrastructure has given rise to the “hybrid networking” idea - meaning operating networks at multiple layers in a coherent way • Three years ago, Internet2 created the Hybrid Optical and Packet Infrastructure (HOPI) project to examine this

5. HOPI Project - Overview • How does one effectively use a richer set of capabilities available to network designers and end users? • Core IP packet switched networks • A set of optically switched lightpaths available for dynamic provisioning • Examine a hybrid of shared IP packet switching and dynamically provisioned circuits • HOPI Project (Hybrid Optical and Packet Infrastructure) to examine how to put it all together? • Dynamic Provisioning - setup and teardown of optical paths • Hybrid Question - how do end hosts use the combined packet and circuit switched infrastructures?

6. HOPI General Problem

7. HOPI Topology

8. HOPI Control Plane • HOPI emulates an optical circuit switched environment while using Ethernet switches • Need a control plane and development • Internet2 created an RFP for a HOPI Testbed Support Center (TSC) • The MAX, NCREN, and Indiana GigaPoPs responded and won the RFP • Lead to the use of the DRAGON control plane elements • DRAGON is NSF funded and GMPLS based - Dynamic Resource Allocation of GMPLS Optical Networks (DRAGON)

9. Ethernet Router SONET Switch Lambda Switch Heterogeneous Network TechnologiesComplex End to End Paths “Horizontal” multi-layer adaptations for multi-domain AS 2 AS 1 IP Control Plane AS 3 IP Control Plane IP Control Plane VLSR Router MPLS LSP Ethernet over SONET VLSR Ethernet over WDM End System End System Ethernet Segment VLSR Established VLAN Ethernet Segment VLSR Established VLAN

10. DRAGON Control Plane -Key Elements • Virtual Label Swapping Router – VLSR • Open source protocols running on PC act as GMPLS network element (OSPF-TE, RSVP-TE) • Control PCs participate in protocol exchanges and provisions covered switch according to protocol events (PATH setup, PATH tear down, state query, etc) • Network Aware Resource Broker – NARB • Intradomain listener, Path Computation, Interdomain Routing • Client System Agent – CSA • End system or client software for signaling into network (UNI or peer mode) • Application Specific Topology Builder – ASTB • User Interface and processing which build topologies on behalf of users • Topologies are a user specific configuration of multiple LSPs

11. XML Interface ASTB Web page CLI Interface One NARB per Domain VLSR- Virtual Label Switching Router • GMPLS Proxy • (OSPF-TE, RSVP-TE) • Local control channel • CLI,TL1, SNMP, others • Used primarily for ethernet switches • Provisioning requests via CLI, XML, or ASTB

12. Interdomain Path Computation – A Hierarchical Architecture • NARB summarizes individual domain topology and advertise it globally using link-state routing protocol, generating an abstract topology. • RCE computes partial paths by combining the abstract global topology and detailed local topology. • NARB’s assemble the partial paths into a full path by speaking to one another across domains.

13. Internet2 Infrastructure

17. Nodes

18. Nodes

19. Nodes

20. Nodes

21. Nodes

22. Nodes

23. Nodes

24. Dynamic Circuit Network • Similar to the IP network model, but using different protocols • Connections are one armed • Physical connection • Control Plane Protocols • Create Circuits in seconds for periods of hours to days • Only through the Ciena equipment at the start, eventually evolving to the full platform

25. Current Control Plane Developments • Intra-domain • Currently on the Ciena platform, to be deployed on the Infinera platform in the future • Have an Installed working configuration on the Cienas • Inter-domain • Working with the DICE (Dante, Internet2, Canarie, ESnet) group • Working on Topology, Path Computation and Scheduling, Signaling, and Lookup Services (think a circuit based DNS) • Web service based communications • Face-to-face meeting every three months and biweekly calls

26. Dynamic Circuit NetworksWeb Service Control Plane Interfaces IDC WS E-NNI Inter-Domain Controller (IDC) WS E-NNI IDC WS I-NNI IF Management System (I-NNI) WS UNI WS UNI WS I-NNI IF WS I-NNI IF MPLS (I-NNI) GMPLS (I-NNI) SONET/TDM (Dataplane) Router(MPLS)/PSC (Dataplane) Ethernet/L2SC (Dataplane) • Web Services provides a mechanism to deal with heterogeneous control planes • inspired by the standards bodies work on control plane protocols, but not just recreating that work at the web service level

27. Web Service based E-NNIMain Components Routing Topology Exchange Domain Abstraction Varying levels of dynamic information Resource Scheduling Multi-Domain path computation techniques Resource identification, reservation, confirmation Signaling path setup, service instantiation Host Lookup Service Uses DNS pointers

28. More on the Demo Start with Lambdastation Assumes the circuit is in place Modifies the routers in real time to switch the flow from the IP path to the circuit path What was new in this demo was the addition of the ability to setup the circuit in real time Lambdastation called the DC network to setup the circuit using the DC network control plane protocols PerfSonar project provided the monitoring tools in the demo

29. Network Research Example - Phoebus • Phoebus - TCP data flows • File transfers over long distance segments not requiring congestion control • Lead by Martin Swaney at the University of Delaware

30. Questions? network@internet2.edu