National Lambda Rail Update

1. National Lambda Rail Update Peter O’Neil Westnet July 2003

2. Rationale for NLR • Market conditions reviving willingness of private sector to renew strong partnerships of academia, industry and government once again • Research community in US and across world have large scale research and development applications calling for increasing bandwidth as evidenced by the Grid and Terascale projects • Unified approach best investment for largest payoff for the most users • Federal program funding support focused on research and experimental applications needing significant (non-production) network capacity • Leadership worldwide in research and education increasingly dependent upon large-scale collaborative research activities • Meet the future needs of the research and academic community economically

3. Goal of NLR “The fundamental and overriding goal of NLR is to provide an enabling experimental infrastructure for new forms and methods of science and engineering.” – Peter O’Neil

4. Goals of NLR • Two broad areas of research activity NLR infrastructure will facilitate, one: • High-performance Computational Science and Engineering Research • grid computing, load migration, latency tolerant algorithms, resource aware algorithm selection and scheduling, MTU, distributed and multidisciplinary efforts involving multiple length and time scales in the simulations and multiple science and engineering disciplines, very tightly-coupled simulation codes and algorithms, not to mention data sensing and collection elements, data repositories, visualization environments, etc.

5. Goals of NLR • Second broad area of research activity: • High Performance Networking Research • conduct network layer or link-layer research on some wavelengths, while maintaining a stable link layer and network layer on other wavelengths to conduct research at the transport layer or higher layers • build the next generation of network protocols using a protocol development environment (PDE) • conduct research activities in the optronics space itself

6. Drivers and Motivations • Cost • Control • Flexibility • Responsiveness • Innovation • End-to-End networking

7. Technology Partners • Level 3 • 20 year fiber IRU, O&M, Co-lo • Cisco Systems • Optronics • Experimental Optronics TBD

8. NLR Membership Status • Signed Agreements • CENIC • PNWGP • Duke • Florida • Georgia Tech • Virginia Tech • UCAID • Service Agreements • PSC - 3 year with options for years 4 and 5 • Case Western – 5 year • Pending • CIC • NCAR • Texas

9. 15808 Terminal 15808 OADM 15808 Regen Fiber route Leased waves NLR Footprint and Physical Layer Topology SEA POR BOI SAC BOS NYC CHI OGD CLE DEN SVL PIT WDC FRE KAN RAL NAS STR LAX PHO WAL ATL SDG OLG DAL JAC

10. 15808 15808 7600 12400 15500 15500 15500 15454 15454 15454 PoP Architecture NLR PoP @ Level3 Colo West East Black =Included Gray =Extra Metro • Metro muxes are in • most cases not part • of NLR • Some cities connect • existing metro networks DWDM 10GE or OC192 1GE

11. Planned Configuration • At least 4 express waves on each 15808 span • Initial footprint to go West coast to at least Raleigh • Capacity up to 20 express waves and 20 add/drop waves per span • 1 wave terminated in NLR 12400 router (with the exception of SAN) • 1 wave terminated on NLR 7600 switch • 1 10GE client facing port on 12400 • 8 1GE client facing ports on 7600 • Client router/switch ports and extra waves connect directly at colo and/or across metro DWDM

12. Why Fiber? • Capacity needed is not otherwise affordable • Capabilities needed are not available • Cheaper in the long range • Leverage with “carriers” • Insurance against monopoly behavior • Stable and predictable anchor points

13. Fiber = Control • “Research can’t be held hostage to what industry thinks the market wants.” • “Infrastructure investments of the past are getting in the way of real innovation.” • “The situation is reminiscent of the telco infrastructure situation before ARPANET.” Bob Kahn CRA-Snowbird ‘02

14. NLR Service Offerings • Waves!!!! • Dark Fiber • ‘MetaPoP’ capabilities • Dedicated OC-192c or 10GE waves • No wave can interfere with others • Shared IP service • Must be able to tolerate “random scheduled downtime.” • Supports experimental code • Subdivided waves for dedicated uses • Delivered as gigabit Ethernet

15. Deployment stages for each route • Chromatic Dispersion Testing (CDT) of the fiber • Material and parts delivery • Manufacturing • Shipping (from NH to staging area in SC and then re-shipp to NLR co-lo facilities) • Installation at NLR co-lo • Testing by Cisco • Route turned over to NLR for live traffic testing

16. NLR Governance • 501c3 not-for-profit (seeking IRS certification) • Joint Network Research Council • Joint Technology Council • Distributed Network Engineering • Operation Centers • Waves governed/managed by various groups and approved by Board of Directors

17. Engineering & Operations • Layer1 Transport Center • Ensures all fiber paths and optical gear are up and operating • Primary point of contact for catastrophic failure affecting all switched and routed transport distribution systems within Shared IP Service Center(s). • Serves as primary vendor contact for coordinating service and repair. • Responsible for "virtual NOC" function through careful coordination with Shared IP Service Center(s) and Experimental Support Center. • Hosts the Help Desk 800 number and has primary responsibility for triage and appropriate IVR forwarding as applicable. • Expected to select provider from existing NLR member qualified 24x7 GigaPop NOCs

18. Engineering & Operations • Shared IP Service Center(s) • Maintain responsibility for all switched and routed services • Work in close coordination with the Layer 1 Transport Center to resolve service-affecting issues • Anticipated that NLR users will expect more stability of service at Layer 2 than Layer 3 • Additional IP Service Centers may be required going forward as requirements evolve/demand

19. Engineering & Operations • Experimental Support Center • Responsible for initial setup configuration for experimental wave segments. • Closely coordinates activities with Layer 1 Transport Center. • Coordinates introduction and addition of experimental hardware with NLR infrastructure. • Tool recommendations to allow researchers to change or alter configurations – a key function • Support the study of the usefulness of the NLR infrastructure for network research

20. KEY New ETF NLR Provides Existing Mountain Region TeraGrid Level3 Colo NCAR ETF Juniper T640 Router Chicago MAN Dark Fiber Cisco ONS 15454 Cisco ONS 15454 ETF 10G ETF 10G Via NLR NLR Cisco ONS 15808 ETF 10G Juniper T320 Router Cisco 6509 NLR Utah ETF 10G Via NLR Spare Cisco ONS 15454 Cisco 6509 NCAR Grid-enabled Resources Utah Grid-enabled Resources

21. Questions?