Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab

1. Internet Measurement Initiatives in the Wisconsin Advanced Internet Lab Paul Barford Computer Science Department University of Wisconsin – Madison Spring, 2003

2. Talk Objectives • Motivate and describe Wisconsin Advanced Internet Lab (WAIL) • Internal lab environment • External lab environment • Provide some detail on three current projects • Anomaly detection and characterization • Distributed intrusion monitoring • Understanding packet loss pb@cs.wisc.edu

3. Motivation for New Tools • Any area of scientific research is limited by the tools available for experimental study • “If your only tool is a hammer then everything looks like a nail” • 2001 NRC report: “network research community is in danger of ossification due to strictures of experimental systems” • Challenge: “Capturing a day in the life of the Internet” • New experimental tools can open up areas of research that have not previously been accessible pb@cs.wisc.edu

4. An Internet Instance Lab • A hands-on test environment designed to recreate paths and conditions identical to those in the Internet from end-to-end-through-core • Requires large amount of routing and end host equipment • Network and host equipment able to recreate (not emulate) a wide range of services, configurations and traffic conditions • Complete instrumentation of end-to-end paths • Deployment of disruptive prototypes pb@cs.wisc.edu

5. Key Challenges • Design • Configurations and management • Traffic generation • Propagation delay • Validation pb@cs.wisc.edu

6. The Wisconsin Advanced Internet Lab • Our realization of an IIL • Developed over past 18 months by UW/Cisco team • Supported by $3.5M equipment grant from Cisco and UW matching funds • Used to purchase over 75 pieces of networking equipment • Phase 1 nearing completion => Abilene recreation • Other partners: EMC, Spirent, Intel, Fujitsu, Sun • Research initiatives in many areas… pb@cs.wisc.edu

7. External Environment • Essential complement to internal environment • Existing infrastructure • DOMINO systems (1 class A + 2 class B’s + Dshield) • Surveyor + WAWM systems (~70 nodes) • New database and front end by summer ‘03 • Partnerships and other available systems • Condor/Grid Infrastructures • Passive flow measurements • FlowScan data from UW, Internet2, others… pb@cs.wisc.edu

8. Project 1: Detecting Anomalies in IP Flows • Motivation: Anomaly detection remains difficult • Objective: Improve understanding of traffic anomalies • Approach: Multiresolution analysis of data set that includes IP flow, SNMP and an anomaly catalog • Method: Integrated Measurement Analysis Platform for Internet Traffic (IMAPIT) • Results: Identify anomaly characteristics using wavelets and develop new method for exposing short-lived events pb@cs.wisc.edu

9. Our Data Sets • Consider anomalies in IP flow and SNMP data • Collected at UW border router (Juniper M10) • Archive of ~6 months worth of data (packets, bytes, flows) • Includes catalog of anomalies (after-the-fact analysis) • Group observed anomalies into four categories • Network anomalies (41) • Steep drop offs in service followed by quick return to normal behavior • Flash crowd anomalies (4) • Steep increase in service followed by slow return to normal behavior • Attack anomalies (46) • Steep increase in flows in one direction followed by quick return to normal behavior • Measurement anomalies (18) • Short-lived anomalies which are not network anomalies or attacks pb@cs.wisc.edu

10. Multiresolution Analysis • Wavelets provide a means for describing time series data that considers both frequency and time • Powerful means for characterizing data with sharp spikes and discontinuities • Using wavelets can be quite tricky • We use tools developed at UW which together make up IMAPIT • FlowScan software • The IDR Framenet software pb@cs.wisc.edu

11. Ambient IP Flow Traffic pb@cs.wisc.edu

12. Flow Traffic During DoS Attacks pb@cs.wisc.edu

13. Deviation Score for Three Anomalies pb@cs.wisc.edu

14. Project 2: Coordinated Intrusion Detection • Motivation: Intrusion detection is a moving target • Objective: Coordinate intrusion monitoring between multiple sites around the Internet • Approach: Share data from firewalls, NIDS and tarpits (on unused IP space) • Method: Distributed Overlay for Monitoring Internet Outbreaks (DOMINO) • Results: Blacklists can be rapidly generated, false positives can be substantially lowered, new outbreaks can be easily identified pb@cs.wisc.edu

15. DOMINO: A new approach to DNIDS • Partnership with dshield.org • 1600 firewall and NIDS logs • Tarpits • Active monitor of unused IP space • 1 class A (this week), 2 class B’s • A protocol for node participation, data sharing and alert clustering • Chord-based overlay network • Extension of Intrusion Detection Message Exchange Format • Various clustering methods pb@cs.wisc.edu

16. Marginal Utility of Adding Nodes pb@cs.wisc.edu

17. SQL-Sapphire Analysis pb@cs.wisc.edu

18. Project 3: Understanding Packet Loss • Motivation: Many of the most basic aspects of packet loss are not understood • Where, when, how long, how often? • Focus: Developing a comprehensive understanding of packet loss in the Internet • Approach: Combine understanding of protocols and queue behavior to create a probe train which can accurately measure delay and loss. • Implications: End-to-end tools for pin-pointing loss, better transport protocols, better network management for congestion pb@cs.wisc.edu

19. Active versus Passive Loss Measures • Hypothesis: Active measures of loss are correlated with passive measures of loss • Assessment in Abilene • SNMP loss measures on all backbone routers • Active probes via Ping/Zing in Surveyor nodes at 10Hz, 20Hz and 100Hz • Tests in full mesh over one month period pb@cs.wisc.edu

20. Result: Active <> Passive pb@cs.wisc.edu

21. Summary • Both internal lab building initiatives and external measurement initiatives in WAIL • Internal facilities are intended to be open • We are seeking partnerships in external measurement projects. • DOMINO in particular pb@cs.wisc.edu