1 / 51

TCP STREAM PROCESSING AT GIGABIT LINE RATES

TCP STREAM PROCESSING AT GIGABIT LINE RATES. David Vincent Schuehler Dissertation Defense Washington University in St. Louis Department of Computer Science and Engineering November 3, 2004. Outline. Motivation and Background Architecture and Related Work Live Internet Traffic Processing

lauren
Télécharger la présentation

TCP STREAM PROCESSING AT GIGABIT LINE RATES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TCP STREAM PROCESSINGAT GIGABIT LINE RATES David Vincent Schuehler Dissertation Defense Washington University in St. Louis Department of Computer Science and Engineering November 3, 2004

  2. Outline • Motivation and Background • Architecture and Related Work • Live Internet Traffic Processing • Conclusion and Future Work

  3. Motivation • Inspect data moving through networks • Enable application level data processing • Secure networks • Safeguard confidential data • Detect and prevent intrusions • Worms, viruses, spam, espionage • Mitigate denial of service attacks • Characterize and analyze network traffic • Operate at multi-gigabit data rates

  4. Transmission Control Protocol • 86% to 90% of all Internet traffic uses TCP • Web, email, file transfer, remote login, secure communications • Provides virtual bit pipe between two end systems • Retransmission services • Data reordering services • Flow control services • Congestion avoidance services

  5. Internet

  6. Internet

  7. Internet

  8. Cost of Internet Attacks

  9. Economic Damage Estimate

  10. Design Requirements • Architecture that is fast • Hardware-based system • High-performance (multi-gigabit networks) • Per-flow context storage & retrieval • Architecture that is scalable • Performance improves with advances in technology • In-line traffic processing model • Implementation using reasonable resources • FPGA implementation can be done in research lab • Framework that is flexible • Integrates with multiple applications • Multi-device coordination of TCP stream processing

  11. Outline • Motivation and Background • Architecture and Related Work • Live Internet Traffic Processing • Conclusion and Future Work

  12. TCP-Processor Architecture

  13. TCP Processing Engine TCP Processing Engine Frame FIFO Checksum Engine Control & State FIFO Input State Machine Output State Machine TCP State Processing Flow Hash Computation State Store Manager

  14. Challenges and Design Choices • Performance • Operate at multi-gigabit data rates • Hardware-based design exploiting pipelining and parallelism • Flow classification • Open addressing hash with limited bucket sizes • Context storage and retrieval • Requires memory read and write for each packet • 64-byte per-flow context - use burst read/write operations • Reassembly of out-of-order packets • Multiple processing modes (guaranteed and passive) • TCP processing • Flow monitoring instead of flow termination

  15. Link Speeds and Packet Rates

  16. Systems with TCP Processors • Load balancing systems • Content (cookie) based request routing • Delayed binding technique • Limited to scanning start of flow • TCP offload engines • Move TCP protocol processing to NIC • Targeting Gigabit NIC market • Intel, NEC, Adaptec, Lucent, and others • SSL Accelerators • Offload encryption/decryption • Protocol translation • Intrusion Detection Systems • Traffic Rates < 1Gbps • Perform content scanning and some stream reassembly

  17. Related Work in TCP Processing • Software-based TCP processing • Ethereal, tcpdump, etc – require post processing • Snort w/TCP option – larger virtual packets • Cluster-based online monitoring system (Mao: WIDM’01) • Bro – rule based processing (Paxson: Computer Networks’99) • STAT/STATL – state based processing (Vingna: DISCEX’00) • Intel – Xeon as packet processor (Regnier: HotI’03) • Hardware-based TCP processing • Georgia Tech – 1 flow/circuit (Necker: FCCM’02) • University of Oslo – 1 flow/ circuit (Li: FPL’03) • Indiana University and Imperial College – Netflow statistics • University of Tokyo – multi-flow stream scanning (Sugawara: FPL’04) • Intel TCP processor – 8k connections, 9Gbps (Xu: HotChips’03) • Network processors • Intel IXP 1200, 2400, 2800, 2850 • Motorola PowerQUICC

  18. Taxonomy of Packet Processors

  19. Multi-Device Coordination • Encodes interface signals • Regenerates waveforms on separate device • Provides extensible format & self describing structure

  20. Place & Route Results • Including Protocol Wrappers & Encoder/Decoder • Target Xilinx Virtex XCV2000E-8 • FPX Platform • Number of BLOCKRAMs • 95 out of 160 (59%) • Number of SLICEs • 7279 out of 19200 (37%) • Maximum clock frequency: 85.565MHz • Maximum data throughput: 2.7 Gbps • Maximum packets per second: 2.9M packets/sec • Min 29 clock cycles per packet (345 ns) • Throughput limited by memory latency

  21. Content Scanning TCP circuit Scan circuit Control Interface Network Traffic

  22. Outline • Motivation and Background • Architecture and Related Work • Live Internet Traffic Processing • Conclusion and Future Work

  23. Washington University Network • 384 Mbps total Internet bandwidth • 300 Mbps Internet • 84 Mbps Internet2 • Approx 19,000 active end systems • Approx 10,000 students • Traffic analyzed for 5 week period • Aug 20th to Sep 24th • Over 1000 charts generated • Selected highlights presented

  24. Washington University Network Internet / Internet2 To TCP Processor

  25. Live Internet Traffic Analysis WUGS-20 Standalone FPX-in-a-Box External Stats Monitor

  26. Data Collection

  27. Current Live Traffic

  28. Collected Statistics Port Statistics TCP Statistics RTR Bypass Packets EGR Client Packets In EGR Bypass Packets In EGR TCP Checksum Update EGR Packets Out Configuration Information SSM New Connections SSM End Connections SSM Reused Connections SSM Active Connections INB Input Words INB Input Packets INB Dropped Packets INB Output Packets ENG TCP Packets ENG SYN Packets ENG FIN Packets ENG RST Packets ENG Zero Length Packets ENG Retransmitted Packets ENG Out-of-Sequence Pkts ENG Bad Checksums RTR TCP Data Bytes RTR Client Packets FTP SSH Telnet SMTP TIM Nameserv Whois Login DNS TFTP Gopher Finger HTTP POP SFTP SQL NNTP NetBIOS SNMP BGP GACP IRC DLS LDAP HTTPS DHCP Lower Upper Protocol Statistics Cells In Cells Dropped Cells Bypass Cells Out Frame Words In Frame Packets In IP Packets Dropped IP Packet Fragments IP Packets In IP Words In IP Packets Bypass IP Words Bypass IP Bad Checksum Scan Statistics String 1 String 2 String 3 String 4

  29. Typical Daily Traffic Pattern Lowest activity Highest activity

  30. IP and TCP Traffic Rates >90% TCP packets

  31. Zero Length TCP Packets 20-40% zero length pkts

  32. Fragmented IP Packets .25% Fragmented

  33. Packet Sequencing 3x-4x more retransmitted

  34. Packet Sequencing (cont) 3%-4% Retransmitted 1% Out of Seq

  35. Worm/Virus Detection • Search for digital signatures • MyDoom (appeared 1/26/04) • Spread via email attachment • Opens back door via ports 3127-3198 • Contains SMTP engine to replicate itself • Contains denial of service attack (25% operational) • At Peak, 1 in 12 emails contained virus • Netsky (appeared 3/1/04) • Spread via email attachment • Scans drives C through Z looking for email addresses • Contains SMTP engine to replicate itself

  36. MyDoom Virus Detection

  37. Netsky Virus Detection

  38. Denial of Service Attack • TCP SYN Attack • 8 minutes in duration • 71,000 TCP pkts/sec avg (34,000 normal) • 40,000 TCP SYN pkts/sec avg (2,000 normal) • IP attack (non TCP traffic) • 3.5 minutes in duration • 91,000 IP pkts/sec peak (36,000 normal) • 57,000 Non-TCP pkts/sec peak (2,000 normal)

  39. Attack Difficult to Detect TCP: 10:25 to 10:34am IP: 10:37 to 10:41am

  40. Both Attacks Visible Non-TCP attack TCP attack

  41. TCP SYN Attack 20x increase in SYN packets

  42. Attack Directed at SSH Port counter saturated True spike at 2.4 M pkts

  43. Non-TCP Attack 29x increase in non-TCP packets

  44. Flow Classification and Attacks • State store contains 1 million records • Record removed after TCP FIN or RST • Stale records are not aged out • 500,000 to 800,000 active records normal • DoS attack can cause flow saturation • Table quickly settles back to normal range

  45. Active State Store Records 400,000 new flows

  46. Outline • Motivation and Background • Architecture and Related Work • Live Internet Traffic Processing • Conclusion and Future Work

  47. Insights • 20%-40% zero length packets • Increase from 18% to 22% (Shalunov: Internet2‘01) • Implies larger amount of 1-way traffic • Optimization skips processing of these packets • 5% out of order packets • Agrees with results from (Jaiswal: Infocom‘03) • Flow classification tables need to be larger • Flow table ½ to ¾ full during normal processing • 1M entry table saturated during attack • Automated response systems required • Short lived attacks difficult to address manually

  48. Contributions • Developed Architecture for TCP-Processor • Hardware-based system • High-performance (multi-gigabit networks) • Per-flow context storage & retrieval • Implemented TCP-Processor in Reprogrammable Hardware • Operates at 85Mhz on Xilinx Virtex 2000E FPGA • Maximum throughput of 2.7 Gbps • Maximum 2.9M packets/sec • Created inter-device protocol TCP applications • Multi-device coordination of TCP stream processing • Interfaces with TCP-Processor • Self-describing/extensible transport protocol • Analyzed live Internet traffic • Insight into Internet traffic profiles • Supported academic and commercial endeavors

  49. Future Work • Packet defragmentation • Flow classification • Packet storage manager • 10Gbps and 40Gbps data processing • Histogram (packet size, packet type, etc) • Event rate detection • Traffic sampling and real-time analysis • Application integration

  50. Advisor & committee John Lockwood (advisor) Chris Gill Ron Loui Ron Indeck Dave Schimmel ARL faculty & staff Jon Turner Patrick Crowley Fred Kuhns John DeHart CSE faculty & staff ARL & FPX students NTS Steve Wiese Global Velocity Matthew Kulig Reuters (formerly Bridge) Scott Parsons Deb Grossman John Leighton Recommendations Scott Parsons Don Bertier Andy Cox Chris Gray Reviewers Tanya Yatzeck James Hartley Family Jerry & Lois (parents) Chris & Kreslyn Nancy, Jeff & Nathan Friends Acknowledgments

More Related