Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms

1. Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li1, Lanjia Wang2, Yan Chen1and Judy Fu3 1 Lab for Internet and Security Technology (LIST), Northwestern Univ. 2 Tsinghua University, China 3 Motorola Labs, USA

2. The Spread of Sapphire/Slammer Worms

3. 1010101 10111101 11111100 00010111 Limitations of Content Based Signature Signature: 10.*01 Traffic Filtering Internet Our network X X Polymorphism! Polymorphic worm might not have exactly content based signature

4. Vulnerability Signature Work for polymorphic worms Work for all the worms which target the same vulnerability Vulnerability signature trafficfiltering Internet X X Our network X X Vulnerability

5. Network Based Detection • At the early stage of the worm, only limited worm samples. • Host based sensors can only cover limited IP space, which might have scalability issues. Thus they might not be able to detect the worm in its early stage Internet Gateway routers Our network Host based detection

6. Design Space and Related Work Network Based Host Based • Most host approaches depend on lots of host information, such as source/binary code of the vulnerable program, vulnerability condition, execution traces, etc. Exploit Based Vulnerability Based

7. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Discussions and Conclusions 7

8. Key Ideas • At least 75% vulnerabilities are due to buffer overflow • Some protocol fields might map to the vulnerable buffer to trigger the vulnerability • The length of some protocol field have to longer than the buffer length • Intrinsic to buffer overflow vulnerability and hard to evade • However, there could be thousands of fields to select the optimal field set is hard

9. Framework • Sniff network traffic from network gateways • Filter out known worms • Existing flow classifiers • Separate traffic into a suspicious traffic pool and a normal traffic pool • E.g. port scan detector, honeynets • LESG Signature Generator

10. LESG Signature Generator

11. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Discussions and Conclusions 11

12. Field Hierarchies DNS PDU

13. Length-based Signature Definition Signature is signature length for field Matching: for flow • if , flow X is labeled as a worm flow Signature Set • worm flows: match at least one signature Ground truth signature is the vulnerable buffer length 2014/9/22 13

14. Problem Formulation Coverage in the suspicious pool is bounded by 1- Suspicious pool LESG Signature Normal pool Coverage bound 1- Minimize the false positives in the normal pool With noise NP-Hard!

15. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Discussions and Conclusions 15

16. Stage I and II COV=1%FP=0.1% Trade off Score function Score(COV,FP) Stage I: Field Filtering Stage II: Length Optimization 16

17. Stage III • Find the optimal set of fields as the signature approximately • Separate the fields to two sets, FP=0 and FP>0 • Opportunistic step (FP=0) • Attack Resilience step (FP>0) • The similar greedy algorithm for each step • Every time find the field with maximum residual coverage and the coverage is no less than a threshold. 17

18. Attack Resilience Bounds High Ground Truth Signature b0 Know the vulnerable field b1 Multiple field Optimal LESG Signature • With different assumptions on b0 and whether deliberated noise injection (DNI) exists, get bound b1 • DNI: Theorem2 and 3 • No DNI: Theorem4 and 5 • With 90% noise in the suspicious pool, we can get the FN<10% and FP<1.8% • Resilient to most proposed attacks Accuracy Low 18

19. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Discussions and Conclusions 19

20. Methodology • Protocol parsing with Bro and BINPAC • Worm workload • Eight polymorphic worms created based on real world vulnerabilities • DNS, SNMP, FTP, SMTP • Normal traffic data • 27GB from a university gateway and 123GB email log. • Experiment Settings 20

21. Results • Single/Multiple worms with noise • Noise ratio: 0~80% • False negative: 0~1% (mostly 0) • False positive: 0~0.01% (mostly 0) • Speed and memory consumption • For DNS, parsing 58 secs, LESG 18 secs for (500,320K) • Pool size requirement • 10 or 20 is enough 21

22. Results – Attack Resilience • The worm not only spread worms but also spread worse case faked noise to mislead the signature generation • DNS Lion worm, noise ratio: 8%~92%, suspicious pool size 200 22

23. Conclusions A novel network-based automated worm signature generation approach • Work for zero day polymorphic worms with unknown vulnerabilities • Vulnerability based and Network based • Length-based signatures for buffer overflow worms • Provable attack resilience • Fast and accurate through experiments 2014/9/22 23

24. Backup Slides

25. Discussionsof Practical Issues Speed of signature matching • Major over head: protocol parsing • Software (Bro with Binpac): 50~200Mbps • Optimized Binpac: 600Mbps • Hardware: 3Gbps Relationship between fields and buffers • Mostly direct mapping between fields • Analyzed 19 vulnerabilities, 1 exception 2014/9/22 25

26. LEngth-based Signature Generator (LESG) Thwart zero-day polymorphic worms Target buffer overflow worms Only use network level info Attack resilient LESG Network-based Noise tolerant Can detect zero-day worm in real-time Vulnerability-based 75% of Vulnerabilities based on buffer overflow Efficient signature matching