Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms

1. Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li1, Lanjia Wang2, Yan Chen1and Judy Fu3 1 Lab for Internet and Security Technology (LIST), Northwestern Univ. 2 Tsinghua University, China 3 Motorola Labs, USA

2. LESG (LEngth-based Signature Generation) Based on the observation that buffer overflow is one of the most common vulnerability types exploited remotely and certain protocol fields might map to the vulnerable buffer. Authors propose a three-step algorithm to generate the protocol field length signatures with analytical attack resilience bound.

3. Outline • Motivation and Related Work • Design of LESG • Problem Statement • Three Stage Algorithm • Attack Resilience Analysis • Evaluation • Conclusions

4. Desired Requirements for Polymorphic Worm Signature Generation[14] • Network-based signature generation • Worms spread in exponential speed, to detect them in their early stage is very crucial • At their early stage there are limited worm samples. A host is unlikely to see the early worm packets. • The high speed network router may see more worm samples. • Signature generation should be high speed to keep up with the network speed!

5. Desired Requirements for Polymorphic Worm Signature Generation[14] • Noise tolerant • Most network flow classifiers suffer false positives. • Even host based approaches can be injected with noise. • Attack resilience • Attackers always try to evade the detection systems • Efficient signature matching for high-speed links

6. Design Space and Related Work Network Based Host Based • Existing vulnerability-based signature generation schemes are host-based and cannot work at the network router/gateway level. Exploit Based Vulnerability Based

7. Signature Generation Classess • Two Classes • Vulnerability-based: inherent to the vulnerability that the worm tries to exploit • Unique, and hard to evade • Exploit-based: capture certain characteristics of a specific worm implementation • Less acurate and can be evaded

8. Exploit-based Schemes • Finds invariant substrings of exploit flow • Polygraph[15], Hamsa[14] • Finds symbolic similarity by using full-system symbolic execution on every machine code • DACODA[18] • Finds structural similarities between different worm binary codes • CFG (Control Flow Graph) [24]

9. Vulnerability-based Schemes • Uses the properties of vulnerable program • A vulnerability signature matches all exploits of a given vulnerability

10. Outline • Motivation and Related Work • Design of LESG • Problem Statement • Three Stage Algorithm • Attack Resilience Analysis • Evaluation • Conclusions

11. Basic Ideas • At least 75% vulnerabilities are due to buffer overflow • Intrinsic to buffer overflow vulnerability and hard to evade • However, there could be thousands of fields to select the optimal field set is hard Overflow! Protocol message Vulnerable buffer

12. Deployment of LESG First, sniff traffic from networks and classify the traffic as different application level protocols. Next, we filter out known worms and then further separate the traffic into a suspicious traffic pool and a normal traffic reservoir.

13. Framework

14. LESG Signature Generator

15. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

16. Field Hierarchies DNS PDU • Each of the application sessions (flows) usually contains one or more Protocol Data Units (PDUs) • A PDU is a sequence of bytes and can be dissected into multiple fields.

17. Problem Formulation Worms which are not covered in the suspicious pool are at most  Suspicious pool LESG Signature Normal pool  Minimize the false positives in the normal pool With noise NP-Hard!

18. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

19. Three Stages • Step 1: Field Filtering • Select possible signature field candidates. • Step 2: Signature Length Optimization • Optimize the signature lengths for each eld. • Step 3: Signature Pruning • Find the optimal subset of candidate signatures with low false positives and false negatives.

20. Stages I and II Trade off between specificity and sensitivityScore function Score(COV,FP) COV≥1%FP≤0.1% Stage I: Field Filtering Stage II: Length Optimization

21. Stage I Inputs: FP0 - false positives COV0 - detection coverage. M – suspicious traffic pool |M| - number of suspicious flows in M N – normal traffic pool |N| - number of normal flows in N S – signature set A signature is a pair Sj = (fj ; lj), where fj is the signature field ID, and lj is the corresponding signature length for field fj . The total running time Since |M| is usually far smaller than |N|, the overall time cost is

22. Stage II • Optimize the length value of each candidate signature to nd the best tradeoff between the coverage and false positives. • If the length signature selected is too long, there will be less coverage of malicious worm flows. • If the length selected is too short, there will be a lot of false positives. • Aims to maximize

23. Stage II

24. Stage III • Find the optimal set of fields as the signature with high coverage and low false positive • Separate the fields to two sets, FP=0 and FP>0 • Opportunistic step (FP=0) • Attack Resilience step (FP>0)

25. Stage III

26. Attack Resilience Bounds Depend on whether deliberated noise injection (DNI) exists, we get different bounds. With 50% noise in the suspicious pool, we can get the worse case bound FN<2% and FP<1% In practice, the DNI attack can only achieve FP<0.2% Resilient to most proposed attacks (proposed in other papers)

27. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

28. Methodology • Protocol parsing with Bro and BINPAC (IMC2006) • Worm workload • Eight polymorphic worms created based on real world vulnerabilities including CodeRed II and Lion worms. • DNS, SNMP, FTP, SMTP • Normal traffic data • 27GB from a university gateway and 123GB email log

29. Results • Single/Multiple worms with noise • Noise ratio: 0~80% • False negative: 0~1% (mostly 0) • False positive: 0~0.01% (mostly 0) • Pool size requirement • 10 or 20 flows are enough even with 20% noises • Speed results • With 500 samples in suspicious pool and 320K samples in normal pool, For DNS, parsing 58 secs, LESG 18 secs

30. The range of the signatures we generated and their accuracy.

32. Conclusions • A novel network-based automated worm signature generation approach • Work for zero day polymorphic worms with unknown vulnerabilities • First work which is both Vulnerability based and Network based using length signature for buffer overflow vulnerabilities • Provable attack resilience • Fast and accurate through experiments