1 / 14

Enhancing Polymorphic Worm Detection through Instruction Distribution Analysis

This paper presents a novel approach to detecting polymorphic worms by analyzing instruction distribution in executable code. Traditionally, Intrusion Detection Systems (IDS) have struggled against sophisticated worms that alter their payloads to evade detection. The proposed method leverages a static analysis technique to distinguish between legitimate and malicious code in real-time by examining the frequency of specific instruction types. This approach aims to improve the accuracy and speed of worm detection without significant overhead, facilitating more effective network security.

truda
Télécharger la présentation

Enhancing Polymorphic Worm Detection through Instruction Distribution Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Polymorphic Worm Detection by Instruction Distribution Kihun Lee HPC Lab., Postech 2nd Joint Workshop between Security Research Labs in Korea and Japan 2006. 2. 20

  2. Contents • Introduction • Background • Polymorphic Worm • Related Works • Polygraph • Using a Control Flow Graph • Problem Definition • Proposal Idea • Conclusions and Future Works

  3. Introduction • Toward defending against Internet worms, NIDSs have been proposed by the security community. • IDS searches inbound traffic for known patterns, or “signature”. • Unfortunately, the worms became more sophisticated! • Substantially changes its payload.

  4. Background Polymorphic Worm (1/2) • IDSs search for similar byte sequence • Author of worm have to prevent this: • ciphering techniques • obfuscating the decryption routine • Can’t find a sufficiently specific sequence. <Figure 1.> Typical polymorphic worm structure

  5. Polymorphic Worm (2/2) Background <Figure 2.> Polymorphic worm cycle

  6. Polygraph Related Works • The system that proposed to defense the polymorphic worms • Idea : use the combination of “short invariant contents” • Assumption : combination of many general contents is sufficiently specific. • Problems : • Even though combine all of them, an outcome can be remain too general. • Decision time is too late. • Token of the signature can be located after a long garbage sequence.

  7. Using a Control Flow Graph Related Works • A complementary approach to reach the same goal with Polygraph • Idea : using structural information of executables • Assumption : at least some parts of a worm contain executable machine code. • Decryptor part of polymorphic worm • Problems : • Because of huge performance overhead, it cannot operate on-line. • generating a graph, coloring the graph • Manufacturing the control flow is not difficult technique.

  8. Problem Definition • Scope of problem: • The worm of which propagation mechanism is using a vulnerability of a server application. • Assumption : • At least some parts of a worm contain executable machine code. • Linear disassemble has a little overhead so that can operate on-line. • Problem definition • Make a decision whether the inbound packet has an executable code or not. • Make a decision whether the executable code is a polymorphic exploit code or a legitimate code.

  9. Motivated Experiment • If disassemble the packet, • Case 1 : executable code tend to • Kinds of instruction : Ü • Number of each instruction : Û • Case 2 : non-executable code tent to • Kinds of instruction : Û • Number of each instruction : Ü • Decoding error (invalid instruction) : Û <Figure 3.> number of each instruction ( sorted by decreasing order)

  10. Threshold Find Executable Code Proposal Idea • Let K = “kinds of instruction” , T = “total number of instructions”, E = “the number of decoding error”. • Calculate the expression : • Non-executable code : tend to very small value. • Executable code : tend to relatively large value. <Figure 4.> Distinguish between executable code and non-executable code

  11. Distinguish Legitimate Code (1/2) Proposal Idea • Use the “verifying instruction” • For example, “call”, “ret”, “int”, etc. • Typically, normal executable code has a lot of “call” instructions. • One “call” instruction per 10~15 instructions. • NOP sled cannot include any “call” instruction. • Decryptor is a very simple routine so that it rarely has a “call” instruction. • Moreover, decryptor can’t know the address of the function of dynamic linking library.

  12. Threshold Distinguish Legitimate Code (2/2) Proposal Idea • Let V = “the number of verifying instruction” • Calculate the expression : • Polymorphic exploit code : is relatively small value. • Legitimate code : is relatively large value. <Figure 4.> Distinguish between exploit code and legitimate code

  13. Conclusions and Future Works • Conclusions • Proposed idea can identify and isolate the polymorphic worm. • It is based on static analysis; so it can runs in real-time. • It can discover the worm traffic by not flow level but packet level examination. • Future Works • Refine the idea. • Investigate more samples to get a generality. • How to extract a signature?

  14. References • J. Newsome, B. Karp, and D. Song. Polygraph: Automatically Generating Signatures for Polymorphic Worms. In IEEE Symposium on Security and Privacy, 2005. • C. Kruegel, E. Kirda, D. Mutz, W. Robertson, and G. Vigna. Polymorphic Worm Detection Using Structural Information of Executables. In RAID 2005. • O. Kolesnikov, and W. Lee. Advanced Polymorphic Worms: Evading IDS by Blending in with Normal Traffic. In 12th ACM conference on Computer and communications security. • P. Akritidis, E.P. Markatos, M. Polychronakis, and K. Anagnostakis. STRIDE: Polymorphic Sled Detection Through Instruction Sequence Analysis. 12th ACM conference on Computer and communications security. • T. DeTristan, T. Ulenspiegel, Y. Malcom, and M. von Underduk. Polymorphic Shellcode Engine Using Spectrum Analysis. http://www.phrack.org/show.php?p=61&a=9 • Etc.

More Related