Packet Vaccine: Black-box Exploit Detection and Signature Generation

Packet Vaccine:Black-box Exploit Detection and Signature Generation Su Yong Kim

Contents • Stack Overflow Example • Packet Vaccine • Evaluation • Limitation • Conclusion

Vulnerable Program void foo (char *bar) { char c[12]; strcpy(c, bar); // no bounds checking... } int main (intargc, char **argv) { foo(argv[1]); return 1; }

Stack Layout • Just After calling foo()

Stack Layout Example • Calling foo(“hello”)

Stack Overflow Example I • Calling foo(“AA……”) • Direct Jump

Stack Overflow Example II • Indirect Jump • Calling foo(“AA……”) esp Code Section Ox 7e8ecf49 FF D4 (jmpesp) \x49 \xcf \x8e \x7e

Stack Overflow Example III • Return into Library • Calling foo(“AA……”) Ox 7C8623AD Kernel32.dll Ox 7C8623AD WinExec API \xAD \x23 \x86 \x7C

Main Idea • Detect the packet with any address-like string • Test if an anomalous packet is malicious or not • Scrambling the address-like string of the packet • Send the packet into the target server • If any exception will occur on the target server, drop the original packet and generate a signature • If no exception, send the original packet into the target server

Detecting Anomalous Packets • Key Idea • Check every 4-byte existing-sequence(32 bit system) in a packet’s application payload • Detect a jump address to redirect the control flow of a vulnerable program • Jump Address • Stack & Heap address range • Address range of the global libraries such as msvcrt.dll or libc.so • system(), execve() • Empirical study on Linux • 0xbfff0000 to 0xbfffffff for the stack • 0x08040000 to 0x08ffffff for the heap

Determining Stack & Heap Address range • Monitor stack and heap usage of the protected program • Calculate stack heap address range • From • Stack base address – α * (typical stack maximum usage), α > 1 • To • Stack base address • Calculate stack heap address range • From • Heap base address • To • Heap base address + α * (typical heap maximum usage), α > 1

Making Packet Vaccine • Packet Vaccine • Weakened exploit packet with important elements scrambled • Issues • Preserving the exploit semantics • Control flow should not be changed • Preventing & Detecting malicious behavior by the exploit • Scrambling some fields of the exploit

Preserving the exploit semantics Does User’s Input contains “GET”? Does User’s Input contains “/default.ida”? Call strcpy

Preventing & Detecting Malicious Behavior • After scrambling • Exploit fails • Exception happens \x41

Detecting Exploit • If an exception occurs on protected program • Correlate the exception with one of the byte sequences being scrambled • Value of EIP or CR2 regster == byte sequence • Validate the correlation • Randomize all bytes of byte sequence • Check whether the exception happens again

Generating Signature(1/2) • Application-independent Signature Generation • Generates packet vaccines randomizing each byte except scrambled jump address • Test them in the vulnerable application • If no exception, record the randomized byte as a signature token • Repeat all bytes except scrambled jump address • Signature : Signature Tokens + Target Address Set • Good Performance : Small size of exploit, Parallel testing and Block-searching technique

Generating Signature(2/2) • Using Protocol Information • Identify the application field that includes the jump address • Estimate that field’s length using the number of the bytes prior to the address • Iteratively alters the field size to generate new vaccines • If a new vaccine makes the exception disappear, increate the size • Otherwize, shrink the size • Signature form (application, command, field.name, max.field.size)

Signature Quality Evaluation • Comparison Target • Brumley’s approach to generate a signature on the basis of static analysis of a vulnerable program’s binary code • Comparison Measure • Source code of vulnerable application

Quality of the Token-Sequence Signature • Vulnerable Program • BIND 8.2.2 • Signature Result

Quality of the Application-level Signature • Vulnerable Program • ATP-httpd • Brumley’s Signature • Command : GET or HEAD • Max.field.size : 812 bytes • Performance : more than a second • Packet Vaccine • Command : GET • Max.field.size : 703 bytes • Performance : 0.274 seconds

In Summary • Brumley’s approach is more accurate than Packet Vaccine • Packet Vaccine is nearly as accurate as Brumley’s approach when Packet Vaccine can use multiple exploits • Brumley’s approach cannot be used in obfuscated binaries, while Packet Vaccine can • Packet Vaccine is significantly faster

Performance Evaluation • Experiments Environment • Protected Program • Apache 2.0.55 on Linux • Performance Tester • ApacheBench 2.0.41-dev • Architecture

Server overheads • D0, D1 : On different hosts • S0, S1 : On the Same host Apache Apache, proxy Apache, proxy, packet vaccine Apache, proxy Apache, proxy, packet vaccine

Client-side Delay • Local Round Trip Delay : 300μs • Remote Round Trip Delay : 75ms

Limitation by Author • False negative in exploit detection • If packet vaccine destroy the exploit semantics • Especially for binary protocol • Difficulty to applyon packets with encrypted payload or checksums • Application-level proxy is needed • Less expressive signature • Description for exploit condition is impossible

Limitation by Presenter • Alphanumeric return address can be used to fool Packet Vaccine • Exceptions are more common than exploitable vulnerabilities • Packet Vaccine is so application-dependent • Heap & Stack size should be calculated • Whenever the application is updated • Syntax tokens should be gathered • Decoding mechanism is not simple • It is difficult to protect single-threaded application by using Packet Vaccine • Test Server is needed • Parallel vaccine testing is impossible

Conclusion • Black-box exploit detection • Effective signature generation • Low false positive by using host information • Low overhead and easy deployment • It is needed to install only a lightweight collector to gather forensic data from an exception on the host

Q&A • Thanks for Listening!

Packet Vaccine: Black-box Exploit Detection and Signature Generation