SCP: A System Call Protector against Buffer Overflow Attacks

1. SCP: A System Call Protector against Buffer Overflow Attacks dove 邱秉誠

2. Outline • Introduction • Attacking Method • Related Work • SCP System Design • Experimental Result • Conclusion • Future Work

3. Introduction (1) • Buffer Overflow Attack • Easily launched • Huge amount of targets • Strongly damage • The most dangerous threat in the internet

4. Introduction (2) • Many countermeasures were publishedBut also were defeated • Developing an efficient and effective approach becomes a critical and emergent issue

5. Attacking Method (1) • Stack Overflow Attack • Overflow the return address to launch injected code (shell code)

6. Attacking Method (2) • Return-into-libc Attack Overwrite the return address to execute system() function call High Address “/bin/sh” Pointer to system()’s arg Fake RA ESP system() EIP (RA) EBP buffer Low Address String format:

7. Attacking Method (3) • Heap Overflow Attack • Similar to stack overflow, but on the heap area

8. Attacking Method (4) • Scanning Code Attack • After overflow, scan the process image to find patterns and then jump into it • Usually used to bypass some protections

9. Related Work (1) • Some Countermeasures • StackGuard / StackShield • Address Obfuscation • Exec Shield • Binary Obfuscation • PointGuard™ • Instruction Set Randomization • RAD

10. High Address High Address return address return address canary word … Buffer … … Buffer Saved return address Low Address Related Work (2) • StackGuard-- add canary word before return address • StackShield--copy return address to confirm when return • Bypassing StackGuard and StackShield Low Address

11. Related Work (3) • Address Obfuscation • PaX ASLR project - Randomize the base address of memory regions -- Randomize the base address of stack/heap -- Randomize the starting address of dynamic-linked libraries -- Randomize the locations of routines and static data • ASLP • Internal fragmentation problem • Derandomization Attack

12. Related Work (4) • Bound Checking • Bound Checking for C • Require source code / recompile • Runtime overhead are huge - 4x / 5x when best case - 10x general case - 100x worst case

13. Related Work (5) • Exec Shield • Data/Stack section non-executable • Code section non-writable • Compatibility problem - ELF file format -- Add PT_GNU_STACK and PT_GNU_HEAP - Nested function - Recompile / porting - sigreturn() system call • Return into libc attack can be launched

14. Related Work (6) • Binary Obfuscation • Change ELF layout (add new section) • Add new system call to notify the return address • Insert junk code in binary object • Attacker may use the new system call • Return into libc attack / scanning code attack can be launched

15. Related Work (7) • PointGuard™ • Encrypt pointer, decrypt when reference object

16. Related Work (8) • Instruction Set Randomization • Hardware solution, encrypt / decrypt CPU instructions • porting binaries

17. Related Work (9) • RAD • Return Address Defender • Compiler solution • Push return address to RAR when prologue • Pop return address from RAR when eprologue • Need recompile

18. SCP System Design (1) • Principles • Prevent attacker from executing int 80 offered by attacker • Prevent attacker from executing int 80 existed in system • Goal • Low overhead • Efficient to protect • Do not require source code • Compatibility • Use less system resource • Easy to maintain

19. SCP System Design (2) • Assumptions • Malicious code have to use system call to damage system • vulnerable program is dynamic linked

20. SCP System Design (3) • System overview • protect int 0x80’s return address • Use secure enter kernel instead of all sysenter (int $0x80) movl sys#, %eax movl arg1, %ebx … int $0x80

21. SCP System Design (4) • The system call path

22. SCP System Design (5) • The system call path High Address ( kernel ) sys_open() Kernel Space 5. return 4. trap into kernel ( libc wrapper routine ) __libc_open() User Space 3. call wrapper routine ( GOT entry ) Address of __libc_open 2. lookup GOT 6. return ( PLT ) jmp *GOT[__libc_open] 1. go to PLT ( user program ) open(); Low Address

23. SCP System Design (6) • Pseudo Code (a) Secure Enter Kernel (b) Trap Code save_all_registers; page = 0; size = 0; if ( page == 0 ) { page = mmap2(); size = copy_trap_code(page); notify_kernel(size+6); } restore_all_registers; call page; restore_sys#_in_eax; int $0x80; (sysenter) return_to_libc; machine code: \x8B\x44\x24\x04 \xCD\x80 \x83\xC4\x08 \xC3

24. SCP System Design (7) • New system call path (with SCP system) High Address ( kernel ) sys_open() 6. return Kernel Space sys_read() 5. trap into kernel ( trap page ) int $0x80 ( libc wrapper routine ) __libc_open() 4. call trap page User Space __libc_read() 7. return 3. call wrapper routine ( GOT entry ) Address of __libc_open Address of __libc_read 2. lookup GOT 8. return ( PLT ) jmp *GOT[__libc_open] jmp *GOT[__libc_read] 1. go to PLT ( user program ) open(); read(); Low Address

25. SCP System Design (8) • Example with SCP system (lazy binding) Glibc: printf() here 4. Call printf() 6. Normal system call 5. Notify kernel the RA 3. Load libc.so.6 kernel Loader (ld-linux.so.2) 1. Notify kernel the RA Inject code 2. Loading program program

26. SCP System Design (9) • Example with SCP system (lazy binding) Glibc: printf() here 4. Call printf() 6. Normal system call 5. Notify kernel the RA 3. Load libc.so.6 kernel Loader (ld-linux.so.2) 1. Notify kernel the RA Inject code 2. Loading program program

27. SCP System Design (10) • Modify kernel • Add new system call notify_kernel() - It can only be called 2 times per process • Add new structures in task_struct to restore addresses - loader_return_address - syscall_return_address • do_fork() - Addresses copied from parent process • sys_execve() - loader_return_address = 0 - syscall_return_address = 0

28. SCP System Design (11) • Introduce fake pages

29. (user program) system_call (libc) wrapper routine (heap) int $0x80 (PLT) (GOT) SCP System Design (12) • Attack analysis • Attacker can launch scanning code attack to trace real int $0x80 • Possible solution: - Non-readable but executable code segment - … future work

30. SCP System Design (13) • SCP system analysis • Allow executable stack- General debugger support - Nested function - Do not require ELF modified - Do not require to recompile • Allow execute programs without ASLR - No internal fragment problem - Process crashes are decided by process owners - Easy to maintain

31. Experimental Result (1) • Efficiency test • Buffer overflow attack with injected code

32. Experimental Result (2) • Efficiency test • Calling notify_kernel test

33. Experimental Result (3) • Micro test • 10,000,000 times per system call

34. Experimental Result (4) • Original libc & kernel execve("./micro-test.open", ["./micro-test.open"], [/* 29 vars */]) = 0 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 49.40 828.885786 41 20000000 gettimeofday 25.81 432.1044308 43 10000003 1 open 24.59 411.1630811 41 10000002 close 0.04 0.700248 700248 1 brk 0.04 0.700211 700211 1 mprotect 0.04 0.700202 700202 1 read 0.02 0.402909 201455 2 write 0.02 0.400418 200209 2 munmap 0.01 0.201221 33537 6 old_mmap 0.01 0.100622 33541 3 fstat64 ------ ----------- ----------- --------- --------- ---------------- 100.00 1677.766736 40000021 1 total

35. avg timediff = 0.0000073216265842318534575253186069687672 sec = 7.32162658 usec Command exited with non-zero status 81 Command being timed: "./micro-test.open" User time (seconds): 0.69 System time (seconds): 122.31 Percent of CPU this job got: 100% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:03.00 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 82 Minor (reclaiming a frame) page faults: 7 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 81

36. Experimental Result (5) • Modified libc & kernel execve("./micro-test.open", ["./micro-test.open"], [/* 29 vars */]) = 0 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 47.44 821.1515639 41 20000000 gettimeofday 28.54 494.834607 49 10000003 1 open 23.82 412.936479 41 10000002 close 0.04 0.700382 700382 1 read 0.04 0.700380 700380 1 brk 0.02 0.403299 201650 2 write 0.02 0.400813 200407 2 mmap2 0.02 0.400782 200391 2 munmap 0.02 0.400761 200381 2 utimes 0.01 0.202308 33718 6 old_mmap 0.01 0.101257 33752 3 mprotect 0.01 0.101186 33729 3 fstat64 ------ ----------- ----------- --------- --------- ---------------- 100.00 1733.697893 40000027 1 total

37. avg timediff = 0.0000123534624044317750067014868853298992 sec = 12.35346240 usec Command exited with non-zero status 82 Command being timed: "./micro-test.open" User time (seconds): 0.68 System time (seconds): 173.26 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 2:53.94 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 0 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 104 Minor (reclaiming a frame) page faults: 13 Voluntary context switches: 0 Involuntary context switches: 0 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 82

38. Experimental Result (5) • Macro test • 100,000 times per command

39. Conclusion • We propose a new method to protect system calls by registering valid int 80 on premise • Without recompile • Allow executable stack • Use ASLR optionally • Compatible with other protections

40. Future Work • More secure • Implement “executable but non-readable” region in segment section on i386 • The NX Bit chip • AMD 64 CPU

41. Thanks • Q & A