DICOTS and StackGuard: Two current approaches to tolerating malicious code

1. DICOTS and StackGuard: Two current approaches to tolerating maliciouscode Carl Landwehr Mitretek Systems, Inc. 7525 Colshire Dr. McLean, VA 22102 Carl.Landwehr@mitretek.org (703)610-1576

2. DICOTS:Data Integrity for COTS Databases • Research project at Naval Research Lab • Based on Integrity Cluster concept (J. McDermott) • Developed and prototyped by B. Montrose and L. Day

3. Malicious code embedded in COTS products may compromise the integrity of this data. Easter Eggs 1 Internet Explorer Excel 97 Select About from help menu Open a New Sheet Ctrl-drag “e” graphic onto Earth Press the F5 key 2 Type X97:L97 Release Ctrl Key Ctrl-drag “e” again to push text out of the way Press Enter then Tab Hold Ctrl-Shift while you click on the Chart Wizard Button in the toolbar Press “unlock” button Return “e” to Earth What is the Threat?

4. Technical Approach Ammunition Sybase DB 9mm Cart. 4 Hellfire Missile 8 Scud Missile 7 Ammunition Oracle DB DICOTS Majority Vote Mismatch in Informix DB Ammunition Informix DB Diverse & Redundant Databases with a Trustworthy Voter

5. Supporting Research Integrity Cluster: diverse, redundant data-storing components accessed and controlled exclusively by an Integrity Controller. Integrity Cluster Criticisms The Integrity Controller replicates transactions to all of its attached data storages in a one-copy serializable fashion. Data integrity is verified by comparing replicas. Maintaining at least three redundant replicas at one site is impractical. Remedy: DICOTS can be made to work with existing distributed databases. Performance cost (from serialization) is intuitively high. Remedy: measure the actual performance, assess, seek speed-ups Integrity Clusters

6. Technical Approach Ammunition Sybase DB Ammunition Oracle DB DICOTS Ammunition Informix DB Consistency Among Replicas

7. Technical Solution Open Database Connectivity Sybase ODBC Driver Sybase DB COTS ODBC Software Client Host Vendor provides ODBC driver designed to run directly on DBMS Practical implementation of DICOTS requires network between client and database server Sybase Host

8. Custom ODBC Driver Custom ODBC Driver Custom ODBC Driver Client Host Client Host Client Host ODBC Proxy Interface ODBC Proxy Interface ODBC Proxy Interface COTS ODBC Software Oracle ODBC Driver Sybase ODBC Driver Informix ODBC Driver COTS ODBC Software COTS ODBC Software Oracle DB Sybase DB DICOTS Informix DB Oracle Host Informix Host Sybase Host DICOTS Operation • DICOTS implements custom ODBC driver as DLL • Application generates std database calls; DLL packages data, initiates TCP/IP connection, and transfers to proxy • Proxy polls incoming links round robin • When transaction found, it accepts and initiates it at all three back ends • Proxy waits for all backends to finish • If data returned, proxy votes it and reports failure if appropriate (note format issues) • Then accept another transaction

9. DICOTS preliminary results • Feasible to construct the architecture • Partial set of ODBC API implemented • Some compatibility issues encountered • Benchmarking in progress • Initial evidence: retrieval times considerably longer than update • Current implementation represents baseline, near worst case

10. StackGuard:Tolerating Stack Smashing Attacks • Stack Smashing Attacks are Legion • Vulnerabilities attributable to buffer overflow/stack smashing attacks in CERT advisories since 1997: • 1997: 14 of 28 • 1998: 7 of 12 • 1999: 5 of 12 (to Oct 1) Effective countermeasure to stack smashing could eliminate many actual attacks

11. Execution point Program Text [write protected] Data (static) Stack [read/write OK] Problem:Buffer Overflow Attack Normal Stack Low address • Attacker feeds a big string to an input routine that does not do bounds checking • String over-writes return address • String injects code • Function return jumps to injected code Buffer Stack frame … Other local varioable ... Return Address Calling stack frame High address

12. Program Text [write protected] Data (static) Stack [read/write OK] After a Successful Attack Smashed Stack Low address • Return address points into buffer • Buffer contains attack code • Executing return invokes attack code Attack code Stack frame … Other variable ... Return Address High address

13. Problems for the attacker • Hacker must find: program that runs with system privileges (e.g., root privilege) and fails to check the length of its input • Hacker needs to know: • Address of the start of the buffer within the stack, to set up return address properly • Distance between start of buffer and the return address in the stack, to write a string long enough to rewrite the return address • The actual attack code (binary instructions) to create a new process [exec(“sh”)] • Hacker must be able to enter entire attack string as single string input (typically, no embedded nulls) • May have to specialize machine instruction to avoid nulls in attack code

14. Attacker tricks Low address • By starting the attack code with NOPs, the rewritten return address only need be approximate • By writing the guessed return address several times, the guessed displacement from buffer start to return address need only be approximate NOP ... NOP Attack Code Return Address Return Address Return Address High address

15. StackGuard Solution: Detect Stack Smashing As it HappensCrispin Cowan, Calton Pu, Oregon Grad Inst. • StackGuard is a compiler that emits robust bug tolerant code • Stack smashing vulnerabilities are not repaired, nor are the attacks prevented • Instead, program detects attack in progress • Stack smash corrupts running function’s activation record • StackGuard code generator detects corruption before return, preventing attacker from getting control

16. StackGuard 1:Defend Programs with a Stack Integrity Check Normal Stack Low address • Stack smash goes through • Attack code injected • Return address altered • But Stack smash also smashes the Canary • Function checks for Canary before returning • If Canary smashed, program halts instead of yielding control to the attacker Buffer Stack frame Other local variables ... Canary Return Address Calling stack frame High address

17. Canary Integrity • Problem: • attacker customizes attack string so that string over-write replaces proper canary word • Original solution: random canaries • Choose random canary values at exec() time • Requires lookup on each function call and return • Slow • Can’t be used with shared libraries

18. Faster, Better Canary Integrity: StackGuard 1.1 • New solution: “terminator” canary • canary word is Null, CR, LF, and -1 • I.e. the common termination symbols for C string functions • Hard for attacker to embed a termination symbol in the attack string • Improvement: static canary lookup • Faster • Can be used with shared libraries

19. What this solution requires • Recompile vulnerable programs with StackGuard compiler • Changes call sequence to add canary to stack • Changes return sequence to check canary before returning • If canary intact, return normally • If canary not intact, abort the process • Programs not recompiled by Stackguard are not protected, but not affected (no compatibility issue)

20. How good is the protection? • Possible to rewrite return address randomly in some situations, but much more difficult; virtually impossible for this kind of attack • Overhead for StackGuard 1.1 (fixed canary) is barely measurable (time and space)

21. References McDermott, J. and Froscher, J. “Practical Defenses Against Storage Jamming,” in Proceedings of the 20th National Information Systems Security Conference, Baltimore, MD, October 1997, pp. 162-173 McDermott, J. Integrity Clusters. Submitted for publication, Journal of Computer Security, 1999 Aleph One. Smashing Stacks for Fun and Profit. Phrack V. 7, issue. 49. Http://www.reality.sgi.com/nate/machines/security/P49-14-Aleph-One Cowan, Crispin, Calton Pu, et. Al. StackGuard Automatic Adaptive Detection and Prevention of Buffer Overflow Attacks. Proceedings 7th USENIX Conf. Available at http://www.cse.ogi.edu/DISC/projects/immunix/StackGuard/