Designing and Writing Secure Code: Principles and Defenses Against Buffer Overflow

1. Outline • Designing and Writing Secure Code • General principles for architects/managers • Example: sendmail vs qmail (optional in backup slides) • Buffer Overflow Attacks • Defense for Buffer Overflow Attacks

2. General Principles • Compartmentalization • Principle of least privilege • Minimize trust relationships • Defense in depth • Use more than one security mechanism • Secure the weakest link • Fail securely • Promote privacy • Keep it simple • Consult experts • Don’t build what you can easily borrow/steal • Open review is effective and informative Have you applied them in your design / evaluation?

3. Compartmentalization • Divide system into modules • Each module serves a specific purpose • Assign different access rights to different modules • Read/write access to files • Read user or network input • Execute privileged instructions (e.g., Unix root) • Principle of least privilege • Give each module only the rights it needs • Minimize trust relationships • Clients, servers should not trust each other • Both can get hacked • Trusted code should not call untrusted code

4. Defense in Depth • Failure is unavoidable – plan for it • Have a series of defenses • If an error or attack is not caught by one mechanism, it should be caught by another • Examples • Firewall + network intrusion detection • Fail securely • Many, many vulnerabilities are related to error handling, debugging or testing features, error messages • Ensure that you handle errors • Do not expose system internals even in case of errors • Stack traces, internal errors, ... shown to clients • Test if your system fails securely

5. Check security Check security Secure resource with an ACL Check security Application.dll Application.dll Check security Defense in Depth Application.exe [MSDN]

6. Secure the weakest link • Think about possible attacks • How would someone try to attack this? • What would they want to accomplish? • Find weakest link(s) • Crypto library is probably pretty good • Is there a way to work around crypto? • Data stored in encrypted form; where is key stored? • Main point • Do security analysis of the whole system • Spend your time where it matters

7. Promote Privacy • Discard information when no longer needed • No one can attack system to get information • Examples • Don’t keep log of old session keys • Delete firewall logs • Don’t run unnecessary services (fingerd) • Hiding sensitive information is hard • Information in compiled binaries can be found • Insider attacks are common

8. Keep It Simple • Use standard, tested components • Don’t implement your own cryptography • Don’t add unnecessary features • Extra functionality  more ways to attack • Use simple algorithms that are easy to verify • A trick that may save a few instructions may • Make it harder to get the code right • Make it harder to modify and maintain code

9. Don’t reinvent the wheel • Consult experts • Allow public review • Use software, designs that others have used • Examples • Bad use of crypto: 802.11b • Protocols without expert review: early 802.11i • Use standard url parser, crypto library, good random number generator, …

10. Outline • Designing and Writing Secure Code • General principles for architects/managers • Example: sendmail vs qmail (optional in backup slides) • Buffer Overflow Attacks • Defense for Buffer Overflow Attacks

11. Preventing Buffer Overflow Attacks

12. Some unsafe C lib functions strcpy (char *dest, const char *src) strcat (char *dest, const char *src) gets (char *s) scanf ( const char *format, … ) printf (conts char *format, … )

13. Preventing buf overflow attacks • Main problem: • strcpy(), strcat(), sprintf() have no range checking. • Use “safe” versions strncpy(), strncat() very carefully • Defenses: • Type safe languages (Java, ML). Legacy code? • Mark stack as non-execute. • Static source code analysis. • Run time checking: StackGuard, Libsafe, SafeC, (Purify). • Black box testing (e.g. eEye Retina, ISIC ).

14. Marking stack as non-execute • Basic stack exploit can be prevented by marking stack segment as non-executable • Code patches exist for Linux and Solaris. • Problems: • Some apps need executable stack (e.g. LISP interpreters). • Does not block more general overflow exploits: • Overflow on heap: overflow buffer next to func pointer. • Cannot make all the data segment non-executable • More recent UNIX and MS windows emit dynamic code into program data for performance optimizations

15. Static source code analysis • Statically check source to detect buffer overflows. • Several consulting companies. • Several tools exist to automate the review process: • Stanford: Engler, et al. Test trust inconsistency. • @stake.com (l0pht.com): SLINT (designed for UNIX) • Berkeley: Wagner, et al. Test constraint violations. • Find lots of bugs, but not all.

16. Run time checking: StackGuard • Many many run-time checking techniques … • Solution: StackGuard (WireX) • Run time tests for stack integrity. • Enhance the code generator for emitting code to set up and tear down functions • Embeds “canaries” in stack frames and verify their integrity prior to function return. Frame 2 Frame 1 topofstack sfp ret str local canary sfp ret str local canary

17. Canary Types • Random canary:(used in Visual Studio 2003) • Choose random string at program startup. • Insert canary string into every stack frame. • Verify canary before returning from function. • To corrupt random canary, attacker must learn current random string. • Terminator canary:Canary = 0 (null), newline, linefeed, EOF • String functions will not copy beyond terminator. • Hence, attacker cannot use string functions to corrupt stack.

18. StackGuard (Cont.) • StackGuard implemented as a GCC patch. • Program must be recompiled. • Minimal performance effects • Worst case: 8% for Apache.

19. End of Quarter Review • Cryptography • Symmetric encryption case study: DES/AES algorithms • Asymmetric encryption case study: RSA • One-way hash function and message digests: MD5, SHA1, SHA2 • Authentications • Authentication mechanisms: password authentication, challenge-response authentication protocols, biometrics, token-based authentication • Trusted Intermediary • Symmetric crypto: KDC and Kerberos • Asymmetric crypto: CA and certificates in SSL/TLS

20. Thread One: Attacks • Viruses, worms, and botnets (C&C) • Scan for open ports/services • Send exploits for vulnerabilities of the discovered services • Tools: nmap, nessus, and hydra (homework 8) • Web attacks and defense • XSS (CSRF) • SQL injection • DoS attacks and defense • SYN flooding attacks

21. Thread Two: Integrated Defense Cisco Security Agent Cisco IPS Cisco Firewall Cisco NAC • IDS/IPS and monitoring • Host based • Network based • Snort • Firewalls • Stateless/Stateful Packet filters • Application-level Proxy • Other variants • Network Access Control (Cisco guest lecture) • Wireless authentication: WEP vs. WPA CS MARS

22. Emerging Landscape and Topics Upon Requests • Cloud Security • Software Security • Buffer overflow attacks and defense

23. Backup Slides

24. Example: Mail Transport Agents • Sendmail • Complicated system • Source of many vulnerabilities • Qmail • Simpler system designed with security in mind • Gaining popularity Qmail was written by Dan Bernstein, starting 1995 $500 reward for successful attack; no one has collected

25. Simplified Mail Transactions Mail User Agent Mail Transport Agent Mail Transport Agent Mail User Agent Mail Delivery Agent Mail Delivery Agent mbox mbox • Message composed using an MUA • MUA gives message to MTA for delivery • If local, the MTA gives it to the local MDA • If remote, transfer to another MTA

26. Example: Qmail • Compartmentalize • Nine separate modules • If one module compromised, others not • Move separate functions into mutually untrusting programs • Always validate input from other modules

27. THE BIG Qmail PICTURE SMTP from network from local remote mailserver tcpserver / tcp-env / inetd MUA qmail-smtpd qmail-inject forwarded message qmail-queue qmail-system qmail-send qmail-rspawn qmail-lspawn qmail-remote qmail-local mbox / maildir / program delivery remote mailserver to local

28. Structure of qmail qmail-smtpd qmail-inject qmail-queue Other incoming mail Incoming SMTP mail qmail-send qmail-rspawn qmail-lspawn qmail-remote qmail-local

29. Structure of qmail qmail-smtpd qmail-inject qmail-queue • Reads the message and creates an entry in the mail queue • Signals qmail-send qmail-send qmail-rspawn qmail-lspawn qmail-remote qmail-local

30. Structure of qmail qmail-smtpd qmail-inject qmail-queue • qmail-send signals • qmail-lspawn if local • qmail-remote if remote qmail-send qmail-rspawn qmail-lspawn qmail-remote qmail-local

31. Structure of qmail qmail-smtpd qmail-inject qmail-queue qmail-send qmail-lspawn • qmail-lspawn • Spawns qmail-local • qmail-local runs with ID of user receiving local mail qmail-local

32. Structure of qmail qmail-smtpd qmail-inject qmail-queue qmail-send qmail-lspawn • qmail-local • Handles alias expansion • Delivers local mail • Calls qmail-queue if needed qmail-local

33. Structure of qmail qmail-smtpd qmail-inject qmail-queue qmail-send qmail-rspawn • qmail-remote • Delivers message to remote MTA qmail-remote

34. Least Privilege in Qmail • Each module uses least privileges necessary • Each runs under different non-privileged UID in four groups: qmaild, qmailr, qmails, and qmailq • Except one as root • Only one run as root: qmail-lspawn (except qmail-start) • Spawns the local delivery program under the UID and GID of the user being delivered to • Always changes effective uid to recipient before running user-specified program

35. Principles, sendmail vs qmail • Do as little as possible in setuid programs • Of 20 recent sendmail security holes, 11 worked only because the entire sendmail system is setuid • Only qmail-queue is setuid • Its only function is add a new message to the queue • Do as little as possible as root • The entire sendmail system runs as root • Operating system protection has no effect • Only qmail-start and qmail-lspawn run as root.

36. Least privilege qmail-smtpd qmail-inject qmail-queue setuid qmail-send qmail-rspawn qmail-lspawn root qmail-remote qmail-local

37. Keep it simple • Parsing • Limited parsing of strings • Minimizes risk of security holes from configuration errors • Modules do parsing are isolated and run with user privilege • Libraries • Avoid standard C library, stdio • Small code is more secure • Plug in interposing modules rather than complicating the core code

38. Security by Obscurity … Is NOT Secure !!! • Information in compiled binaries can be found • Reverse engineering • Disassembler: machine code to assembly • Discomplier: machine code to high-level language • Insider attacks are common • Firewalls do not protect against inside attacks • Assume an attacker knows everything you know • Why? • If attacker has 1-in-a-million chance, and there are a million attackers, you are out of luck

39. Secure Programming Techniques: An Abstract View of Program Program Component • Avoid buffer overflow • Secure software design • Language-specific problems • Application-specific issues Respond judiciously Validate input Call other code carefully

40. Secure Programming • Validate all your inputs • Command line inputs, environment variables, CGI inputs, … • Don't just reject “bad” input, define “good” and reject all else • Avoid buffer overflow • Carefully call out to other resources • Check all system calls and return values

41. Comparison

42. Comparison with other MTAs