Testing for Software Security

1. Testing for Software Security ECEN5053 Software Engineering of Distributed Systems University of Colorado, Boulder Testing for Software Security, Hebert Thompson, James Whittaker, Dr. Dobb’s Journal, November, 2002, pp. 24-34

2. When is a security bug not like a bug? • Traditional non-security bugs -- often defined as a violation of a specification. • Security bugs -- additional behavior, not originally intended • Meanwhile, it is doing what it is supposed to do • Traditional techniques not good at finding • Even in inspections, tend to look for • missing behavior • incorrect behavior • Neglect to look for ... undesirable side-effects Testing for SW Security, ECEN5053, University of Colorado, Boulder

3. Intended vs. Implemented Behavior Actual Software Functionality Traditional faults Unintended, undocumented, unknown functionality Intended Functionality Testing for SW Security, ECEN5053, University of Colorado, Boulder

4. Traditional faults • Incorrect • Supposed to do A but did B instead • Missing • Supposed to do A and B but did only A. Testing for SW Security, ECEN5053, University of Colorado, Boulder

5. Security Bugs • Side effects • Supposed to do A, and it did. • In the course of doing A, it also did B • Monitoring for side effects and their impact on security can be challenging • Side effects can be subtle and hidden • Examples: file writes, registry entries, extra network packets with unencrypted data Testing for SW Security, ECEN5053, University of Colorado, Boulder

6. Tools • Commercially available tools • Mutek’s AppSight http://www.identify.com/products/appsightsuite.html • Holodeck Lite http://se.fit.edu/holodeck/ (freeware developed by the authors using techniques similar to those in the handout Listing One to help easily monitor and obstruct common system calls) • Write your own customized monitoring solution Testing for SW Security, ECEN5053, University of Colorado, Boulder

7. Ways in / sources of interaction with the environment • Human interface -- UI • Set of API’s get input from kbd, mouse, etc. • Concerns: unauthorized access, privilege escalation and sabotage • File system • Provides data stored in binary or text format • Trusted to store sensitive data • Test how stored, retrieved, encrypted, managed • API -- input in form of return values of API calls • Operating system kernel -- memory, file pointers, time and date functions, etc. Testing for SW Security, ECEN5053, University of Colorado, Boulder

8. Memory is vulnerable • Any information an app uses passes through memory eventually • Encrypted -- usually ok • Decrypted and stored -- at risk of being read • Encryption keys, CD keys, passwords and other sensitive information are eventually used in an unencrypted form • Must protect their exposure in memory Testing for SW Security, ECEN5053, University of Colorado, Boulder

9. Stress • Stress testing for low memory and other faulty operating conditions that may cause an application to crash • App’s tolerance to environmental stress can prevent • denial of service • situations where app may crash before completing an important task like encrypting passwords • Once it crashes, the state of stored data is ... ? Testing for SW Security, ECEN5053, University of Colorado, Boulder

10. What won’t work • Look at each method of input delivery • Bombard that interface with input Why that won’t work well enough • Most revealing attacks require you to apply inputs through multiple interfaces • Search said these categories can be used to expose vulnerabilities • Dependency attacks • Design-and-implementation attacks Testing for SW Security, ECEN5053, University of Colorado, Boulder

11. Attacking Dependencies • Apps rely heavily on their environment to work properly • Not as overt as human input but there are lots of sources of input • Like any input, if software receives a value outside of its expected range, it can fail. • Environment failures lead to calls on error-handling code (if it exists) • Error handlers are the security weak point of an application Testing for SW Security, ECEN5053, University of Colorado, Boulder

12. Testing error handlers • Failures in the sw’s environment that exercise these code paths are difficult to produce in a test lab situation • Tests that involve disk errors, memory failures, and network problems are only superficially explored • Illusion of security but ... • Servers do run out of disk space • Network connectivity can be intermittent • File permissions can be set improperly • Need to integrate failures into tests to evaluate Testing for SW Security, ECEN5053, University of Colorado, Boulder

13. Just do the impossible ... • Create environmental failures • Tamper with the application code • Simulate specific failing responses from the o.s. • Yeah, right • Takes huge amount of time, effort, and expertise needed to simulate such a failure • Determine where in the code the app uses these resources • How to make appropriate changes to simulate a real failure Testing for SW Security, ECEN5053, University of Colorado, Boulder

14. Alternative approach • Run-time fault injection • Simulating errors to the app in a black-box fashion at run time • Advantage • Nonintrusive • Lets you test production binaries, not contrived versions of the app that have phony return values hard-coded Testing for SW Security, ECEN5053, University of Colorado, Boulder

15. HOW • Overwrite the first few bytes of the actual function to be called in the process space and insert a JMP statement to fault injection code in its place. • Modify import address tables (For doing this in Windows environment, authors recommend Jeffrey Richter, Programming Applications for Microsoft Windows, 4th Edition, Microsoft Press, 1999) Testing for SW Security, ECEN5053, University of Colorado, Boulder

16. WHY? • You redirect a particular system call to your own impostor function • Can log events • Watch the application for file, memory, and registry activity • In control, you can forward a system request to the actual OS function or deny the request by returning any error message you choose. Testing for SW Security, ECEN5053, University of Colorado, Boulder

17. What about registry bugs? • Security problem with the “registry” is trust • When developers read from the registry, assume • values are accurate • haven’t been tampered with maliciously • especially, if their code wrote those values in the first place • “Try and buy” -- users have limited functionality or a time limit in which to try the software. • App can be unlocked if purchased/registered • App may check a registry key at startup • protected with weak encryption or it’s a 1/0 Testing for SW Security, ECEN5053, University of Colorado, Boulder

18. Test app’s use of corrupted files & names • App reads/writes hundreds of files • Handle bad data gracefully without • exposing sensitive information • allowing unsafe behavior • Test with the approach that can happen but is often not tested for • Change a file in a way not covered in requirements • Numerical data --> add text characters • “Successful” test results in denial of service • Crashes the application or the whole system • May expose data during the crash Testing for SW Security, ECEN5053, University of Colorado, Boulder

19. Test low memory/disk/network availability • Deprive the app of resources to see robustness under stress • Rule of thumb -- block a resource when an app seems most in need of it • memory when in an intense computation • disk errors -- introduce faults when doing writes/reads (modify code in Listing One to intercept appropriate system functions) Testing for SW Security, ECEN5053, University of Colorado, Boulder

20. Attack (Test) Design and Implementation • There are subtle security implications made during the design phase • Even if design is secure, choices made during implementation can impact security • Tests that expose these: • Force all error messages • Seek unprotected test APIs • Overflow input buffers • Connect to all ports Testing for SW Security, ECEN5053, University of Colorado, Boulder

21. Force all error messages • Purposes • Try values that should result in error msgs to see how many are handled properly • Make sure error msgs do not reveal unintended information Testing for SW Security, ECEN5053, University of Colorado, Boulder

22. Seek unprotected APIs • Difficult to test complex apps by relying on the APIs extended for normal users alone. • Many designers therefore include additional hooks (extended but not published API) • These often bypass normal security because they are intended for use by the good guys • Intended for custom test harnesses • They mean to remove them ... but then ... they’ll be handy for the next release • So integrated into the code and testing process, removal may destabilize the code • If purposely left in, verify they are not trouble if found Testing for SW Security, ECEN5053, University of Colorado, Boulder

23. Overflow input buffers • Test ability to handle long strings in input fields • Especially where long strings are entered into fields that have an assumed, but not enforced, length, e.g. ZIP codes, state names • API calls notorious for unconstrained inputs • GUI can filter inputs as they are entered • API parameters dealt with internally • checks to ensure values are appropriate before they are used • Most vulnerable -- seldom used or supporting legacy functionality Testing for SW Security, ECEN5053, University of Colorado, Boulder

24. Connect to all ports • Sometimes apps open custom ports to connect with remote servers • Create maintenance channels to automatic updates • Relic from test automation • http://www.ntbugtraq.com/ for many cases where these are left open and unsecured • If this is done in the app • Test the port in the release version • Check what kind of data flows through it Testing for SW Security, ECEN5053, University of Colorado, Boulder

25. Conclusion • These attacks by testers can expose vulnerabilities before release • Only part of complete security-testing methods • This area of research is in the infant stages Testing for SW Security, ECEN5053, University of Colorado, Boulder