Secure Software Development

1. Secure Software Development Risk-Based Security Testing Chapter 7 Rasool Jalili & M.S. Dousti Dept. of Computer Engineering Fall 2010

2. Risk-Based Security Testing

3. Security testing • Security testing has recently moved beyond the realm of network port scanning to include probing software behavior as a critical aspect of system behavior. • Security testing done properly goes much deeper than simple black box probing on the presentation layer • and even beyond the functional testing of security mechanism. • Testers must carry out a risk-based approach, grounded in both the system's architectural reality and the attacker's mindset. • By identifying risks in the system and creating tests driven by those risks, a software security tester can properly focus on areas of code where an attack is likely to succeed.

4. The main difference of security testing and pentest: • the level of approach and • the timing of the testing itself. • Pen-testing happens once software is complete and installed in its operational environment. Also, penetration testing is focused outsidein and is somewhat brief. • Security testing can be applied before the software is complete, at the unit level, in a testing environment with stubs and pre-integration. • Should start at the feature or component/unit level, prior to system integration.

5. Security unit testing carries the benefit of breaking system security down into a number of discrete parts. • Theoretically, if each component is implemented safely and fulfills inter-component design criteria, the greater system should be in reasonable shape. • Security testing should continue at the system level and should be directed at properties of the integrated software system. • This is precisely where penetration testing meets security testing. • Finally, abuse cases developed earlier in the lifecycle should be used to enhance a test plan with adversarial tests based on possible abuse scenarios. • Security testing involves as much black hat thinking as white hat thinking.

6. From OutsideIn to InsideOut • Traditional approaches to computer and network security testing focus on network infrastructure, firewalls, and port scanning. • This is especially true of network and software penetration testing (application penetration testing). • Better penetration testing approaches take architectural risks, code scanning results, and security requirements into account, • but still focus on an outsidein perspective. • A classic outsidein paradigm focusing on firewall placement.

7. Problem: the perimeter is only apparent at the network/packet level. • At the level of software applications, the perimeter has all but disappeared. • That's because firewalls have been configured (or mis-configured, depending on your perspective) to allow advanced applications to tunnel right through them. • A good example of this phenomenon is the SOAP protocol, which is designed (on purpose) to carry traffic through port 80 for various different applications. • In some sense, SOAP is an anti-security device invented by software people. • In the brave new world of Service Oriented Architecture (SOA) for applications, we should not be surprised that the firewall is quickly becoming irrelevant. • By contrast, in an insideoutapproach to security, whereby software is itself subjected to rigorous risk management and security testing.

8. What's So Different about Security? • Software security is about making software behave in the presence of a malicious attack. • Standard software testing literature is only concerned with what happens when software fails, regardless of intent. • Software security - Software safety =an intelligent adversary. • Most safety-critical systems (and high-assurance systems) posit a white hat world. • Fact is, we live in a world with plenty of black hats as well, and we need to address that (head on). • Attackers generally don't care whether a vulnerability is due to a flaw or a bug, although bugs tend to be easier to exploit. • Design-level vulnerabilities are the hardest defect category to handle.

9. End of today lecture.

10. Risk Management and Security Testing • Different tasks to manage software security risks, such as: • Creating security abuse/misuse cases • Listing normative security requirements (and security features and functions) • Performing architectural risk analysis • Building risk-based security test plans • Using static analysis tools • Performing security tests • Performing penetration testing in the final environment • Cleaning up security breaches

11. Do not forget: Software security is notsecurity software. • So, SWS testing must necessarily involve two diverse approaches: • Functional security testing: testing security mechanisms • Adversarial security testing: performing risk-based security testing motivated by simulating the attacker's approach • Together, these two distinct activities are a mix of white hat (security functionality) and black hat (security attack) philosophies. • Wrong belief: liberal application and use of various security features "adding SSL“ to securing an application. • Software security practitioners regret the over-reliance on "magic crypto fairy dust“.

12. How to Approach Security Testing • Security testing: determining who should do the testing and what activities they should undertake. • Who? • Standard testing organizations using a traditional approach  functional security testing. • Traditional QA staff? They have more difficulty due to the lack of expertise. • Security tests are difficult to design, due to thinking like an attacker. • Security tests don't often cause direct security exploit. • Bottom line: Risk-based security testing relies more on expertise and security experience (not testing experience).

13. How: • Books; helps a lot … • White & Black box testing with/ without having access to source. • In Black Box testing, malicious input in an effort to break the program; • Any testing method can reveal possible software risks and potential exploits. • However, One problem with security testing (black or white box) is the lack of it. • There is no end-point for software security; even a reasonable security testing regimen is just a start. • Unfortunately, security is being sold as a product, and most defensive mechanisms on the market do little to address the heart of the problem, which is bad software. • Any testing approach is deeply impacted by software process issues.

14. Thinking about (Malicious) Input • The biggest problems in software security AS software takes input. • Trust the input is a critical question that all software builders must consider. • From buffer overflow (which involves putting too much input in too small a place) to SQL injection attack and cross-site scripting (XSS) attacks, trusting input is the common root cause. • Carefully handling input is dominant to software security. • If your program consumes data from "out there," you need to think carefully about who can feed and the stuff your program eats. • STOMATH IS THE ROOT OF ALL DESEASES. • Attacker toolkits focus plenty of attention on input, with a plenty of fault injection tools, re-players, and the like. • By its very nature, penetration testing is focused on input as well • If your program accepts input over the network, it needs to be very doubtful of what it is getting. • Using a black-list approach (which tries to enumerate all possible bad input) will not work. Instead, software needs to defend its input space with a white-list approach.

15. Getting Over Input • Testing around malicious input is a necessary but not sufficient condition. • Security testing needs to focusing on data structures, components, APIs, program state, and so on. • In addition to building tests around risks that remain in the system, testers should consider things like: • Sockets • Pipes • The Win32 Registry • Files • Remote procedure calls (RPCs) • Command-line arguments • …. • Time is a critical issue; two major aspects of time to consider • One problem is that most developers are unfamiliar with the effects of multithreading on their systems. That means they often overlook subtle time-based attacks.

16. Unless a security tester thinks like a bad guy (black hat firmly on head), security testing will not be effective.