How To Speed Up Your Static Code Analysis

1. How To Speed Up Your Static Code Analysis OWASP NJ Meetup Presentation – September 18, 2012

2. About Me • About Me Yang Li OWASP Assistant Organizer NYC/NJ Chapter yang.li@owasp.org Direct: (917) 667-1972 LinkedIn: http://www.linkedin.com/in/yangli8 I’m currently working in the financial industry as an Application Security Specialist. My most recent works are focusing on the application security, including Static Code Analysis / Secure Code Review, web application pen-test and application security training.

3. About Me • About You It’s my pleasure to meet both old and new faces tonight. We’re looking for more volunteers to joinus. If you’re working on any project relate to web application security and interested in contributing your time, please contact us 

4. Static Code Analysis Static Code Analysis – Definition “Static code analysis or (SCA) is the analysis of computer software that is performed without actually executing the programs built from that software (analysis performed on executing programs is known as dynamic analysis). In most cases the analysis is performed on some version of the source code; in the other cases on some form of the object code. The term is usually applied to the analysis performed by an automated tool, with human analysis being called program understanding, program comprehension or code review. “

5. The Challenge: Static Code Analysis (SCA) is an important check point of your SDLC program. In a fast moving environment of today, we're expecting everything in place within a short period of time. It's important for the business to have both the secure software and the fast turn-over of the SCA process. The challenge is to find the right recipe for your organization.

6. Why Static Code Analysis (SCA)? • Catch the security bugs fast and directly in the source code (SQL Injection, Cross Site Scripting, Insecure Storage, Buffer Overflow, Memory Leak, … software defects among the OWAPS Top 10 and more). • Upstream opportunity in the Secure Software Development Life Cycle (SSDLC). • Opportunity to work directly with and learn from the developers to remedy the defects – “for fun and profit”. • SCA tool is both efficient and cost effectively as of today, and could be potentially baked into your SSDLC.

7. Why Static Code Analysis (SCA)? Where we want to be

8. SCA Tools Limitations: • Not a full code review – we don’t read the code line by line along with the developer. • SCA tools generally are not intelligent enough to understand the business logics. As a result, it may fail to catch bugs in the logic (false-negatives). • SCA tools could generate large amount of false-positives because they may not understand : • Code logical context. • Other layers of the compensational controls. • Language nuisances. • Other unexpected conditions.

9. SCA Tool Limitation Examples: a) Tool may not understand the security control logic, such as the following: Code security control – OWAPS Code Review Guide v1.1, P.75: if (action == "doStuff"){ booleanpermit = session.authTable.isAuthorised(action); // check table if authoirsed to do action } if (permit){ doStuff(); } else { throw new (InvalidRequestException("Unauthorised request"); // inform user of no authorization session.invalidate(); // Kill session }

10. SCA Tool Limitation Examples: b) Tool may not understand the language nuisances, such as the followings: /* Define my own dynamic memory management macros*/ #define malloc(x) _my_malloc(x, __FILE__, __LINE__) #define free(x) _my_free(x)

11. Where to start: 1. The OWASP Code Review Project: OWAPS Summer of Code 2008 Project - mature, comprehensive and easy to follow code review guide with a lot of examples.

12. Where to start: 2. The NIST Software Assurance Metrics And Tool Evaluation Reference Dataset (SRD) Project: • NIST’s effort to define this new industry’s standard.

13. Contribution Factors: • Tool selection matrix: http://samate.nist.gov/index.php/Source_Code_Security_Analyzers.html

14. Contribution Factors: • Knowledge buildup / learning curve investment: - OWASP Code Review Guide v1.1 is 216 pages; the reference books listed in the page 216 are all brick-thick; language expertise and real world project experience are relatively rare. • Stakeholders buy-in: - management may NOT want to spend the money; developers do NOT like to be audited; existing security staffs may feel threatened by the new thing. • Limitations of SCA tools were amplified with the ‘Diminishing Marginal Utility’ effect. • The right recipe for your software security pasture: it really depends on a lot of factors within your organization.

15. Find the Right Recipe: • Centralized vs. decentralized SCA process model. • Don’t rush to start. Understand the organization, the environment and the application contexts first: • Talk to the business owner, the product manager, the application architect, the developer, the application administrator … Ask for the purpose of the application, the data flow, the explanation of features, demo of the application, … ask for any insights or feedbacks. You’ll be amazed to what you could learn from them. These efforts will help you in the long way.

16. Find the Right Recipe: • Sometimes I use a nice term – ‘Threat Modeling’ (https://www.owasp.org/index.php/Application_Threat_Modeling). - To me it has the additional meaning - ‘Limit the Scope’, so that I can finish my project on time and within budget.

17. Find the Right Recipe: • Identify the resource hogs, automate your process as much as you can. Software security should be fun. By automating you are making the job enjoyable for everyone.

18. Speed It Up Example: App Background: Backend code performing financial transactions used by world-wide customers. This component was developed in Java in early 2000. We decided to use the Fortify SCA commercial tool as a part of our recipe, primarily based on the time constraint and value of the application. Disclaimer: This is a real world project with all sensitive information being masked. It’s being used as an example only. Please do not blindly follow the example. Use it at your own risk.

19. Speed It Up Example: First scan of the code revealed 7,739 potential items within 98,033 Line of Code (LOC).

20. Speed It Up Example: Validation ruled out 6,707 issues (false-positives) within 7 business days – thanks to the great visualization and tracking features of the Fortify tool:

21. Speed It Up Example: After the validation, I still ended up with 39 high, 213 medium, 780 low severity findings – the code reveal the ‘lack of maintenance’: Note: Each item in the above report may contain multiple findings.

22. Speed It Up Example: Threat Modeling: This portion of code did not handle the web session directly, i.e. user authentication / authorization in the UI level. This fact dramatically limited the scope of human reviews. Additional Caution / Disclaimer Such As the Following: “In the event that the developer knows for sure that the input supplied to construct the SQL query is coming from a trustworthy source (such as a local property file or a trusted database). The client can provide us with an attestation stating this fact. Then this issue can be closed.”

23. Speed It Up Example: Reporting: The deliverable is still an unfortunate large report (time consuming process), where the issue nature, affected source files, line numbers and remediation recommendations were needed to be put together, for the benefit of the developer (the real fixer).

24. Speed It Up Example: Reporting:

25. Speed It Up Example: With the 39 high, 213 medium, 780 low risk findings. It’s a lot of ‘cut and paste’ for my report. I do NOT ENJOY this part. I need to do something about it before I’m getting crazy.

26. Speed It Up Example: To ease the reporting pain, first of all, I want to export the results by using a Fortify CLI utility “reportgenerator.bat”:

27. Speed It Up Example: Hmm…what’s the ‘DeveloperWorkbook.xml’? It’s a default report template from Fortify for the developer (who need more details than most of the others):

28. Speed It Up Example: Where’s the ‘DeveloperWorkbook.xml’? Default location with your installation is: $ /usr/bin/find -name "Develop*.xml" ./Fortify 360 v3.0.0/Core/config/reports/DeveloperWorkbook.xml One important hack, change the line in red below: <Title>Vulnerability Examples by Category</Title> <Description>Results summary of all issue categories. Vulnerability examples are provided by category.</Description> <IssueListing limit="5" listing="true"> to unlock the default limitation of data export (5 items per issue): <Title>Vulnerability Examples by Category</Title> <Description>Results summary of all issue categories. Vulnerability examples are provided by category.</Description> <IssueListing limit="-1" listing="true">

29. Speed It Up Example: The CLI utility give me a XML report. Further processing of the XML report, what I want is really like this:

30. Speed It Up Example: To do that, I wrote a simple script ‘fortify_xml_dump.pl’, to massage the XML report into a text file (Spreadsheet ready?!!): $ ./fortify_xml_dump.pl Program name: fortify_xml_dump.pl Developed by: Yang Li Version: 0.2 Functional Description: This program is designed to parse and consolidate Fortify xml reports in batch. The input is a file contains the path to different XML reports in each line. The output is a CSV format table with data merging from all the reports. Syntax: $ fortify_xml_dump.pl ?|-h|--help -h|?|help Print help message -i|input Mandatory input file with paths to XML report in each line -vv|verbose Program in verbose mode -v|version Program version Usage Example: To merge multiple Fortify XML reports in one command, $ ./fortify_xml_dump.pl -i list_path_xml li_y@BTBSG-V0U26SDC6 ~ $ echo "rpt.xml" > list li_y@BTBSG-V0U26SDC6 ~ $ ./fortify_xml_dump.pl -i list > rpt.txt

31. Speed It Up Example: Open the text file in the MS Excel. It saved me weeks of copy and paste. Anyone else like the MS spreadsheet :)

32. Goal Achieved: This project was done with the convincible results in 2 weeks – a FAST turn-over. A BASELINE with the real security metrics were established for this application. In addition, this process could be repeated again for the next code release. The code should become safer over the time. Disclaimer: Again, your environment is unique. Weight all the factors and choose your own recipe carefully. You’ll be able to enjoy your own soup in the end.

33. Sample Script Download Link: https://sites.google.com/site/yangsspaghettihacks/file-cabinet/fortify_xml_dump.zip

34. Credits: OWASP Code Review Project: https://www.owasp.org/index.php/Category:OWASP_Code_Review_Project I’m only covering the beginning of the topic in this talk. Come and join us. Hope you’re enjoying my talk so far .

35. Questions: