Group A3: Java-Based Tools

1. Group A3: Java-Based Tools James Gimourginas

2. Overview • Introduction • Types of Problematic Software • Compilation Time Tools • For Detecting Bugs • Runtime Tools • For Detecting Bugs • For Detecting Malicious Code • Discussion

3. Motivation for Presenting Java-Based Tools • Heavy use of Java for software development • Web Applications • Web Services • Portlets • Frequent use of JUnit and other testing mechanisms to validate software is “good” before Integration or System testing begins • What other tools are available that could prevent faulty code and reduce test team findings? • Never exposed to the inter-workings of the Java Virtual Machine (JVM)

4. References • C.Flanagan, K.R.M.Leino, K.Lillibridge, G.Nelson, J.B.Saxe and R.Stata. Extended Static Checking for Java (http://www.seas.gwu.edu/~simhaweb/security/spring2007/papers/Flanagan.pdf) • Java Pathfinder project (http://javapathfinder.sourceforge.net/) • Sunil Soman, Chandra Krintz, and Giovanni Vigna. Detecting Malicious Java Code Using Virtual Machine Auditing. 12th USENIX Security Symposium, Washington DC, Aug. 4-8, 2003. (http://cs.ucsb.edu/~ckrintz/papers/usenix03.pdf)

5. External References • http://en.wikipedia.org/wiki/Computer_bug • http://en.wikipedia.org/wiki/Malicious_software • http://en.wikipedia.org/wiki/Intrusion_detection_system • http://en.wikipedia.org/wiki/Type_system#Type_checking • http://en.wikipedia.org/wiki/Java_Native_Interface

6. Overview • Introduction • Types of Problematic Software • Compilation Time Tools • For Detecting Bugs • Runtime Tools • For Detecting Bugs • For Detecting Malicious Code • Discussion

7. Software Bugs • From Wikipedia: “A software bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended (e.g., producing an incorrect result).” • In other words, a software bug is an unintended mistake in a program that causes the program to break in certain situations

8. How Can These Be Eliminated? • Software bugs are discovered throughout a development cycle: • Design Reviews • Code Reviews • Using Software Tools • Manual (Peer) Reviews • Software Testing – Unit, Integration, System, Acceptance • Find a bug earliest is best as it reduces the cost to fix the bug

9. Malicious Software • From Wikipedia: “Malware or malicious software is software designed to infiltrate or damage a computer system without the owner's informed consent” • In other words, malware is software that was designed to impact other software programs in adverse ways

10. How Can These Be Eliminated? • Very difficult to prevent malicious software from executing initially • Intrusion Detection “is the act of detecting actions that attempt to compromise the confidentiality, integrity or availability of a resource.” (Wikipedia) • Different varieties of Intrusion Detection Systems (IDSs) including: • Host-Based (HIDS) • Network (NIDS) • Protocol-Based (PIDS)

11. Overview • Introduction • Types of Problematic Software • Compilation Time Tools • For Detecting Bugs • Runtime Tools • For Detecting Bugs • For Detecting Malicious Code • Discussion

12. Compilation Time Tools for Detecting Bugs • Extended Static Checking for Java (ESC/Java) developed by Compaq Systems Research Center • ESC/Java is “an experimental compile-time program checker that finds common programming errors.” • A way to detect software bugs before a program is executed

13. What is Extended Static Checking? • Java is a statically checked, strongly typed language • Java compilation program, javac, does type checking when compiling a Java program • Extended Static Checking tool attempts to find software bugs that will not be found by type checking • Requires annotations in source code, which are used by the ESC/Java tool • ESC/Java is “unsound” and “incomplete” – may miss an error, may report an error when none exists • ESC/Java attempts to validate the design constraints, described through annotations, are never violated by the program

14. Static Tool Spectrum

15. ESC/Java Annotations • Annotation describes proper use of a variable or method • Take: “int[] elements” • Replace with: “/*@non_null*/ int [] elements;” • Indicates elements array should never be set to null • Needed for ESC/Java to validate a program • Other than allowing ESC/Java to execute, annotations have other useful side effects (that we’ll discuss later)

16. ESC/Java Architecture

17. ESC/Java Execution I • Front end parses and type checks Java code and ESC/Java annotations • Front end creates Abstract Syntax Trees (ASTs), which are passed to Translator component • Also creates a Type-specific Background Predicate, which are passed to the Theorem Prover, for each class with methods that are invoked by the class being examined • Type-specific Background Predicates provide information about external methods or classes that are used by the examined class

18. ESC/Java Execution II • Translator examines each routine body from the provided ASTs and creates Guarded Commands (GCs) for each routine • GCs are simply another way to express what a code routine does in a way that can be understood by the Verification Condition (VC) Generator • Some important notes on the Translator • Not a perfect translation between ASTs and GCs • When creating GCs, Translator relies on annotations for called routines (and not the implementation of called routines)

19. ESC/Java Execution III • Verification Conditions (VCs) Generator creates VCs for each GC • A VC holds true only if all execution paths for the GC end in a valid state, as defined in the annotations • VCs are sent to the Theorem Prover along with Type-specific Background Predicates (from Front End) and Universal Background Predicates (common information shared about Java code) • Theorem Prover decides whether routine is valid based on inputs (i.e. routine cannot reach an invalid state)

20. ESC/Java Execution IV • Post Processor gets Theorem Provers results and presents them to the user • Post Processor is responsible for giving “good” feedback to the user • Information about source code is passed through ESC/Java so specifics, such as the type and location of errors, can be displayed

21. ESC/Java Isn’t Perfect • Annotations increase code size by 5-10% • Authors found that programmers could annotate 300-600 lines of code an hour • Time consuming to backfill existing projects • Unlike written comments, annotations have a specific syntax that must be learned • As stated before, ESC/Java can miss real bugs and can report incorrect bugs • This is possible even if annotations are written correctly

22. ESC/Java Provides • An additional mechanism, above regular type checking, that can be used to identify bugs • Motivation for developers to comment code • Annotations are necessary for ESC/Java to execute and force the developers to document design choices • Tool to force students to write useful comments about pre and post conditions, expected values, etc. • Authors talk about use in the classroom as a learning tool

23. Final Comments on ESC/Java • ESC/Java is an interesting tool • Does find bugs that might otherwise be missed • But, if good programming practices (documenting code, unit testing, holding code reviews) are used, is this type of tool worth the overhead? • As stated by the authors, “feedback from our users suggest the tool has not reached the desired level of cost effectiveness”

24. Overview • Introduction • Types of Problematic Software • Compilation Time Tools • For Detecting Bugs • Runtime Tools • For Detecting Bugs • For Detecting Malicious Code • Discussion

25. Runtime Tools for Detecting Bugs • Java Path Finder (JPF) developed by the NASA Ames Research Center • Verifies Java byte code programs (i.e. compiled programs) at run time • Acts as a Software Model Checker • Represents program as finite state diagram • Explores all execution paths of a program to validate no “bad” states can be reached • Has expanded from a Software Model Checker developed for NASA to a “Swiss army knife” tool for Java runtime verification

26. JPF Architecture

27. JPF Features • Unlike other tools, JPF will output the entire flow of events that leads to a bad state • Highlighted as a great way to find concurrency bugs • Interleaves atomic portions of code in all possible combinations • Helps identify deadlocks and other common (but difficult to identify) concurrency errors • JPF examines all combinations of paths/interleavings to account for non-deterministic nature of a program • Far superior to standard testing • Uses state machine concept and backtracking

28. JPF Downsides • State Space Explosion (the 400lb. elephant) • As code size grows, state machine grows, leading to exponential number of possible execution paths • Recommended application size limit is about 10 KLOC • JPF cannot execute OS-specific system calls (native methods) • Cannot be run on programs that use common Java features: • java.net library • portions of java.io library • Java reflection

29. JPF State Space Explosion

30. JPF Flexibility • JPF developers recognize the scalability challenge • Allow users to customize JPF to fit their needs • Improvement of extensibility is now driving force for JPF development • Custom Search and VM Listeners can be created and registered to handle events • Model Java Interface (MJI) can be used to implement custom libraries • Separates and allows communication between JPF from JVM running JPF • Choice Generators (CGs) used to customize scheduling and data used in execution

31. Other JPF Improvements • Three main aspects that can improve scalability: • Configurable Search Strategies • Heuristics detail what should be searched and the order • Reducing Number of Program States • Many mechanism to allow for more efficient execution • Reducing State Storage Information • State collapsing so only differences are recorded • Similar to file compression in that only differences are tracked

32. Final Thoughts on JPF • Because of all its capabilities, JPF could be its own 45 minute presentation • Extensibility allows for great amount of customization to fit the problem set • Writing custom listener classes to validate almost any feature of program • Out of the box support for concurrency validation is very useful • Race conditions are often difficult to find and difficult to reproduce

33. Overview • Introduction • Types of Problematic Software • Compilation Time Tools • For Detecting Bugs • Runtime Tools • For Detecting Bugs • For Detecting Malicious Code • Discussion

34. Runtime Tools for Identifying Malicious Code • Java often chosen to support web application development, portlet development, and other program types that are portable between systems and architectures • No IDS specifically for monitoring the JVM • Typical Intrusion Detection Systems (IDSs) will view JVM as a single process • In reality, JVM could be running multiple Java programs each with multiple threads

35. Detecting Malicious Java Code • University of California at Santa Barbara (UCSB) researchers developed an auditing system for the JVM (using JikesRVM) • Used audit output as input to an existing IDS (STAT) • In essence, created an IDS for a JVM • Signature-based IDS where signatures are created using a specialized syntax • First of its kind at the writing of the paper • Hoped to identify, at the Java Thread level, suspicious activities and then, in some cases, take action to halt a malicious Java program

36. System Architecture

37. System Components • JikesRVM • Open source JVM • Modified to log certain system events affecting highly exploitable areas • STAT • Host-Based IDS developed by the authors prior to the creation of the JVM Malicious Code Detection System • Audit Log • Stores events created by enhanced JikesRVM • Populated by an Event Logger Thread

38. Auditing System Details

39. Event Driver • JikesRVM is enhanced to invoke the Event Driver when certain events occur • Audited events include: • Class Events • When a thread loads a class • System Call Events • When OS resources are accessed • Java Native Interface (JNI) Events • When non-Java methods are called • Thread Interaction Events • All thread interactions that could be harmful (e.g. Thread.stop())

40. Event Logger Thread • Executes as a JikesRVM system thread • Reduces delay seen by applications running in JVM • Uses XML to log events to promote interoperability among IDSs • One unanswered question - Can this thread be killed by a malicious thread?

41. Wait For It… • The performance hit for execution on the system is massive (and Java isn’t a “fast” language to begin with) • When executed against 9 benchmark programs, the average performance fell 44% when full auditing was performed • Amount of content to be logged is configurable • Can do a partial auditing in an effort to improve performance • Even with partial auditing the performance was degraded by 26%

42. Java Tools Conclusion • ESC/Java, Java Path Finder and the Virtual Machine Auditing programs are useful in their own ways • One attempts to eliminate bugs during compilation, one attempts to identify dynamic/complex bugs at runtime, another attempts to detect malicious programs and act accordingly • ESC/Java and VM Auditing experience significant obstacles when deployed because of overhead costs and performance degradation, respectively • Java Path Find does not scale well and cannot be used, unless significantly enhanced by the user, on programs with routine library calls • As always, a cost-benefit analysis is needed on a per use case basis

43. Overview • Introduction • Types of Problematic Software • Compilation Time Tools • Demonstration • Runtime Tools • Discussion