Java Security

1. CS 242 2008 Java Security John Mitchell Reading: Chapter 13 (+Slides contain additional material)

2. Language Overview History and design goals Classes and Inheritance Object features Encapsulation Inheritance Types and Subtyping Primitive and ref types Interfaces; arrays Exception hierarchy Generics Subtype polymorphism. generic programming Virtual machine overview Loader and initialization Linker and verifier Bytecode interpreter Method lookup four different bytecodes Verifier analysis Implementation of generics Security Buffer overflow Java “sandbox” Type safety and attacks Outline

3. Java Security • Security : “Bad stuff does not happen” • Prevent unauthorized use of computational resources • Java security • Java code can read input from careless user or malicious attacker • Java code can be transmitted over network – code may be written by careless friend or malicious attacker • Distributed applications • Browser applets Java is designed to reduce many security risks

4. Bad-input-to-good-Java-code attack: Buffer Overflow Attack • Most prevalent general security problem today • Large number of CERT advisories are related to buffer overflow vulnerabilities in OS, other code • General network-based attack • Attacker sends carefully designed network msgs • Input causes privileged program (e.g., Sendmail) to do something it was not designed to do • Does not work in Java • Illustrates what Java was designed to prevent

5. void f (char *str) { char buffer[16]; … strcpy(buffer,str); } void main() { char large_string[256]; int i; for( i = 0; i < 255; i++) large_string[i] = 'A'; f(large_string); } Function Copies str into buffer until null character found Could write past end of buffer, over function retun addr Calling program Writes 'A' over f activation record Function f “returns” to location 0x4141414141 This causes segmentation fault Variations Put meaningful address in string Put code in string and jump to it !! Sample C code to illustrate attack See: Smashing the stack for fun and profit

6. + Bad-Java-code-on-good-platform attacks: Java Security Mechanisms • Sandboxing • Run program in restricted environment • Analogy: child’s sandbox with only safe toys • This term refers to • Features of loader, verifier, interpreter that restrict program • Java Security Manager, a special object that acts as access control “reference monitor” • Code signing • Use cryptography to establish origin of class file • This info can be used by security manager

7. JVM sandbox local code remote code (applets) resources The Java 1.0 sandbox • Local code subject to Java language restrictions only • Downloaded code (applet) further restricted to sandbox • cannot access the local file system • cannot access system resources, • can establish network connection only to originating web server

8. [JDK 1.0 – JDK 1.1] JavaVM Policy Enforcment From java.io.File: public boolean delete() { SecurityManager security = System.getSecurityManager(); if (security != null) { security.checkDelete(path); } if (isDirectory()) return rmdir0(); else return delete0(); } checkDelete throws a SecurityExecption if the delete would violate the policy (re-thrown by delete) What could go seriously wrong with this?! Slide: David Evans

9. public class AppletSecurity extends SecurityManager { ... public synchronized void checkDelete(String file) throws Security Exception { checkWrite(file); } } HotJava’s Policy (JDK 1.1.7) Slide: David Evans

10. JVM local code sandbox signed and trusted remote code (applets) unsigned, or signed and untrusted resources Java 1.1 trusted code • Trust code from trusted sources • Local file system • Signed by trusted principal

11. Java 2 Finer grained access control • All access to system resources based on policy file • Protection domain: code source and permissions granted • Code source consists of a URL and an optional signature • Permissions granted in policy file grant CodeBase “http://java.sun.com”, SignedBy “Sun” { permission java.io.FilePermission “${user.home}${/}*”, “read, write”; permission java.net.SocketPermission “localhost:1024-”, “listen”;}; JVM local or remote code (signed or unsigned) class loaders policy file resources

12. Reference Monitor • Basic concept in security • Introduced by Lampson in 1970s • Part of “Trusted Computing Base” (TCB) • Requirements for reference monitor • Mediate all requests to access protected resources • Ideally: • Tamperproof • Verifiable • Always invoked These are ideal requirements, not generally met in practice

13. Java Sandbox • Four complementary mechanisms • Class loader • Separate namespaces for separate class loaders • Associates protection domain with each class • Verifier and JVM run-time tests • NO unchecked casts or other type errors, NO array overflow • Preserves private, protected visibility levels • Security Manager • Called by library functions to decide if request is allowed • Uses protection domain associated with code, user policy • Coming up in a few slides: stack inspection

14. The Java Virtual Machine (JVM) JVM network class loader instance class file verifier heap JIT class area primordial class loader execution engine local untrusted classes trusted classes native method area native method loader Security Manager native methods operating system native code More details than earlier picture ... Java code

15. Who loads the class loader? • Class loaders • locate and load classes into the JVM • primordial class loader • loads trusted classes (system classes on the boot class path) • integral part of the JVM • class loader instances • Implemented by applications • Load untrusted classes from file system or network • Instances of java.net.URLClassLoader • which extends SecureClassLoader

16. Class loader security mechanism • separate name spaces • classes loaded by a class loader instance belong to the same name space • since classes with the same name may exist on different Web sites, different Web sites are handled by different instances of the applet class loader • a class in one name space cannot access a class in another name space  classes from different Web sites cannot access each other • establish the protection domain (set of permissions) for a loaded class • enforce a search order that prevents trusted system classes from being replaced by classes from less trusted sources • see next two slide …

17. Class loading process when a class is referenced • JVM: invokes the class loader associated with the requesting program • class loader: has the class already been loaded? • yes: • does the program have permission to access the class? • yes: return object reference • no: security exception • no: • does the program have permission to create the requested class? • yes: • first delegate loading task to parent • if parent returns success, then return (class is loaded) • if parent returned failure, then load class and return • no: security exception

18. 4a loads class from boot class path 3 4b failure 5a loads class from class path 2 5b success / failure 6 loads class from URL 1 Class loading task delegation primordial class loader (searches on the boot class path) a class loader instance started at JVM startup (searches on the class path) a class loader instance associated with a URL (searches on the site specified by the URL) class request

19. Byte code verifier • performs static analysis of the bytecode • syntactic analysis • all arguments to flow control instructions must cause branches to the start of a valid instruction • all references to local variables must be legal • all references to the constant pool must be to an entry of appropriate type • all opcodes must have the correct number of arguments • exception handlers must start at the beginning of a valid instruction • … • data flow analysis • attempts to reconstruct the behavior of the code at run time without actually running the code • keeps track only of types not the actual values in the stack and in local variables • it is theoretically impossible to identify all problems that may occur at run time with static analysis

20. Verifier (should be) Conservative JVML programs Safe programs Verifiable programs Slide: Nate Paul’s ACSAC talk

21. Complexity Increases Risk JVML programs Safe programs Verifiable programs Bug Slide: Nate Paul’s ACSAC talk

22. Security Manager • Java library functions call security manager • Security manager object answers at run time • Decide if calling code is allowed to do operation • Examine protection domain of calling class • Signer: organization that signed code before loading • Location: URL where the Java classes came from • Uses the system policy to decide access permission

23. The Security Manager • Implements a checkPermission() method • CheckPermission() is called from trusted system classes • e.g., if you want to open a socket you need to create a Socket object • the Socket class is a trusted system class that always invokes the checkPermission() method • Is effective to the extent that • system resources are accessible only via trusted system classes • trusted system classes cannot be overwritten • Relies on class loader • To make sure trusted system classes are not overridden.

24. Redefining the Security Manager • JVM allows only one active SM at a time • Default SM provided by the JDK • Java programs can replace the default SM • Two permissions are needed: • create an instance of SecurityManager • set an SM instance as active • Example: grant CodeBase “…”, SignedBy “…” { permission java.lang.RuntimePermission “createSecurityManager”; permission java.lang.RuntimePermission “setSecurityManager”;}; • SecurityManager constructor, setSecurityManager() call the checkPermissions() method of current SM to verify the caller has needed permissions

25. Confused Deputy • KeyKOS story • Compiler writes errors into user-designated log file • User can’t change accounting file, but compiler can • User designates accounting file as compiler log file • Reference: Norm Hardy , The Confused Deputy (…) • Early Netscape problem • Applet asks font manager for “password file” font • Font manager can’t find it so asks system registry • System registry trusts font manager, so turns it over • Font manager gives password file to applet

26. Stack Inspection • Permission depends on • Permission of calling method • Permission of all methods above it on stack • Up to method that is trusted and asserts this trust Many details omitted here method f method g method h java.io.FileInputStream • Stories: Netscape font / passwd bug; Shockwave plug-in

27. Vulnerabilities in JavaVM 45 40 35 30 25 Vulnerabilities Reported 20 15 10 5 0 0 1 2 3 4 5 6 7 8 9 July 1996 Years Since First Release July 2005 Slide: David Evans

28. Where are They? several of these were because of jsr complexity Slide: David Evans

29. Beyond JVM security • JVM does not prevent • Denial of service attacks • Applet creates large windows and ignores mouse • Certain network behavior • Applet can connect to port 25 on client machine, forge email (on some implementations) • URL spoofing • Applet can write false URL on browser status line • Annoying behavior • Applet can play loud sound • Applet can reload pages in new windows

30. E: Electric Communities • Electronic communities • Planned network of interacting virtual realities • Example: automated garden (in Netherlands?) with watering can • E programming language • Distributed communication • An object may send messages directly to objects that exist in other machines • Capability system • Goal: detailed control over sensitive functions within a single machine or across a network • Optimistic computation • Reduces the effect of communications latency in distributed systems

31. Java capability example • Give untrusted object read access to a file • Pass the name of the file • Pass the File object • Pass a ReadStream on the File object • Problem • Given Java ReadStream, can request the File object of the ReadStream, then access file, directory structure, etc. • E approach • Capabilities are granted to E objects at load time. • A cryptographic signature and a set of capability requirements are attached to every E object See Amoeba OS papers for good description of capability system

32. Lots more information in books, online