ClassLoading Survival Guide

1. ClassLoading Survival Guide Peter Keavney Andrea O. K. Wright Chariot Solutions www.chariotsolutions.com

2. ClassLoading Survival Guide Understanding Java ClassLoading With an understanding of how ClassLoading works, you will be able to debug and avoid common pitfalls. We will not only look at ways to prevent ClassLoading from breaking an application, we will also talk about how applications may be enhanced with custom ClassLoaders.

3. Agenda ClassLoading Basics ClassLoading in the J2EE 1.3 and 1.4 Specs App Server ClassLoading Policies Diagnosing Common ClassLoading Problems Enhancing Applications with Techniques Involving ClassLoaders

4. Java ClassLoading Basics • What is a class loader? • mechanism for loading classes into the JVM • subclass of java.lang.ClassLoader • Hierarchical structure • Parent-child relationships • Uses delegation • Load requests delegated to parent ClassLoader-A Class-X ClassLoader-B ClassLoader-C Class-Y Class-Z Class-Z

5. Java ClassLoading Basics • Unidirectional visibility • Parent-loaded classes cannot see child-loaded classes • Thread.getContextClassloader() • Unique Namespaces • Classes referenced as classloader.classname • Variety of class file locations • File system • Databases • Remote server locations ClassLoader-A Class-X ClassLoader-B ClassLoader-C Class-Y Class-R Class-Z Class-D Remote Server File system Database

6. Java ClassLoading Basics • Bootstrap ClassLoader • aka Primordial ClassLoader • Created by the JVM • Loads the JDK classes included in the JVM (including java.lang.ClassLoader) JVM BootstrapClassLoader • Extensions ClassLoader • Loads JRE classes ExtensionsClassLoader System ClassLoader • System ClassLoader • loads classes specified in the classpath • Loads applications, shared libraries, jars, etc. Application JAR Shared Lib

7. Java ClassLoading BasicsTypical Loading Scenario Determine Initial ClassLoader Delegate to Parent ClassLoader Class Loaded? No – Delegate to Parent Lastly – If class is not loaded in parent hierarchy then the Initial loader will attempt to load the class itself. Bootstrap ClassLoader Extension ClassLoader JVM System ClassLoader Class Loader A Class Loader B Class Loader C MyClass

8. Agenda ClassLoading Basics ClassLoading in the J2EE 1.3 and 1.4 Specs App Server ClassLoading Policies Diagnosing Common ClassLoading Problems Enhancing Applications with Techniques Involving ClassLoaders

9. J2EE Specification Requirements • J2EE v1.3 – section 8.1.1.2 mandates application servers must load dependent JAR files listed in the Manifest Class-Path entry of a primary JAR file (typically an EJB JAR). This requirement is not mandated for EAR or WAR files. • J2EE v1.4 – section 8.3.1 states the following: • Classes loaded by lower-level class loaders must be able to discover the top-level application class loader used to dynamically load application classes.    • Applications, deployed as archives, must ensure that the Java classes in the application are in a separate namespace from classes in other Java applications. • J2EE containers must provide a context class loader per thread that can be used to load top-level application classes.

10. Agenda ClassLoading Basics ClassLoading in the J2EE 1.3 and 1.4 Specs App Server ClassLoading Policies Diagnosing Common ClassLoading Problems Enhancing Applications with Techniques Involving ClassLoaders

11. WebLogic • System ClassLoader • Loads WebLogic Server classes • Loads deployed applications • Loads resources shared across applications • Individual Applications • Load EJB JARs • Load resources shared within application • Load Web Applications • Web Applications • Load servlets, JSPs, etc Bootstrap ClassLoader JVM Extension ClassLoader System ClassLoader EAR JAR WAR RAR EJB JAR EJB JAR EJB JAR WAR

12. WebSphere • WebSphere ClassLoader • Loads WebSphere run-time • Loads required J2EE classes • Application Extensions ClassLoader • Loads application(s) into a single namespace when Policy is set to SINGLE • Module ClassLoader • Loads Web Applications when Isolation Policy set to Module Bootstrap ClassLoader JVM Extension ClassLoader System ClassLoader WebSphere ClassLoader Application Ext ClassLoader Module ClassLoader EAR WAR WAR

13. Oracle iAS • System ClassLoader • Loads OC4J System Classes • Global Connectors ClassLoader • Loads connectors resources shared across applications • Global Applications ClassLoader • Loads applications Bootstrap ClassLoader JVM Extension ClassLoader System ClassLoader Global Connectors ClassLoader Global Applications ClassLoader EAR EAR EAR EAR

14. JBoss • UnifiedClassLoader3 • Extends java.net.URLClassLoader • Loads classes and resources into UnifiedLoaderRepository3 overriding the standard parent delegation model. • UnifiedLoaderRepository3 • Registers Multiple UnifiedClassLoaders Bootstrap ClassLoader JVM Extensions ClassLoader System ClassLoader UnifiedClassLoader3 UnifiedClassLoader3 UnifiedClassLoader3 Unified Loader Repository3 EAR EJB JAR WAR EAR EAR

15. WebLogic Oracle iAS JVM JVM System ClassLoader System ClassLoader EAR Global Connectors ClassLoader JAR WAR Global Applications ClassLoader RAR EAR EAR EJB JAR EJB JAR WAR EAR EAR JVM JVM System ClassLoader System ClassLoader WebSphere ClassLoader UnifiedClassLoader3 UnifiedClassLoader3 UnifiedClassLoader3 Application Ext ClassLoader Unified Loader Repository Module ClassLoader EAR WAR WebSphere JBoss

16. Server Configuration

17. Agenda ClassLoading Basics ClassLoading in the J2EE 1.3 and 1.4 Specs App Server ClassLoading Policies Diagnosing Common ClassLoading Problems Enhancing Applications with Techniques Involving ClassLoaders

18. The Class Could Not Be Found EJB.JAR STRUTS.JAR WAR A WAR B AppAForm AppBForm

19. The Class Could Not Be Found • ClassNotFoundException • Used String representation (e.g. ClassLoader.LoadClass(className)) • Typically, dependent Class is only visible to children • NoClassDefFoundError • Class existed at compile time • Typically, dependent Class is only visible to children • NoClassDefFoundException • An interesting phenomenon: this Exception is not in the JDK, but it appears in dozens of topics in developer forums 

20. The Same Class Was Loaded By Multiple ClassLoaders EJB JAR public void utilityMethod (Object o) { ((Apple)o).doSomething(); } CLASSCASTEXCEPTION Apple WAR A WAR B Apple Apple

21. The Same Class Was Loaded By Multiple ClassLoaders • ClassCastException • This is usually a bug  • But sometimes… same Class was loaded by two different ClassLoaders • ClassCircularityError • A Class and one of its dependents are both dependent on a third Class; different ClassLoaders are used to load that third Class

22. Diagnostic Statements • Class.getClassLoader() • System.getSystemClassLoader() • Thread.currentThread.getContextClassLoader() • ClassLoader.getParent() • Class.getProtectionDomain().getCodeSource().getLocation() • URLClassLoader.getURLs() • Printing out the contents of a JAR: JarEntry jarEntry; Jar InputStream jarInputStream = new JarInputStream(urls[i].openConnection().getInputStream()); while((jarEntry=jarInputStream.getNextJarEntry())!=null) { System.out.println(jarEntry.getName()); }

23. Getting Diagnostic Information • Use the -verbose option • Place diagnostic statements in the application code • Place diagnostic statements in a custom ClassLoader • Place diagnostic statements in the core Java ClassLoaders. The -Xbootclasspath option makes to possible to supersede core Classes in a development environment. C:\>java -Xbootclasspath:C:\debug;j2sdk1.4.2_03\jre\lib\rt.jar …[all other boot.class.path jars] app.java

24. Sample App for Debugging TopHat.java import java.io.File; import java.net.MalformedURLException; import java.net.URLClassLoader; import java.net.URL; import demo.Rabbit; /** * Instantiates and displays a Rabbit. */ public class TopHat { public static void main (String args[]) { try { // Create a ClassLoader that knows where to find demo.Rabbit URL rabbitURL = new File("rabbit.jar").toURL(); URL[] urls = new URL[]{rabbitURL}; URLClassLoader rabbitClassLoader = new URLClassLoader(urls,Thread.currentThread().getContextClassLoader()); // Set the ContextClassLoader for the current Thread so that it can find Rabbit.class Thread.currentThread().setContextClassLoader(rabbitClassLoader); // Make a Rabbit appear. System.out.println(new Rabbit()); } catch (MalformedURLException malformedURLException) { malformedURLException.printStackTrace(); } } }

25. Compile Time and Runtime Environments Rabbit.java package demo; public class Rabbit { } rabbit.jar C:\> jar tf rabbit.jar demo/ demo.Rabbit.class Compilation C:\> javac –classpath rabbit.jar TopHat.java Command Line C:\> java TopHat

26. Desired Behavior vs. Actual Behavior Desired Behavior Tophat should instantiate and display a String representation of a Rabbit, even though rabbit.jar is not referenced on the application classpath. Actual Behavior C:\>java TopHat Exception in thread "main" java.lang.NoClassDefFoundError: demo/Rabbit at TopHat.main(TopHat.java:23)

27. TopHat.java with Diagnostic Statements // Create a ClassLoader that knows where to find demo. Rabbit URL rabbitURL = new File("rabbit.jar").toURL(); URL[] urls = new URL[]{rabbitURL}; URLClassLoader rabbitClassLoader = new URLClassLoader(urls,Thread.currentThread().getContextClassLoader()); System.out.println("---> ClassLoader for TopHat: " + TopHat.class.getClassLoader()); System.out.println("---> Before setting a custom ContextClassLoader, ContextClassLoader is: “); System.out.println( Thread.currentThread().getContextClassLoader()); // Set the ContextClassLoader for the current Thread so that it can find // Rabbit Thread.currentThread().setContextClassLoader(rabbitClassLoader); System.out.println("---> After setting a custom ContextClassLoader ContextClassLoader is: "); System.out.println( Thread.currentThread().getContextClassLoader()); // Make a Rabbit appear. System.out.println(new Rabbit());

28. Output From Diagnostics in the Application C:\>java TopHat ---> ClassLoader for TopHat: sun.misc.Launcher$AppClassLoader@e80a59 ---> Before setting a custom ContextClassLoader, ContextClassLoader is: sun.misc.Launcher$AppClassLoader@e80a59 ---> After setting a custom ContextClassLoader, ContextClassLoader is: java.net.URLClassLoader@eee36c Exception in thread "main" java.lang.NoClassDefFoundError: demo/Rabbit at TopHat.main(TopHat.java:28)

29. Diagnostics in java.net.URLClassLoader protected Class findClass(final String name) throws ClassNotFoundException { try { return (Class) AccessController.doPrivileged(new PrivilegedExceptionAction() { public Object run() throws ClassNotFoundException { String path = name.replace('.', '/').concat(".class"); Resource res = ucp.getResource(path, false); if (res != null) { try { return defineClass(name, res); } catch (IOException e) { throw new ClassNotFoundException(name, e); } } else { System.out.println(“-- Looked in:”); URL[] urls = getURLs(); for (int i = 0;i<urls.length;i++) { System.out.println(urls[i]); } throw new ClassNotFoundException(name); } } }, acc); } catch (java.security.PrivilegedActionException pae) { throw (ClassNotFoundException) pae.getException(); } }

30. Diagnostics in java.lang.ClassLoader protected synchronized Class loadClass(String name, boolean resolve) throws ClassNotFoundException { Class c = findLoadedClass(name); if (c == null) { try { if (name.equals("demo.Rabbit")) System.out.println("---> " + this + "is asking parent (" + parent + ") to try to load " + name + "."); if (parent != null) { c = parent.loadClass(name, false); } else { c = findBootstrapClass0(name); } } catch (ClassNotFoundException e) { // If still not found, then call findClass in order // to find the class. if (name.equals("demo.Rabbit")) System.out.println("---> " + name + " was not found by parent (" + parent + "), so " + this + " will try."); c = findClass(name); } } if (resolve) { resolveClass(c); } return c; } void addClass(Class c) { if (c.toString().equals("class demo.Rabbit")) System.out.println("---> " + this + "loaded " + c); classes.addElement(c); } private synchronized Class loadClassInternal(String name)throws ClassNotFoundException { if (name.equals("demo.Rabbit")) { System.out.println("JVM is requesting that " + this +" load "+ name); } return loadClass(name); }

31. Output from Diagnostics in the Core Java Classes --->JVM is requesting that sun.misc.Launcher$AppClassLoader@e80a59 load demo.Rabbit ---> sun.misc.Launcher$AppClassLoader@e80a59 is asking parent (sun.misc.Launcher$ ExtClassLoader@1ff5ea7) to try to load demo.Rabbit. ---> sun.misc.Launcher$ExtClassLoader@1ff5ea7 is asking parent (null) to try to load demo.Rabbit. ---> demo.Rabbit was not found by parent (null), so sun.misc.Launcher$ExtClassLoader@1ff5ea7 will try. --> Looked in: file:/C:/j2sdk1.4.2_03/jre/lib/ext/dnsns.jar file:/C:/j2sdk1.4.2_03/jre/lib/ext/ldapsec.jar file:/C:/j2sdk1.4.2_03/jre/lib/ext/localedata.jar file:/C:/j2sdk1.4.2_03/jre/lib/ext/sunjce_provider.jar ---> demo.Rabbit was not found by parent (sun.misc.Launcher$ExtClassLoader@1ff5ea7), so sun.misc.Launcher$AppClassLoader@e80a59 will try. --> Looked in: file:/C:/ Exception in thread "main" java.lang.NoClassDefFoundError: demo/Rabbit at TopHat.main(TopHat.java:28)

32. Analysis • Regardless of how the Context ClassLoader is set, the classloader that loaded the calling Class will always beused to load a requested Class when no ClassLoader is specified • new ClassName() • Class.forName(“package.ClassName”) • Context ClassLoaders are used by • JNDI • JAXP

33. Possible Solutions ClassLoader loader = Thread.currentThread().getContextClassLoader(); loader.loadClass(“demo.Rabbit”).newInstance(); – or – ClassLoader loader = Thread.currentThread().getContextClassLoader(); Class.forName(“demo.Rabbit”, loader).newInstance();

34. Rabbit Appears ---> java.net.URLClassLoader@defa1a is asking parent (sun.misc.Launcher$AppClassL oader@e80a59) to try to load demo.Rabbit. ---> sun.misc.Launcher$AppClassLoader@e80a59 is asking parent (sun.misc.Launcher$ ExtClassLoader@1ff5ea7) to try to load demo.Rabbit. ---> sun.misc.Launcher$ExtClassLoader@1ff5ea7 is asking parent (null) to try to load demo.Rabbit. ---> demo.Rabbit was not found by parent (null), so sun.misc.Launcher$ExtClassLo ader@1ff5ea7 will try. --> Looked in: file:/C:/j2sdk1.4.2_03/jre/lib/ext/dnsns.jar file:/C:/j2sdk1.4.2_03/jre/lib/ext/ldapsec.jar file:/C:/j2sdk1.4.2_03/jre/lib/ext/localedata.jar file:/C:/j2sdk1.4.2_03/jre/lib/ext/sunjce_provider.jar ---> demo.Rabbit was not found by parent (sun.misc.Launcher$ExtClassLoader@1ff5e a7), so sun.misc.Launcher$AppClassLoader@e80a59 will try. --> Looked in: file:/C:/ ---> demo.Rabbit was not found by parent (sun.misc.Launcher$AppClassLoader@e80a5 9), so java.net.URLClassLoader@defa1a will try. ---> java.net.URLClassLoader@defa1a loaded class demo.Rabbit demo.Rabbit@a18aa2

35. Sample Web App for Debugging Rabbit.java package webdemo; public class Rabbit { public static boolean visible = true; } manifest.mf Manifest-Version: 1.0 Class-Path: rabbit.jar C:\>jar tf thinair.war META-INF/ META-INF/MANIFEST.MF thinair.jsp C:\>jar tf tophat.war META-INF/ META-INF/MANIFEST.MF tophat.jsp C:\slides5>jar tf magic.ear META-INF/ META-INF/application.xml rabbit.jar thinair.war tophat.war

36. <%@ page import=“webdemo.Rabbit" %> <HTML> <BODY> <% if (Rabbit.visible) %> <pre> ^ ^ / | / | / ^| / ^| \ ^/ \ ^/ \/ -- \/ / \ | 6 6 | \ =@= / \ o / / \ </pre> <% Rabbit.visible = !Rabbit.visible; %> </BODY> </HTML> thinair.jsp tophat.jsp <%@ page import=“webdemo.Rabbit" %> <HTML> <BODY> <% if (Rabbit.visible) %> <pre> ^ ^ / | / | / ^| / ^| \ ^/ \ ^/ \/ -- \/ / \ | 6 6 | \ =@= / \ o / / \ </pre> <% else for (int i = 0; i < 9; i++) { %> <BR> <% } Rabbit.visible = !Rabbit.visible; %> <pre> =================== |MWMWMWMWMW| |MWMWMWMWMW| |MWMWMWMWMW| |MWMWMWMWMW| |MWMWMWMWMW| |MWMWMWMWMM| </pre> </BODY> </HTML>

43. The –verbose Option [Loaded jsp_servlet.__thinair] [Loaded jsp_servlet.__thinair] [Loaded weblogic.utils.http.QueryParams] [Loaded javax.servlet.jsp.JspWriter] [Loaded javax.servlet.jsp.tagext.BodyContent] [Loaded weblogic.servlet.jsp.ByteWriter] [Loaded weblogic.servlet.jsp.BodyContentImpl] [Loaded weblogic.servlet.jsp.JspWriterImpl] [Loaded weblogic.servlet.internal.DelegateChunkWriter] [Loaded weblogic.utils.UnsyncHashtable] [Loaded weblogic.utils.UnsyncHTEntry] [Loaded javax.ejb.Handle] [Loaded weblogic.servlet.internal.session.FileSessionData] [Loaded javax.servlet.http.Cookie] [Loaded weblogic.servlet.internal.ChunkUtils] [Loaded demo.Rabbit] [Loaded weblogic.servlet.internal.CookieParser] [Loaded weblogic.servlet.logging.FormatStringBuffer] [Loaded weblogic.servlet.internal.InetAddressCacheRecord] [Loaded weblogic.utils.concurrent.Lock] [Loaded weblogic.utils.concurrent.Mutex] [Loaded jsp_servlet.__tophat] [Loaded jsp_servlet.__tophat] [Loaded demo.Rabbit] [Loaded weblogic.drs.internal.HeartbeatMessage]

44. Diagnostics in Core ClassLoaders ---> demo.Rabbit was not found by parent (sun.misc.Launcher$AppClassLoader@f38798), so weblogic.utils.classloaders.GenericClassLoader@6b51d8 finder: weblogic.utils.classloaders.MultiClassFinder@a2c691 annotation: will try. ---> demo.Rabbit was not found by parent (weblogic.utils.classloaders.GenericClassLoader@6b51d8 finder: weblogic.utils.classloaders.MultiClassFinder@a2c691 annotation: ), so weblogic.utils.classloaders.GenericClassLoader@6d4f30 finder: weblogic.utils.classloaders.MultiClassFinder@bf9839 annotation: ApplicationClassLoader@ will try. ---> demo.Rabbit was not found by parent (weblogic.utils.classloaders.GenericClassLoader@6d4f30 finder: weblogic.utils.classloaders.MultiClassFinder@bf9839 annotation: ApplicationClassLoader@), so weblogic.utils.classloaders.GenericClassLoader@7227a8 finder: weblogic.utils.classloaders.MultiClassFinder@98c92c annotation: magic@ will try. ---> demo.Rabbit was not found by parent (weblogic.utils.classloaders.GenericClassLoader@7227a8 finder: weblogic.utils.classloaders.MultiClassFinder@98c92c annotation: magic@), so weblogic.utils.classloaders.ChangeAwareClassLoader@899e6a finder: weblogic.utils.classloaders.MultiClassFinder@1444b03 annotation: magic@thinair will try. ---> weblogic.utils.classloaders.ChangeAwareClassLoader@899e6a finder: weblogic.utils.classloaders.MultiClassFinder@1444b03 annotation: magic@thinair loaded class demo.Rabbit ---> demo.Rabbit was not found by parent (sun.misc.Launcher$AppClassLoader@f38798), so weblogic.utils.classloaders.GenericClassLoader@6b51d8 finder: weblogic.utils.classloaders.MultiClassFinder@a2c691 annotation: will try. ---> demo.Rabbit was not found by parent (weblogic.utils.classloaders.GenericClassLoader@6b51d8 finder: weblogic.utils.classloaders.MultiClassFinder@a2c691 annotation: ), so weblogic.utils.classloaders.GenericClassLoader@6d4f30 finder: weblogic.utils.classloaders.MultiClassFinder@bf9839 annotation: ApplicationClassLoader@ will try. ---> demo.Rabbit was not found by parent (weblogic.utils.classloaders.GenericClassLoader@6d4f30 finder: weblogic.utils.classloaders.MultiClassFinder@bf9839 annotation: ApplicationClassLoader@), so weblogic.utils.classloaders.GenericClassLoader@7227a8 finder: weblogic.utils.classloaders.MultiClassFinder@98c92c annotation: magic@ will try. ---> demo.Rabbit was not found by parent (weblogic.utils.classloaders.GenericClassLoader@7227a8 finder: weblogic.utils.classloaders.MultiClassFinder@98c92c annotation: magic@), so weblogic.utils.classloaders.ChangeAwareClassLoader@cdf872 finder: weblogic.utils.classloaders.MultiClassFinder@7d0e5e annotation: magic@tophat will try. ---> weblogic.utils.classloaders.ChangeAwareClassLoader@cdf872 finder: weblogic.utils.classloaders.MultiClassFinder@7d0e5e annotation: magic@tophat loaded classdemo.Rabbit

45. Possible Solutions • J2EE spec does not provide ClassLoading guidelines for Classes specified using the class-path tag in a web app’s manifest • WebLogic uses the web app’s ClassLoader • Server classpath • Downside: Server needs to be restarted whenever a library class changes • Use vendor-specific means of specifying a ClassLoader hierarchy. • Downside: Complicates porting to another server

46. Agenda ClassLoading Basics ClassLoading in the J2EE 1.3 and 1.4 Specs App Server ClassLoading Policies Diagnosing Common ClassLoading Problems Enhancing Applications with Techniques Involving ClassLoaders

47. Enhancing Applications with Techniques Involving ClassLoaders • Customizing Security • Improving Performance • Providing Flexibility

48. Modifying the Default Security Policy Using a Custom ClassLoader Default Permissions Checking: • When a permissions check is requested, policy configuration files are consulted. • Permissions may be granted based on user and/or codebase • Sample policy file entries: grant principal "Max" { permission demo.PrintBackStagePassesPermission; }; grant codeBase “http://honorable.com" { permission java.io.FilePermission "C:\\secrets\*", "read" };

49. Augmenting Permissions Granted By the Default Mechanism • Extend URLClassLoader to override getPermissions(CodeSource codesource) • Call super.getPermissions(codesource) • Adds FilePermissions or ConnectionPermissions based on the location of the Class and the URL protocol • Authorization information may be obtained from the database • Permissions may be role-based protected PermissionCollection getPermissions(CodeSource cs){ PermissionsCollection pc = super.getPermissions(cs); if (checkRoles(username,"Magician's Assistant")) { pc.add(new demo.PrintBackStagePassesPermission()); } return pc; }

50. A Custom ClassLoader that Helps Protect Against Decompilation • Encrypt Classes before deployment • Extend URLClassLoader, and override findClass to decrypt the class before calling defineClass • If the encrypted Classes are delegated to the System ClassLoader, it will throw an Exception, so either… • ensure that the encrypted Classes are not on the classpath • or override loadClass so that only core Classes are delegated to the System loader • Caveat • Class data must eventually pass through the final java.lang.ClassLoader defineClass() • Using the -Xbootclasspath option we looked at earlier, a modified java.lang.ClassLoader could be installed. It's definition of defineClass could be modified to send the decrypted data to a file!