A Portable Mechanism for Thread Persistence and Migration

1. A Portable Mechanism for Thread Persistence and Migration Wei Tao The School of Computing University of Utah Doctoral Dissertation Defense January 3, 2001

2. Outline • Research objectives • Problem statement • Survey of related work • Our solution • saving control state with an example • synchronization • Prototype implementation • Performance • Discussion • Research contribution • Future work

3. Research objectives • To investigate a general approach for thread persistence and migration which is • portable: applicable to any platforms and environments • transparent: hiding most code from application users • adaptable: general enough to be used by most frameworks • To show effectiveness and viability of this mechanism by • prototype implementation • To evaluate this mechanism by • performance measurement • definition of constraints

4. The Java programming language • Java supports basic forms of distribution • Architecture-independent byte code with compact format • Dynamic loading and linking • Safety and security models enforcing type and access rules • Synchronized multiple threads of control providing concurrency • Thread migration requires externalizing and moving execution states • Java provides no mechanism for migrating execution states • Java’s RMI is based on the RPC model • Existing Java mobile agent systems transfer agent objects without their associated control state • Need a strategy to externalize thread’s state

5. Externalizing Java threads • A thread’s execution state includes • Data component • Values of local variables in the activation records • In Java, it contains thread’s control stack and related data • Code component • Program’s executable code • Pointers into this code (program counter values) • In Java, it contains the byte code and a pointer into the code • Object sharing and multithreading • Deep or shallow copy • Synchronization of multiple threads

6. Possible solutions • Checkpoint Java Virtual Machine • Saves a memory image of JVM process in a flat file • Has platform specific data representation • Is inefficient and inflexible • Works only for complete programs • Modify Java Virtual Machine implementation • Takes architecture-independent checkpoints in JVM • Makes JVM save or restore the execution state when program shuts down or restarts • Negates the ubiquity benefit of standard JVM • Define thread saving in application code • Insert state saving code into either source code or byte code • Platform independence

7. State of the art • Persistence in Java • Persistent thread is a feature missing in existing persistence systems in Java • Object serialization • Persistent Storage Engine (PSE) • The JDBC interface • Orthogonal persistence: PJAMA • Java mobile agent systems • Strong mobility is not supported in most Java mobile agent systems • IBM’s AGLETS • ObjectSpace’s VOYAGER • General Magic’s ODYSSEY

8. State of the art (continued) • Thread migration in Java • Specialized thread migration or mobility systems • DELTA EXECUTION: integrated with JESSICA • MOBA: designed for Grid-based thread migration • SUMATRA: integrated in its mobile agent system • Extended JVM • Provides new services in JDK for saving threads • Not portable to standard JVMs • No support for synchronization • Source code level transformation • Big performance overhead • Cannot save complete execution state • May extend the language

9. Our approach • Byte code level transformation • Inserting byte code to save and rebuild control state by a class file processor • Special features • Portability • No modification to JVM or Java language • Transparency • Code are inserted automatically by the post-processor • Adaptability • Is a general strategy to be used with any persistence or migration systems • Low overheads • Ability to save programs with complex synchronization states

10. Design and implementation • New APIs for thread externalization • New exception classes: ShutdownException, MoveException • New methods initiate externalization • checkStop: throws ShutdownException if shutdown flag is set • moveTo(host): starts to externalize by throwing MoveException • ShutdownManager • Provides public method to initiate shutdown: checkStop • Provides methods to be inserted by the processor • Manages the shutdown and restart process • MigrationManager • Provides public method to start migration: moveTo(host) • Manages migration process when necessary

11. Two exceptions publicclass ShutdownException extends Exception { private Stack record; public void push(Object item) { … } public Object pop() { … } public Stack getRecord() { … } } public class MoveException extends ShutdownException { private String host; public MoveException(String host) { … } public String getHost() { … } }

12. Saving control state • Collapsing a control stack • Wraps each statement that throws either exception with handlers • In the handlers, saves local data and byte code label into the exception and re-throws it • Externalized thread state is saved or transmitted at top level • Rebuilding a control stack • Check the saved state for current thread at the beginning of each processed method • Execution jumps to the start of shutdown code if the saved state is null (not a restart) or empty (restart process is complete) • Otherwise the saved label is popped from the saved state record • A switch instruction is used to dispatch to the proper code to restore local data • Execution jumps to the instruction with the saved label after restoring local data

13. A simple example: merge sort publicvoid mergeSort( int[ ] A, int p, int r ) throws ShutdownException { if (p >= r) return; int q = (p + r)/2; mergeSort(A, p, q); mergeSort(A, q+1, r); merge(A, p, q, r); } publicvoid merge( int[ ] A, int p, int q, int r ) throws ShutdownException { ShutdownManager.checkStop( ); ... }

14. Merge sort: transformed code publicvoid merge( int[ ] A, int p, int q, int r ) throws ShutdownException { … // retrieve the saved record of current thread to se if( se != null && !se.isEmpty()) { goto r } try { ShutdownManager.checkStop( ); } catch (ShutdownException se) { se.pushInt( r ); se.pushInt( q ); se.pushInt( p ); se.pushObject( A ); throw se; } r : ... }

15. Merge sort: transformed code publicvoid mergeSort(int[ ] A, int p, int r)throws ShutdownException { … // retrieve the saved record of current thread to se if( se != null && !se.isEmpty( ) ) { int pc = se.popInt( ); switch( pc ) { case 0: q = se.popInt( ); t1 = (int[])se.popObject( ); t2 = se.popInt( ); t3 = se.popInt( ); goto r1 case 1: q = se.popInt( ); t1 = (int[])se.popObject( ); t2 = se.popInt( ); t3 = se.popInt( ); goto r2 case 2: q = se.popInt( ); t1 = (int[])se.popObject( ); t2 = se.popInt( ); t3 = se.popInt( ); t4 = se.popInt( ); goto r3 }

16. Transformed code (continued) if (p >= r) return; int q = (p + r)/2; t1 = A; t2 = p; t3 = q; r1:try { mergeSort( t1, t2, t3 ); } catch (ShutdownException se) { se.pushInt( q ); se.pushInt( 0 ); se.pushInt( r ); se.pushInt( p ); se.pushObject( A ); throw se; } t1 = A; t2 = q + 1; t3 = r; r2:try { mergeSort( t1, t2, t3 ); } catch (ShutdownException se) { se.pushInt( q ); se.pushInt( 1 ); se.pushInt( r ); se.pushInt( p ); se.pushObject( A ); throw se; } t1 = A; t2 = p; t3 = q; t4 = r; r3:try { merge( t1, t2, t3, t4 ); } catch (ShutdownException se) { se.pushInt( q ); se.pushInt( 2 ); se.pushInt( r ); se.pushInt( p ); se.pushObject( A ); throw se; } }

17. Problems with multi-threading • Inconsistent state • A thread releases all locks it holds after shutdown, possibly leaving the unlocked objects in inconsistent states • Threads should be prevented from accessing those resources with inconsistent states • Indefinite blocking • No threads should be blocked indefinitely because of other threads’ shutdown • Missed signals • A thread should not send signals to threads that have already shut down • Nested monitor lockouts • Threads should be restarted in a correct order

18. Synchronizing shutdown • Saving synchronization state • Locking information is collected during shutdown process • Object wait set information is collected at execution time • Before a thread releases a lock during shutdown process, it • sets the locked object to be dirty • A thread starts shutdown when trying to access a dirty object • interrupts all threads that are waiting for the lock • All waiting threads start shutdown after receiving interruption • Notify signal is never missed due to shutdown • Wait threads will not start shutdown until the owner thread starts shutdown and interrupts them

19. Synchronizing restart • Threads should be restarted in a specific order • The threads in wait set of a lock should restart before the thread who owned the lock at shutdown time • A wait thread acquires the lock and jumps to execute wait() • It releases the lock when entering wait state, allowing other threads to acquire the lock • A central barrier object is used to block new computation • A thread should not start new computation until all other threads have finished the restart process • Threads in wait state when shutdown occurred would wait again • Other threads are blocked by a central barrier controlled by shutdown manager

20. Prototype implementation • Thread persistence • All active threads in a program are saved and restarted • A shutdown manager is created when a program starts • Experimental frameworks • Java serialization • Persistent storage engine • Thread migration • A thread migrates transparently with control state • resumes execution from exactly where it left • Experimental frameworks • Java sockets • ObjectSpace’s VOYAGER

21. Using serialization • Persistent types • Classes implement java.io.Serializable or java.io.Externalizable • Object sharing • A complete representation of the entire object graph is serialized • Objects inside the object graph maintain their sharing status • Non-serializable types • Declaring a field to be transient prevents it from being serialized • Transient objects become null after serialization • They may need to be newly created after deserialization • Local variables with non-serializable types are saved as null • Define a corresponding wrapper class for each non-serializable types that need to be recreated

22. Using persistent storage engine • PSE has the following advantages over serialization • Better performance and reliability • Database management features such as transactions • Same APIs and byte code transformations as used in the object serialization • Two implementation differences • The captured states are saved with the objects to a persistent store in PSE instead of a file • The object of a class implements java.lang.Runnable is saved • New thread objects are created using saved Runnable objects • A look-up table is used to swizzle old references to the corresponding new thread objects

23. Using Java sockets • Class MobileThread • serves as a super class for all mobile threads • has a local field keeping the thread’s control state • provides method send( ) to send out the thread • Each server which may host mobile threads • maintains a server socket with specified address and port. • creates a migration manager for each client socket • manages a local name service to identify local resources • A migration manager • receives object streams and recreates mobile thread objects • starts the received mobile threads

24. Integrating with VOYAGER • Voyager • Voyager is a Java distributed object system which supports object mobility and mobile agents • Objects are transferred, but not the control states • Programmer must manually encode control states • forced to adopt event style programming • migration cannot occur deep in recursion, iteration … • Supporting control migration in Voyager agents • Use the additional APIs to initiate externalization • Insert externalization code via post-processor • Enable agents to migrate to a new host in any control context

25. Performance measurements • Four benchmarked applications • Bank: average time on 10 threads with 30,000 transactions • Crew: total time on two readers and two writers • Buyer agent: total time of 50 repetitions of the bid cycle • Tower of Hanoi: total time of 20 disks • Experiment setup • Environment: • Java SDK 2 on Microsoft Windows98 (PII 300MHz) • Measurement: • Run time: Averaging 10 executions of each program • Space: total file size in bytes for all processed class files

26. Normalized run time

27. Normalized class file sizes

28. Performance analysis • Run time overhead for normal execution depends on • percentage of processed methods • percentage of instructions which may initiate shutdown • total run time of original code • Run time overhead for state saving depends on • how deep in the call chain the shutdown instruction is • how many local data at each level needs to be saved • Space overhead depends on • percentage of processed methods • percentage of instructions which may initiate shutdown • how many local data needs to be saved at each instruction • how intense synchronization is used in the program

29. Discussion • Limitations of language level transformation • System level thread migration • Class and interface initializers • Finally clauses • Constraints for class files • Unsafe type • Unstructured lock • Other restrictions • Requesting shutdown at prohibited spots • Call back of native methods

30. Research contributions • Designed a language level rewriting approach to externalize running threads • Analyzed the correctness of code transformation using denotational semantics • Implemented a Java byte code processor for thread persistence and migration on different persistence and migration frameworks • Presented a mechanism to preserve synchronization when saving multiple threads • Measured and analyzed performance overheads • Discussed the limitations of language level approach

31. Future work • Formal analysis • Byte code transformation • Synchronization • Framework enhancements • Saving a group of related threads and synchronization • Processing byte code during load time • Enabling state saving and restoration for native methods • More applications • Fast activation of applications or applets • Recoverable exceptions • Distributed transactions • Load balancing among heterogeneous platforms

32. The full dissertation is available at http://www.cs.utah.edu/~tao/research/index.html The prototype JTHREAD is available at http://www.cs.utah.edu/~tao/research/Jthread.html Happy New Millennium!