Designing a thread-safe class

1. Designing a thread-safe class • Store all states in public static fields • Verifying thread safety is hard • Modifications to the program hard • Design for thread safety • Identify variables that form object’s state • Identify invariants on these states • Establish policy for managing concurrent access • Objects and variables have a state space • Smaller, the better

2. State-dependent operations • Current state dependent on previous state • cannot remove item from empty queue • Single threaded • Precondition does not hold • Operation fails • Concurrent • Precondition may become true later • Wait for action by another thread • Precondition may be invalidated as well

3. Instance confinement • Encapsulation of data • Object’s methods will access the data • Ensure data is accessed with appropriate lock held • Not escape intended scope • Monitor pattern • Encapsulate all mutable state • Use object’s own intrinsic lock to guard the state Public class PrivateLock { private final Object myLock = new Object(); Widget widget; void someMethod() { synchronized(myLock) { …. } } }

4. Summary • Mutable state • Coordinating access to mutable state • Less mutable, easier to ensure thread safety • Make fields final • If they need not be mutable • Immutable objects are thread-safe • Encapsulation helps thread-safety • Mutable state • Guard with a lock (and the same lock everywhere) • Compound actions are guarded together

5. Summary (contd) • Mutable variable from multiple threads with no synchronization • Broken program • Include thread-safety in design process • Explicit documentation • Document synchronization policy

6. Structuring concurrent applications • Tasks • Abstract, discrete unit of works • Benefits • Simplifies program organization • Facilitates error recovery • Promotes concurrency • Natural structure for parallelizing work • Independent activities • State, result or side-effects of other tasks • Each task represents fraction of application processing capacity

7. Executing tasks sequentially Class WebServer { public static void main(…) { ServerSocket socket = new ServerSocket(80); while(true) { Socket connection = socket.accept(); handleRequest(connection); } } } • Simple and correct • Perform poorly in production • Might work if handleRequest returned immediately

8. Explicitly creating threads for tasks Class WebServer { …. Serversocket socket = new ServerSocket(80); while(true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; new Thread(task).start(); } } • For each connection, new thread is created

9. Consequences • Task processing is offloaded from main thread • Wait for new connections • Tasks can be processed in parallel • Multiple requests serviced simultaneously • Task-handling code must be thread-safe

10. Disadvantages of unbounded thread creation • Thread lifecycle overhead • Thread creation • Thread teardown • Resource consumption • Threads consume resources • Some threads may sit idle consuming resources • Stability • Limit on how many threads can be created • OutOfMemoryError is possible

11. Decoupling • Tasks are logical units of work • Threads to run tasks asynchronously • Decouple task submission and task execution • Task submission – producer • Task execution – consumer • Pre-determined number of threads created • Testing possible for stability • Bounded queue of tasks

12. Execution policies • In what thread will tasks be executed • Order of execution of tasks • Concurrent tasks • Queuing length of the tasks • Task that should be evicted • How application should be notified • Action before/after executing a task

13. Cancellation and shutdown • Stop tasks/threads earlier • How? • Immediately? • Why? • User requested cancellation • Time-limited activities • Best solution within a time • Application events • Decomposition of problem space • Errors • e.g., disk full • Shutdown

14. Task cancellation • public void foo() { • while(!cancelled) { • doSomething(); } } public void bar() { cancelled = true; } • Eventually foo() exits • What is the type of cancelled?

15. Interruption • Each thread has a boolean interrupted status • Some blocking methods detect interrupts and return early with exception • Well behaved methods • Return the interrupt status to the calling method to handle • Poorly behaved methods • Swallow the interrupts • Interruption policy • How thread should interpret interrupt? • What needs to be done? • What units of work are considered atomic? • Time of reaction to interrupts

16. Thread Pools • Dependent tasks • Constraints on execution policy • Avoid liveness problems • Tasks that exploit thread confinement • Single threaded executor • Thread safety is not required • Response-time sensitive tasks • Long running tasks to thread pool with few threads • Tasks that use ThreadLocal • Not to be used to communicate between tasks

17. Thread Pools (contd.) • Thread starvation deadlock • Tasks depend on other tasks in thread pool • Second task sits on work queue • First task waits for result of second task • Long-running tasks • Fewer threads in thread pool • Responsiveness suffers

18. Sizing thread pools • Ideal size • Type of submitted tasks • Characteristics of the deployment system • Not hard-coded • Computed dynamically • Configuration mechanism • Too big • Threads compete for resources • Too small • Throughput suffers

19. Sizing thread pools (contd.) • Number of processors • Amount of memory • Type of task • Computation • I/O • Some combination • Requires scarce resource • Compute-intensive tasks • N+1 threads on N processor system • I/O based tasks • More threads in the pool

20. Optimal pool size • N_cpu = number of CPUs • U_cpu = target CPU utilization, 0 <= U_cpu <= 1 • W/C = ratio of wait time to compute time N_threads = N_cpu * U_cpu * (1 + W/C) • Other parameters • Memory, file handles, etc