Threads versus Events

1. Threads versus Events CSE451 Andrew Whitaker

2. This Class • Threads vs. events is an ongoing debate • So, neat-and-tidy answers aren’t necessarily available • Our goals: • Understand the debate • Analyze arguments • Form our own opinion

3. A Brief History of a Debate • 1995: Why Threads are a Bad Idea (for most purposes) • John Ousterhout, (UC Berkeley, Sun Labs) • 2001: SEDA: An Architecture for Well-Conditioned, Scalable Internet Services • Staged, Event-driven Architecture • M. Welsh, D. Culler, and Eric Brewer (UC Berkeley) • 2003: Why Events are a Bad Idea (for high-concurrency servers) • R. van Behren, J. Condit, Eric Brewer (UC Berkeley)

4. Background HTTP processing • Problem: How do scale up servers to handle manysimultaneous requests • Server workloads have a lot of I/O, a little CPU Send HTTP request over network Read and parse HTTP request Read file from disk Invoke write syscall time Send response over network

5. Quick Example • Q: Suppose it takes 10 seconds for a web server to transfer a file to the client. Of this time, 10 milliseconds is dedicated to CPU processing. How many simultaneous requests do we need to keep the CPU fully utilized? • A: 1000

6. Concurrency Strategy #1: Thread-per-Request • Run each web request in its own thread • Assuming kernel threads, blocking I/O operations only stall one thread

7. These are blocking calls Review: Web Server Worker Thread • Read from socket to extract request URI (e.g., index.html) • Read desired file into a buffer • Write file over the socket • Close the socket while (true) { }

8. Concurrency Strategy #2: Event-driven Execution • Use a single thread for all requests • Use non-blocking I/O • Replace blocking I/O with calls that return immediately • Program is notified about interesting I/O events • This is philosophically similar to hardware interrupts • “Tell me when something interesting happens”

9. Event Loop Pseudocode while (true) { // Ask the OS for Sockets with active I/O; // this blocks if no socket is active Socket sock = getActiveSocket(); if (sock.isReadable()) handleReadEvent(sock); if (sock.isWriteable()) handleWriteEvent(sock); } In a pure event system, this is the only thread

10. Details: UNIX Select System Call • Select takes three arrays of file descriptors • Reads, writes, and exceptional events • Note that file descriptors may represent an open file or a socket • Select returns the subset of each array that is ready for reading, writing, or has an exceptional condition • Or, blocks if no FDs are active int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

11. Event Handler Pseudocode private Hashtable perSocketState; // this code *never* blocks public void handleRead(Socket sock) { // lookup state about this socket SocketState state = perSocketState.get(sock); state.bytesRead += sock.nonBlockingRead(state.buffer,state.bytesRead); // if we’ve read the entire request, send a response if (state.bytesRead >= REQUEST_LENGTH) { sendResponse(sock); } else {} // do nothing; wait for another event }

12. Can You Think of a System that Works This Way? • GUI frameworks (like Swing) • Single event-handling thread • User programs install “event listeners” to receive callbacks Event Loop Event Handlers

13. Why Events? • A potentially simpler programming model • No concurrency, locks, conditional variables, etc. • Less memory footprint • A single stack (at most, a stack per CPU) • More scalable?

14. Cut to Ousterhout

15. Critiquing Ousterhout • Events don’t handle true CPU concurrency • We can fix this with an event-loop per processor • But, then we have multiple threads… • Programming model becomes complex if event handlers do not run to completion • This requires “stack ripping” to maintain state

16. State Transition Diagram for the Web Server • In a threaded system, state transitions are maintained by the stack • In an event system, we must manually maintain the state of each request • “Stack ripping” Reading request Reading File Writing File

17. Andrew’s Opinion: Code Understanding • Event systems are difficult to read / understand • Which events happen in which order? while (true) { // Ask the OS for Sockets with active I/O; // this blocks if no socket is active Socket sock = getActiveSocket(); if (sock.isReadable()) handleReadEvent(sock); if (sock.isWriteable()) handleWriteEvent(sock); }

18. Threads vs Events in Practice • Both are used in production systems • Apache is multi-threaded / multi-process • Lighttpd is event-driven • Which is better? Why?

19. The Devil is in the (Implementation) Details • Many systems did not support kernel threads • Making events more attractive • Support for non-blocking I/O is uneven • e.g., UNIX/Linux do not support non-blocking file I/O • The UNIX select command has poor scalability (Banga et al, 1998) • But, more recent alternatives to select are much more scalable (Linux epoll) • Thread packages often do not scale (Welsh, 2001) • But, optimizations can vastly improve thread scalability • Van Behren 2003, Linux 2.6 and NPTL

20. So, What Can We Say For Sure? • Event-based systems are harder to program • Requires “stack ripping” • Harder to track control flow • Possible exception: run-to-completion event handlers (e.g., Java Swing GUIs) • Event systems use fewer resources • At most a stack per processor • Event systems require less synchronization • But, they still require some on multi-processor systems (which is every system!) • Open question: is this is any easier to program?

21. Software Engineering Issues • Usually, choice of concurrency model is made for “historical reasons” • Unfortunately, changing concurrency models is very difficult Use threads Begin project Use events