Events Can Make Sense

1. Events Can Make Sense Max Krohn (MIT CSAIL), Eddie Kohler (UCLA), Frans Kaashoek (MIT CSAIL)

2. Straw Poll: How To Manage Concurrency In Network Applications? “Threads.”

3. Straw Poll: How To Manage Concurrency In Network Applications? “Events.”

4. As much as I love events, I admit they make code unreadable! True, “stack ripping” sucks!

5. Stack Ripping: Before [Adya et al. 02] int send_email(const char *email) { dnsname hostname = extract_hostname(email); ipaddr addr = gethostbyname(hostname); int fd = tcp_connect(addr); return do_smtp(fd, email); }

6. Stack Ripping: After void send_email_ev(const char *email, event<int> done) { dnsname hostname = extract_hostname(email); gethostbyname(email, wrap(send_email_2, email, done)); } void send_mail_2(const char *email, event<int> done, ipaddr addr) { tcp_connect(addr,wrap(send_email_3,email, done)); } void send_mail_3(const char *email, event<int> done, int fd) { do_smtp(fd,email, wrap(send_mail_4, done)); } void send_mail_4(int result, event<int> done) { done.trigger(result); }

7. Just use threads! But…

8. What’s Wrong with Threads? • Good at launch and then immediate block • Not as good at parallel launch: • Difficult to prevent a thread from blocking • Need to spawn new threads • “Stack-rip” to move state between stacks • Synchronize entry and exit • int rc = read(fd, buf, 1024);

9. Threads: n Serial Sends void send_serial(intresults[],constchar*emails[],int n) { for(int i = 0; i < n; i++) results[i] = send_email(emails[i]); }

10. Threads: n Parallel Sends typedefstruct_action_t{ constchar*email; intresult; pthread_tid; pthread_cond_t*cond; pthread_mutex_t*mut; int*count; }lookup_t; staticvoid* send_action(void*in) { action_t*a=in; a->result=send_email(a->email); pthread_mutex_lock(a->mut); (*a->count)--; pthread_cond_signal(a->cond); pthread_mutex_unlock(a->mut); returnNULL; } void send_parallel(intresults[], constchar*emails[], intn) { pthread_cond_tcond; pthread_mutex_tmut; action_t*actions=(action_t*) malloc(n*sizeof(action_t)); action_t*a; inti; intcount=n; pthread_cond_init(&cond,NULL); pthread_mutex_init(&mut,NULL); for(i=0;i<n;i++){ a=actions+i; a->email=emails[i]; a->result=&results[i]; a->cond=&cond; a->mut=&mut; a->count=&count; pthread_create(&a->id,NULL, send_action,(void*)a); } pthread_mutex_lock(&mut); while(count>0) pthread_cond_wait(&cond,&mut); pthread_mutex_unlock(&mut); free(actions); }

11. Contribution • Tame: new primitives for high-performance network programming • Explicit control over concurrency (as in events) • Good readability (as in threads) • Continues Adya et al.’s line of research [Usenix 2002] • Makes practical ideas from Haskell and Concurrent ML

12. Outline • Design • Implementation • Evaluation

13. Tame primitive: event<> • Creating a new event: • Passed from caller to callee: • Callee “triggers an event” to tell caller when to proceed: event<> e = mkevent(…); • callee(…,e,…); void callee(event<> e) { … e.trigger(); … }

14. Tame primitive: twait{…} • Execute all code in twait{…}. • Only continue past ‘}’ once all events created inside {…} have triggered: twait { … if(x) foo(mkevent()); … while(y) bar(mkevent()); … }

15. Tame Example 1 tamedvoid wait_then_print() { twait { timer(10, mkevent()); } printf(“done!”); }

16. Tame Example 2 tamedvoid f() { printf(“1”); twait { g(mkevent()); printf(“3”); } printf(“5”); } tamedvoid g(event<> done) { printf(“2”); twait { timer(10, mkevent()); } printf(“4”); done.trigger(); } (Some other part of the program...)

17. Stack Ripping: Before int send_email(constchar *email) { dnsname hostname = extract_hostname(email); ipaddr addr = gethostbyname(hostname); int fd = tcp_connect(addr); return do_stmp(fd, email); }

18. Stack Ripping: After tamedvoid send_email_tame(int *result, constchar *email, event<> done) { tvars { dnsname hostname; ipaddr addr; int fd; } hostname = extract_hostname(email); twait { gethostbyname_tame(&addr, hostname, mkevent()); } twait { tcp_connect_tame(&fd, hostname, mkevent()); } twait { do_smtp_tame(result, fd, email, mkevent()); } done.trigger(); }

19. Recall: n Serial Sends void send_serial(constchar*emails[],intresults[],int n) { for(int i = 0; i < n; i++) results[i] = send_email(emails[i]); }

20. Tame: n Serial Sends tamed void send_serial_tame (intresults[],constchar*emails[], int n, event<> done) { tvars { int i; } for(i = 0; i < n; i++) { twait { send_email_tame(&results[i], emails[i], mkevent()); } } done.trigger(); }

21. Tame: n Parallel Sends tamed void send_parallel_tame(intresults[],constchar*emails[], int n, event<> done) { tvars { int i; } twait { for(i = 0; i < n; i++) { send_email_tame(&results[i], emails[i], mkevent()); } } done.trigger(); }

22. twait{…} is Great, but Not Sufficient twait{ email 1 email 1 email 2 email 2 “wait for first” email 3 result 1 result 1 “wait for last” result 2 result 3 email 3 } twait{ email 4

23. New Primitive: rendezvous<> • When making an event, can associate it with an explicit rendezvous<>: • Wait for first event in rendezvous to trigger (or move past if a trigger is queued). • Inspired by condition variables rendezvous<>rv; event<>e= mkevent(rv); • twait(rv);

24. rendezvous<> Example void wait_then_print2() { tvars { rendezvous<> rv; } timer(10, mkevent(rv)); twait(rv); printf(“Done!”) }

25. Tame: w/n Windowed Sends tamedvoid send_windowed(int results[],const char *emails[], intn, int wsz,event<>done) { tvars{intlo(0),hi(0);rendezvous<>rv;} while(lo<n) if(hi<n&&hi-lo<wsz){ send_mail_ev(&results[hi],&emails[hi], mkevent(rv)); hi++; }else{ twait(rv); lo++; } done.trigger(); }

26. Review: Tame Primitives • event<T> • created via mkevent() • triggered via event::trigger(T t) • rendezvous<W> • between producers and consumers of event<>s • twait(rv); • Wait for an event in rv to trigger • twait{…} is semantic sugar based on twait(rv); • tvars{} • A closure that persists across twait points.

27. Other Features (See Paper) • Support for threads and blocking libraries. • Composableevent connectors for handling timeouts and cancellation. • Robust memory management

28. Features You Won’t Find • Support for exceptions • SMP Support • Tame applications run on SMPs as separate processes. • Note: libevent supports SMP

29. Outline • Design • Implementation • Evaluation

30. What Exactly is Tame? • C++ language extensions, implemented by preprocessor. • A C++ runtime library, including event loop • Thus, Tame is portable, compiler-independent • (Works with g++ 3.3 through 4.1.2 …)

31. Implementation (Source Translation) • tvars{}becomes generated structs • One per each tamed function • Rewrite closure variables as references tvars { int i; } int &i = _closure->i;

32. Source Translation (con’t) 3. Rewrite twait points as return+ label: 4. Tamed function entry: • If first time in, then initialize closure • Else, jump to reentry in closure. if(!rv.has_queued_trigger()) { rv.set_closure(_closure); _closure->reentry = &&reentry_5; return; } reentry_5: twait(rv);

33. Other Implementation Details • Built with the libasync libraries [Mazières 01] • Backwards compatible with legacy event code

34. Evaluation • Performance: Does Tame perform as well as events and threads? • Programmability: • Does Tame simplify event code? • Is it useful in practice?

35. Evaluation: Tame vs. Events • Tame has additional costs • allocate / manage closures • Yet, performance roughly equivalent • See paper

36. Evaluation: Tame vs. Threads • Pro: Tame runs in one stack • Saves physical + virtual memory • No need to tune stack size or thread pool size • Con: Tame objects (events, closures, rendezvous) are heap-allocated, put pressure on malloc

37. Benchmark: Tame vs. Capriccio [von Behren et al. 2003] • Capriccio: high-performance user-level thread package • Looks like events + multiple stacks to the OS • Looks like threads to the programmer • Trivial Web server called Knot (1350 LOC) • 1415 LOC once tamed • Tamed version of Knot versus original Knot • Experiment with 6 clients • 200 open connections per client

38. Tame vs. Capriccio: Results

39. Informal User Study • MIT Graduate class: build a distributed file system • Some students used libasync alone, others used Tame. • ~20% less code in source files • ~50% less code in header files

40. Tame In The “Real World” • OKWS: an event-based secure Web server • OkCupid: An online dating Web site built with OKWS + Tame

41. Related Work • Most Related: • Capriccio [von Behren et al. SOSP 03] • Adya et al. from USENIX 02 • Event libraries: libasync, eel, libevent, QT • Thread libraries: pthreads, NPTL, PTH, StateThreads • Functional Languages • Concurrent ML [Reppy] • Haskell [Classen in JFP 99] [Li + Zdancewic in PLDI 07] • Implementation techniques: Protothreads, Duff's Device, porch checkpointer

42. Tame Summary • Design: new primitives for handling concurrency • Solves “stack-ripping” • Makes quick work of common parallel/serial concurrency patterns • Gets semantic power from explicit event allocation and wait points • Implementation: simple translation + libraries • Portable and compiler-independent • Performance: good

43. How To Get • tame: • http://www.okws.org • tamer: • http://www.read.cs.ucla.edu/tamer (under GPLv2)