Programming Paradigms for Concurrency

1. Programming Paradigms for Concurrency Lecture 10 Part III – Message Passing Concurrency

2. Message Passing Paradigms Two important categories of MP paradigms: • Actor or agent-based paradigms • unique receivers: messages are sent directly from one thread to another • Channel-based paradigms • multiple receivers: messages are sent to channels that are shared between threads

3. Channel-based Message Passing • threads perform local computations and synchronize via MP over channels • channels can be shared between threads(no unique receiver) • communication is typically synchronous (blocking send) • threads are anonymous • threads and channels are created dynamically B A A

4. A Brief History of Channel-Based Paradigms • Hoare 1978, 1985: Communicating sequential processes (CSP) • Milner 1980: Calculus of communicating systems (CCS) • May et al. 1985: Occam language • Reppy 1991: Concurrent ML • Milner, Parrow, Walker 1992: ¼ calculus • Gropp, Lusk, Skjellum 1994: Message passing interface (MPI) • Griesemer, Pike, Thompson 2009: Go language

5. Concurrent ML • higher-order concurrent language • extension of Standard ML • part of the SML of New Jersey language distribution: http://www.smlnj.org/ • first-class events: provides special event type and event combinators enables building protocol abstractions • threads and channels are garbage collected supports speculative communication

6. Simple Example: Reference Cells

7. Simple Example: Reference Cells signatureCELL = sig type 'a cell val cell : 'a -> 'a cell val read : 'a cell -> 'a val write : 'a cell -> 'a -> unit end cell x : creates new cell holding value x read c : retrieves value stored in cell c write c x: updates cell c to hold value x

8. Cell implementation • each cell has an associated thread holding the stored value • each cell encapsulates two channels to its associated thread for updating/retrieving the stored value

9. Cell Implementation reqCh?READ datatype 'a request = READ | WRITE of 'a datatype 'a cell = CELL of { reqCh : 'a request chan, replyCh : 'a chan } C1(x) C2(x) reqCh?WRITE(y) replyCh!x

10. Implementing read and write reqCh?READ fun read (CELL {reqCh, replyCh}) = (send (reqCh, READ); recvreplyCh) fun write (CELL {reqCh, ...}) x = send(reqCh, WRITE x) C1(x) C2(x) reqCh?WRITE(y) replyCh!x send : ('a chan* 'a) -> unit send (ch, x) : send value x over channel ch recv : 'a chan -> 'a recvch : receives a value from channel ch

11. Implementing cell reqCh?READ fun cell x = let valreqCh = channel () valreplyCh = channel () fun loop x = caserecvreqChof READ => (send (replyCh, x); loop x) | WRITE y => loop y in spawn (fn () => loop x); CELL{reqCh = reqCh, replyCh = replyCh} end C1(x) C2(x) reqCh?WRITE(y) channel : unit -> 'a chan channel () : creates a new channel replyCh!x spawn : (unit -> unit) -> unit spawn f : creates a new thread executing f

12. Selective Communication Allow a process to block on a nondeterministic choice of several blocking communications • the first communication that becomes enabled is chosen • if two or more communications are available simultaneously, then one is chosen nondeterministically.

13. Simplified Cell Implementation datatype 'a cell = CELL of { readCh : 'a chan, writeCh : 'a chan } fun read (CELL{readCh, ...}) = recvreadCh fun write (CELL{writeCh, ...}) = send (writeCh, x) C(x) readCh?y writeCh!x

14. Simplified Cell Implementation C(x) readCh?y writeCh!x fun cell x = let valreadCh = channel () valwriteCh = channel () fun loop x = select [ wrap (sendEvt(readCh, x), fn () => loop x), wrap (recvEvtwriteCh, loop) ] in spawn (fn () => loop x); CELL{readCh = readCh, writeCh = writeCh} end

15. First-Class Synchronous Events

16. First-Class Events Design principle for MP programming • use communication primitives to implement complex protocols • use abstraction to hide actual communication channels and thus details of thread interactions • implement abstract protocols Problem: standard abstraction mechanisms provided by sequential languages hide too much.

17. Example: RPC Server • Server provides service via a remote procedure call (RPC) protocol • RPC protocol: • a client must send a request before trying to read a reply • following a request, a client must read exactly one reply before issuing another request

18. Example: RPC Server fundoit request = ... (* the actual service *) • fun serverLoop () = • ifserviceAvailable () then • let valrequest = recvreqChin • send (replyCh, doit request); • serverLoop () • end • else doSomethingElse () • Problem: if some client does not respect the RPC protocol, then the server may get out of sync.

19. Example: RPC Server fun serverLoop() = ifserviceAvailable ()then let valrequest = recvreqChin send (replyCh, doit request); serverLoop () end else doSomethingElse () Solution: use procedural abstraction to hide protocol details on client side funclientCall x = (send (reqCh, x); recvreplyCh) • Problem: client blocks if service is not available • one would like to use selective communication in client, but procedural abstraction hides too much

20. First-Class Synchronous Events • Concurrent ML provides special event typetype ‘a event • Events decouple the description of synchronous operations from the actual act of synchronizing on the operation valsync : ‘a event -> ‘a • Event combinatorsenable construction of complex events that implement abstract protocols

21. Primitive Events valrecvEvt : ‘a chan -> ‘a event valsendEvt : ‘a chan * ‘a -> unit event recv and send are defined using these primitive events: funrecvch = sync (recvEvtch) funsend(ch, x) = sync (sendEvt(ch, x))

22. Post-Synchronization Actions: wrap valwrap : ‘a event * (‘a -> ‘b) -> ‘b event wrap (e, f) : returns an event that when synchronized, synchronizes e and applies f to the result

23. Selective Synchronization: choose valchoose : ‘a event list-> ‘a event choosees : returns an event that when synchronized, nondeterministically synchronizes one enabled event in es. funselectes = sync(choosees)

24. Simplified Cell Implementation (revisited) C(x) readCh?y writeCh!x fun cell x = let valreadCh = channel () valwriteCh = channel () fun loop x = select [ wrap (sendEvt(readCh, x), fn () => loop x), wrap (recvEvtwriteCh, loop) ] in spawn (fn () => loop x); CELL{readCh = readCh, writeCh = writeCh} end

25. Example: RPC Client (revisited) funclientCall x : ‘a -> ‘b option event = wrap (sendEvt (reqCh, x), fn () => SOME (recvreplyCh)) fun timeOut () : ‘b option event = let valto = Time.now + Time.fromSeconds 60 in • wrap (atTimeEvt to, fn () => NONE) end fun client request : ‘a -> ‘b option = select [clientCall request, timeOut ()]

26. Example: Swap Channels Swap channels enable symmetric communication between threads where values are exchanged in both directions. signatureSWAP_CHANNEL = sig type ‘a swap_chan valswapChannel : unit -> ‘a swap_chan val swap : (‘a swap_chan * ‘a) -> ‘a event end

27. Swap Channel Implementation (1st try) datatype ‘a swap_chan = SC of ‘a chan funswapChannel () = SC (channel ()) fun swap (SC ch, msgOut) = choose [ wrap (recvEvtch, fnmsgIn => (send(ch, msgOut); msgIn)), wrap (sendEvt (ch, msgOut), fn () => recvinCh) ]

28. Swap mismatch P1 P2 Q1 Q2 swap(ch, 1) swap(ch, 2) swap(ch, 3) swap(ch, 4) send(ch, 1) recvch send(ch, 3) recvch recvch send(ch, 2) recvch send(ch, 4) Race on shared channel!

29. Pre-Synchronization Actions: guard valguard : (unit -> ‘a event) -> ‘a event guard f: returns an event that when synchronized, first evaluates f and then synchronizes the resulting event

30. Swap Channel Implementation datatype ‘a swap_chan = SC of (‘a * ‘a chan) chan funswapChannel () = SC (channel ()) fun swap (SC ch, msgOut) = guard (fn () => let valinCh = channel () in choose [ wrap (recvEvtch, fn (msgIn, outCh) => (send (outCh, msgOut); msgIn)), wrap (sendEvt (ch, (msgOut, inCh)), fn () => recvinCh) ] end) fresh private channel for each swap event

31. Example: RPC Server (revisited) fun serverLoop() = let valrequest = recvreqChin send (replyCh, doit request); serverLoop () end • funclientCall x = • guard (fn () => send (reqCh, x); recvEvtreplyCh) • fun client request = • select [clientCall request, timeoutEvent ()]

32. Client Timeout T C1 S C2 recvreqCh send(reqCh,1) atTimeEvt doit 42 send(replyCh,42) send(reqCh,2) deadlock Server S and other clients might deadlock if client C1 times out

33. Cancellation: withNack valwithNack : (unit event -> ‘a event) -> ‘a event • When withNackf is used in a select, the select evaluates fwith a fresh abort event nack. • The resulting event fnack is then used with the other events in the select. • If the select does not choose fnack when it is synchronized, then nack is synchronized, effectively informing fnack that it is not chosen.

34. Example: RPC Server (revisited) fun serverLoop() = let val(request, nack) = recvreqChin select [sendEvt (replyCh, doit request),nack]; serverLoop () end • funclientCall x = withNack (fn nack => • send (reqCh, (x, nack)); recvEvtreplyCh) • fun client request = • select [clientCall request, timeoutEvent ()]

35. Client Timeout T C1 S C2 recvreqCh send(reqCh,(1,na)) atTimeEvt doit 42 na na send(reqCh,(2,na)) recvreqCh

36. More Features of Concurrent ML • more event combinators • buffered channels • multicast channels • sync variables • built-in RPC support • ...

37. Implementation of First-Class Events

38. Event Represenation Canonicalform of an event is a binary tree with: • wrapped base-event values as leaves • choice operators as internal nodes Canonical form is obtained using the following equivalences: wrap(wrap(ev,g),f) = wrap(ev, f ± g) wrap(choose(ev1,ev2),f) = choose(wrap(ev1,f), wrap(ev2,f))

39. Event Represenation choose choose choose wrap wrap wrap recv choose recv wrap wrap wrap wrap wrap wrap send recv send recv

40. Base Event Representation Each base event is a record of three functions: pollFn doFn blockFn test if base event is enabled synchronize on the enabled base event block calling thread on the base event

41. Protocol for sync Each call to sync from a PCL thread executes the following protocol: • Polling Phase: poll the base events using pollFn to see if any of them are enabled • Commit Phase: if one or more base events are enabled, pick one and synchronize on it using doFn • Blocking Phase: if no base events are enabled: • enqueue a continuation for the calling thread on each of the base events using its blockFn • switch to some other thread • eventually, some other thread will complete the synchronization.

42. First-Class Continuations type ‘a cont valcallcc : (‘a cont -> ‘a) -> ‘a val throw : ‘a cont -> ‘a -> ‘b

43. Event Representation datatypeevent_status = INIT | WAITING | SYNCHED typeevent_state = event_status ref datatype ‘a evt = BEVT of { pollFn : unit -> bool doFn : ‘a cont -> unit blockFn : (event_state * ‘a cont) -> unit } | CHOOSE of ‘a evt * ‘a evt

44. Implementing sync funsyncev = callcc (fnresumeK => let fun poll (ev, enabled) = ... fundoEvt enabled = ... andblockThd () = ... in doEvt (poll (ev, [])) end)

45. Implementing sync funsyncev = callcc (fnresumeK => let fun poll (BEVT{pollFn, doFn, ...}, enabled) = ifpollFn () thendoFn::enabled else enabled | poll (CHOOSE(ev1, ev2)), enabled) = poll(ev2, poll(ev1, enabled)) fundoEvt enabled = ... andblockThd () = ... in doEvt (poll (ev, [])) end)

46. Implementing sync funsyncev = callcc (fnresumeK => let fun poll (ev, enabled) = ... fundoEvt [] = blockThd () | doEvt (doFn::enabled) = doFnresumeK; (* if doFn terminates then sync attempt failed, so continue *) doEvt enabled andblockThd () = ... in doEvt (poll (ev, [])) end)

47. Implementing sync funsyncev = callcc (fnresumeK => let fun poll (ev, enabled) = ... fundoEvt enabled = ... andblockThd () = let valflg = ref INIT fun block (BEVT {blockFn, ...}) = blockFn (flg, resumeK) block (CHOOSE (ev1, ev2)) = (block ev1; block ev2) in block ev; flg := WAITING; dispatch() end in doEvt (poll (ev, [])) end)

48. Implementing wrap funwrap (BEVT {pollFn, doFn, blockFn}, f) = BEVT { pollFn = pollFn, doFn = fn k => callcc (fnretK => throw k (f (callcc (fn k’ => doFn k’; throw retK ()))))), blockFn = fn (flg, k) => callcc (fnretK => throw k (f (callcc (fn k’ => blockFn(flg, k’); throw retK ()))))) | wrap (CHOOSE(ev1, ev2), f) = CHOOSE(wrap(ev1, f), wrap(ev2, f))

49. Event Representation datatype ‘a chan = CH of { lock : bool ref, sendq : (‘a * unit cont) queue recvq : (event_state * ‘a cont) queue }

50. Implementing recvEvt funrecvEvt (CH {lock, sendq, recvq}) = let fun pollFn () = ... fundoFn k = ... funblockFn (flg : event_state, k) = ... in BEVT {pollFn = pollFn, doFn = doFn, blockFn = blockFn} end