The Scala Experience Safe Programming Can be Fun!

1. The Scala ExperienceSafe Programming Can be Fun! Martin Odersky EPFL Lausanne, Switzerland

2. The .NET Context Visual Studio Debuggers, Profilers etc. .NET Common Language Runtime C# XML Libraries GUI Libraries, etc. Visual Basic F# ML Graphics Libraries System.I/O System.Net etc. Sockets etc. Database Connection Libraries

3. F# as a Language Common core language Core ML Core ML Modules-as- values, functors “OCaml-Objects” and other extensions Other extensions .NET Nominal Objects + tools + tools F# OCaml

4. Today • What is Pattern Matching Really? • Active Patterns • Related Issues and Related work

5. What is Pattern Matching • F# has four primary syntactic categories • Expressions • Declarations • Types • Patterns

6. Patterns are Everywhere letpat = expr Binding tryexpr with | pat -> expr | pat -> expr Exception Handling fun pat -> expr Function Values Sequence expressions let f pat ...pat = expr { forpatinexpr forpatinexpr letpat = expr ... -> expr } Function binding matchexprwith | pat -> expr | pat -> expr | pat -> expr Match expressions

7. Patterns are Everywhere { new type with member x.M1(pat,...,pat) = expr ... member x.MN(pat,...,pat) = expr } Object expressions type C(pat,...,pat) = class memberx.M(pat,...,pat) = expr ... memberx.M(pat,...,pat) = expr end Class definitions

8. What’s in a Pattern (pat, ..., pat) -- tuple pattern [pat, ..., pat] -- list pattern [| pat, ..., pat |] -- array pattern { id= pat, ..., id=pat } -- record pattern Point(pat, pat) -- data pattern pat | pat -- “either” pattern _ -- wild pattern x -- variable binding 36 -- constant pattern "36" -- constant pattern System.DayOfWeek.Monday -- constant pattern :? type -- type test pattern :? typeas id -- type test pattern null -- null test pattern

9. Patterns in Action Messages in a multi-threaded app /// Compute requests are sent to the threads. type Request = RequestBlockof view * int /// Response results are returned to the event loop type Response = | ResultBlockofint * ColorBlock | SkippedBlock Consume requests let compute (RequestBlock(view,width)) = ... Consume results match response with | ResultBlock (iframe,block) -> ... | SkippedBlock-> ...

10. Patterns In Action Fetching from fields/properties let fetch obj info = match info with | :? FieldInfoas f ->f.GetValue(obj) | :? PropertyInfoas p ->p.GetValue(obj,null) | _ ->null Exception catching let conflict = try db.SubmitChanges(); false with | :? OptimisticConcurrencyException->true

11. Patterns In Action let rewrite f expr = matchexprwith | HoleExpr _ | ConstExpr _ | VarExpr _ ->expr | AppExpr(x1,x2) -> let x1 = rewrite f x1 let x2 = rewrite f x2 AppExpr(x1, x2) | LambdaExpr(id, argvs, body) -> LambdaExpr(f id, argvs, rewrite f body) Term structure in a compiler

12. Redundancy & Incompleteness let f inp = matchinpwith | [x; y] -> x + y | [x] -> x C:\misc\test.ml(6,4): warning: FS0025: Incomplete pattern match. val f : int list -> int

13. Redundancy & Incompleteness let f inp = matchinpwith | [x] -> 2 * x | [y] -> y | _ -> 0 C:\misc\test.ml(6,4): warning: FS0026: This rule will never be matched val f : int list -> int

14. Patterns are incredibly useful... • A major part of the enduring appeal of typed functional languages • They are a fundamental programmer’s workhorse in F# code • However...

15. Patterns are incredibly bad... • Traditionally no pattern matching on abstract types • Hence encourage people to break abstraction boundaries • They are not extensible • Only one view on a “type” is allowed • Hence mostly effective for sophisticated analysis on term structures in internal implementations

16. But what is pattern matching? • Pattern matching = inferring a view to decompose a value and applying that view • View = take an input, categorize it and return residue data

17. Related Work • Views • Wadler 1986, Okasaki 1998 • Active discriminators, ad hoc patterns, unapply methods • Erwig 1996 (nb. called “active patterns”, have recycled the name) • Tullsen 2000 • F# 1.1, 2006 • Odersky & Emir 2006 • Jambon 2007 • Peyton-Jones proposal for Haskell, 2007 • ...

18. Active Patterns in F#

19. Active Patterns in F# These tags are “active recognizer labels” The whole function is an “active recognizer”. Two views on complex numbers let (|Rect|) (x:complex) = (x.Real, x.Imaginary) let (|Polar|) (x:complex) = (x.Magnitude , x.Phase) They are just ordinary functions with “banana names” letmulViaRect c1 c2 = match c1,c2 with | Rect(ar,ai), Rect(br,bi) -> CreateRect(ar*br - ai*bi, ai*br + bi*ar) letmulViaPolar c1 c2 = match c1,c2 with | Polar(r1,th1),Polar(r2,th2) -> CreatePolar(r1*r2, th1+th2) letmulViaRect (Rect(ar,ai)) (Rect(br,bi)) = CreateRect(ar*br - ai*bi, ai*br + bi*ar) letmulViaPolar (Polar(r1,th1)) (Polar(r2,th2)) = CreatePolar(r1*r2, th1+th2) The use of active recognizer labels implicitly select and apply the function (|Rect|) (|Polar|)

20. Complete Discriminations let (|Named|Array|ByRef|Ptr|Param|) (typ : System.Type) = iftyp.IsGenericTypethen Named(typ.GetGenericTypeDefinition(), typ.GetGenericArguments()) elif not typ.HasElementTypethen Named(typ, [| |]) eliftyp.IsArraythen Array(typ.GetElementType(), typ.GetArrayRank()) eliftyp.IsByRefthenByRef(typ.GetElementType()) eliftyp.IsPointerthenPtr(typ.GetElementType()) eliftyp.IsGenericParameterthenParam(typ.GenericParameterPosition, typ.GetGenericParameterConstraints()) elsefailwith"MSDN says this can't happen" type System.Type≈ | Named of System.Type * System.Type[] | Array of System.Type * int // rank | ByRef of System.Type | Ptr of System.Type | GenericParam of int * System.Type[] // constraints Multiple tags = View = Total Recognizer Tags can be used on RHS Much simpler and shorter than the if/then/else code letrectoStringtyp = matchtypwith | Named (con, args) ->"(" + con.Name + " ...)" | Array (arg, rank) ->"(Array " + " " + toStringarg + ")" | ByRefarg->"(ByRef " + toStringarg + ")" | Ptrarg->"(Ptr " + toStringarg + ")" | Param(pos,cxs) ->"Param" Note we maintain completeness and redundancy checking

21. Complete Discriminations (|Named|Array|ByRef|Ptr|Param|)

22. Partial Recognizers • Total recognizers never fail (except through exceptions and incomplete matches) • Partial recognizers intentionally leave failing cases Option values used to indicate success/failure let (|MulThree|_|) inp = if inp % 3 = 0 then Some(inp/3) else None let (|MulSeven|_|) inp = if inp % 7 = 0 then Some(inp/7) else None match 28 with | MulThree(residue) ->printf"residue = %d!\n" residue | MulSeven(residue) ->printf"residue = %d!\n" residue | _ ->printf"no match!\n"

23. Partial Recognizers • Partial Recognizers are most useful on “heterogeneous” or “very general” types • Strings • XML • Term structures • Abstracting adhoc queries on other types

24. Parameterized Partial Recognizers • It’s very useful if partial recognizers can take parameters • “Split a string at character N” • “Match any attribute A on an XmlNode” • “Match any LINQ Expression Tree involving a call to method M” let (|MulN|_|) n inp = if inp % n = 0 then Some(inp/n) else None match 28 with | MulN 3 residue ->printf"residue = %d!\n" residue | MulN 7 residue ->printf"residue = %d!\n" residue | _ ->printf"no match!\n"

25. Example: XML matching typeGlyphInfo = { bitmapID : int originX : int originY : int width : int height : int } <?xmlversion="1.0" encoding="utf-8" ?> <fontbase="20" height="26"> <bitmaps> <bitmapid="0" name="comic-0.png" size="256x256" /> </bitmaps> <glyphs> <glyphch=" " code="0020" bm="0" origin="0,0" size="1x27" aw="5" lsb="0" /> </glyphs> </font> let (|GlyphElem|_|) inp = matchinpwith | Elem "glyph" (Attributes (Attr"ch" (Char ch) & Attr"code" (NumHex code) & Attr"bm" (Num bm) & Attr"origin" (Pair (Num ox,Numoy)) & Attr"size" (PairX (Num sw,Numsh)) & Attr"aw" (Num aw) & Attr"lsb" (Num lsb))) -> Some {bitmapID = bm; originX = ox; originY = oy; width = sw; height = sh; advanceWidth = aw; leftSideBearing = lsb}

26. “Both” patterns (pat, ..., pat) -- tuple pattern [pat, ..., pat] -- list pattern [| pat, ..., pat |] -- array pattern { id= pat, ..., id=pat } -- array pattern Point(pat, pat) -- data pattern pat | pat -- “either” pattern pat & pat -- “both” pattern _ -- wild pattern x -- variable binding 36 -- constant pattern "36" -- constant pattern System.DayOfWeek.Monday -- constant pattern :? type -- type test pattern :? typeas id -- type test pattern null -- null test pattern

27. Example: XML matching // nb using a -?> b == (a -> b option) val ( |Child|_| ) : string -> #XmlNode -?> XmlElement val ( |Elem|_| ) : string -> #XmlNode -?> XmlNode val ( |Attr|_| ) : string ->XmlAttributeCollection -?> string val ( |Num|_| ) : string -?> int32 val ( |Float|_| ) : string -?> float val ( |NumHex|_| ) : string -?> int val ( |Char|_| ) : string -?> char val ( |Pair|_| ) : string -?> (int32 * int32) val ( |PairX|_| ) : string -?> (int32 * int32) valSelectChildren : (XmlNode -?> 'a) -> #XmlNode-> 'a list val ( |Attributes| ) : #XmlNode->XmlAttributeCollection val ( |ChildNodes| ) : #XmlNode->XmlNodeList

28. Example: Term Structures • Given type Expr = | ConstExpr of ExprConstInfo | VarExpr of ExprVarName | LambdaExpr of ExprVar * Expr | AppExpr of Expr * Expr

29. Example: Term Structures • Some Typical Active Patterns: let (|App1|_|) = functionAppExpr(ConstExpr(k),x) -> Some(k,x) | _ -> None let (|App2|_|) = functionAppExpr(App1(k,x1),x2) -> Some(k,x1,x2) | _ -> None let (|App3|_|) = functionAppExpr(App2(k,x1,x2),x3) -> Some(k,x1,x2,x3) | _ -> None let (|AppN|_|) = letrecqueryAcc e acc = match e with | AppExpr(f,x) ->queryAcc f (x::acc) | ConstExpr (k) -> Some(k,acc) | _ -> None in fun e ->queryAcc e []

30. Example: Term Structures • And more: let (|Cond|_|) = function App3(CondOp,e1,e2,e3) -> Some(e1,e2,e3) | _ -> None let (|Tuple|_|) = functionAppN(TupleMkOp(ty),e) -> Some(ty,e) | _ -> None let (|Equality|_|) = function App2(EqualityOp,e1,e2) -> Some(e1,e2) | _ -> None let (|Lambda|_|) = functionLambdaExpr(a,b) -> Some (a,b) | _ -> None let (|App|_|) = functionAppExpr(a,b) -> Some (a,b) | _ -> None let (|Lambdas|) e = qZeroOrMoreRLinear (|Lambda|_|) e let (|Apps|) e = qZeroOrMoreLLinear (|App|_|) e

31. Example: Term Structures • And more: /// Recognise the compiled form of “a && b” let (|LazyAnd|_|) x = match x with | Cond(Equality(Bool(true),x),y,Bool(false)) -> Some(x,y) | _ -> None let (|LazyOr|_|) x = match x with | Cond(Equality(Bool(true),x),Bool(true),y) -> Some(x,y) | _ -> None let (|BetaReducible|_|) x = match x with | Let((v,e),b) -> Some((v,e),b) | App(Lambda(v,b),e) -> Some((v,e),b) | _ -> None

32. Other examples • Matching on other term structures • LINQ Expressions (in progress) • Phoenix Compiler Expression Trees (in progress) • Lazy Lists • Mutation, but idempotent • Standard examples from the “views” literature • Join lists, Unzip, etc. etc.

33. Issues & Possible Extensions

34. Issues: Syntax/Resolution • Syntax • esp. for parameterized partial patterns • Name resolution • e.g. can you have (|A|B|) and (|A|C|) in scope? • (Yes, but lcan’t mix ‘n match) • Mixing total recognizers • e.g. can you mix A and C from (|A|B|) and (|C|D|)? • (No in the current implementation, but reconsidering this.) • Type checking and inference • Type inference: • Generalization: Recognizers are functions, so Hindley-Milner generalization applies as normal

35. Issues: Semantics • Naive Semantics • Run each rule separately, on failure go to next rule • Select active recognizers based on single labels • Assume “all active recognizers are idempotent” • Assume all active recognizers give equivalent results on equivalent inputs • Assume no side effects on subsequent executions against equivalent inputs • Hence can optimize • Places a semantic burden on the library designer

36. Issues: Possible Extensions • Multiple subsets in partial patterns? • Existentials • GADTs • Monadic Generalization let (|A|B|C|_|) inp = ... Reasonable, but NYI – perhaps never will be

37. Issue: Encoding shows through in types • F# type of total recognizer • Ocaml variants would be useful here: val (|Cons|Nil|) : 'a llist-> Choice<('a * 'a llist),unit> type Choice<'a,'b> = | Choice2_1 of 'a | Choice2_2 of 'b type Choice<'a,'b,'c> = | Choice3_1 of 'a | Choice3_2 of 'b | Choice3_3 of 'c etc. val (|Cons|Nil|) : 'a llist -> [ `Cons of ('a * 'a llist) | `Nil ]

38. Possible Extensions: Existentials? GADTs? • Existentials are a natural extension to pattern matching in languages with subtyping & generics • But what of active patterns? The natural encoding is to permit anonymous existentials on the right of active recognizers: • But what of GADTs? The natural encoding is to permit anonymous constrainedexistentials on the right of active recognizers: matchobjwith | <'a> :? List<'a> as l -> ... | <'a> :? 'a[] asarr-> ... | <'k,'v> :? Dictionary<'key,'value> -> ... match obj with | <'a> AnyList(l : 'a list) -> ... | <'a> AnyArray(arr : 'a[]) -> ... | <'k,'v> AnyDictionary(dict: Dictionary<'k,'v>) -> ... val (|AnyList|_|) : obj -?> ('a. 'a list) val (|Lambda|_|) : Expr<'a> -?> ('b 'c. ('a = 'b -> 'c) => Var<'b> * Expr<'c>)

39. Possible Extensions: Monadic Generalization • Generalize types of pattern functions using monads type Pattern<'a,'b> = 'a -> 'b option type Pattern<'M,'a,'b> = 'a -> 'M<'b> when M :> MonadPlus

40. Possible Extensions: Monadic Generalization • MonadPlus – the pattern matching monad trait MonadPlus<M> ~=~ { static member Return : 'a -> M<'a> static member Bind : M<'a> -> ('a -> M<'b>) -> M<'b> static member Zero : M<'a> static member Plus : M<'a> -> M<'a> -> M<'a> } instance MonadPlus<option> // deterministic evaluation instance MonadPlus<list> // backtracking evaluation instance MonadPlus<STM> // transactional evaluation

41. Possible Extensions: Monadic Generalization • Transactional pattern matching of lock-free data structures let f q = atomically $ match<STM>q with | stmCons (x, stmCons (y, ys)) ->stmQueue (x + y) ys | stmCons (_, xs) ->return xs implicit retry f q = atomically $ do { (x, xs) <- stmCons q; (y, ys) <- stmCons xs; stmQueue (x+y) ys } `orElse` do { (_, xs) <- stmCons q; return xs } desugaring

42. Implementation • F# uses Ramsay’s Generalized Pattern Match Algorithm with a simplistic Left-to-right Heuristic • Dumb but effective • Modification is straight-forward • Choose “first N relevant” edges, instead of “all” edges • Leave remaining edges unexplored • If any partial patterns are used we revert to rule-by-rule to avoid exponential blow up • Non left-to-right heuristics can still be used for matching on concrete data or if purity & termination is guaranteed

43. Performance • Recognizers are functions: many optimizations apply immediately • Currently an allocation on each non-total active pattern invocation • Using .NET structs for Choice and Option would mostly solve this • And no, we don’t yet use these in the F# compiler (which x-compiles with OCaml) • But they feel adequate for our target purposes

44. Summary

45. Summary • Pattern matching is simultaneous discrimination and decomposition • Active Patterns are in F# 1.9 (heading towards an “F# 2.0” in ~August) • It feels like we’ve reached a fairly stable design point • The feature has many good uses, but new active recognizers are best designed by experienced and trained programmers