1. The Scala ExperienceSafe Programming Can be Fun! Martin Odersky EPFL Lausanne, Switzerland

2. The .NET Context Visual Studio Debuggers, Profilers etc. .NET Common Language Runtime C# XML Libraries GUI Libraries, etc. Visual Basic F# ML Graphics Libraries System.I/O System.Net etc. Sockets etc. Database Connection Libraries

3. F# as a Language Common core language Core ML Core ML Modules-as- values, functors “OCaml-Objects” and other extensions Other extensions .NET Nominal Objects + tools + tools F# OCaml

4. Today • What is Pattern Matching Really? • Active Patterns • Related Issues and Related work

5. What is Pattern Matching • F# has four primary syntactic categories • Expressions • Declarations • Types • Patterns

6. Patterns are Everywhere letpat = expr Binding tryexpr with | pat -> expr | pat -> expr Exception Handling fun pat -> expr Function Values Sequence expressions let f pat ...pat = expr { forpatinexpr forpatinexpr letpat = expr ... -> expr } Function binding matchexprwith | pat -> expr | pat -> expr | pat -> expr Match expressions

7. Patterns are Everywhere { new type with member x.M1(pat,...,pat) = expr ... member x.MN(pat,...,pat) = expr } Object expressions type C(pat,...,pat) = class memberx.M(pat,...,pat) = expr ... memberx.M(pat,...,pat) = expr end Class definitions

8. What’s in a Pattern (pat, ..., pat) -- tuple pattern [pat, ..., pat] -- list pattern [| pat, ..., pat |] -- array pattern { id= pat, ..., id=pat } -- record pattern Point(pat, pat) -- data pattern pat | pat -- “either” pattern _ -- wild pattern x -- variable binding 36 -- constant pattern "36" -- constant pattern System.DayOfWeek.Monday -- constant pattern :? type -- type test pattern :? typeas id -- type test pattern null -- null test pattern

9. Patterns in Action Messages in a multi-threaded app /// Compute requests are sent to the threads. type Request = RequestBlockof view * int /// Response results are returned to the event loop type Response = | ResultBlockofint * ColorBlock | SkippedBlock Consume requests let compute (RequestBlock(view,width)) = ... Consume results match response with | ResultBlock (iframe,block) -> ... | SkippedBlock-> ...

10. Patterns In Action Fetching from fields/properties let fetch obj info = match info with | :? FieldInfoas f ->f.GetValue(obj) | :? PropertyInfoas p ->p.GetValue(obj,null) | _ ->null Exception catching let conflict = try db.SubmitChanges(); false with | :? OptimisticConcurrencyException->true

11. Patterns In Action let rewrite f expr = matchexprwith | HoleExpr _ | ConstExpr _ | VarExpr _ ->expr | AppExpr(x1,x2) -> let x1 = rewrite f x1 let x2 = rewrite f x2 AppExpr(x1, x2) | LambdaExpr(id, argvs, body) -> LambdaExpr(f id, argvs, rewrite f body) Term structure in a compiler

12. Redundancy & Incompleteness let f inp = matchinpwith | [x; y] -> x + y | [x] -> x C:\misc\test.ml(6,4): warning: FS0025: Incomplete pattern match. val f : int list -> int

13. Redundancy & Incompleteness let f inp = matchinpwith | [x] -> 2 * x | [y] -> y | _ -> 0 C:\misc\test.ml(6,4): warning: FS0026: This rule will never be matched val f : int list -> int

14. Patterns are incredibly useful... • A major part of the enduring appeal of typed functional languages • They are a fundamental programmer’s workhorse in F# code • However...

15. Patterns are incredibly bad... • Traditionally no pattern matching on abstract types • Hence encourage people to break abstraction boundaries • They are not extensible • Only one view on a “type” is allowed • Hence mostly effective for sophisticated analysis on term structures in internal implementations

16. But what is pattern matching? • Pattern matching = inferring a view to decompose a value and applying that view • View = take an input, categorize it and return residue data

17. Related Work • Views • Wadler 1986, Okasaki 1998 • Active discriminators, ad hoc patterns, unapply methods • Erwig 1996 (nb. called “active patterns”, have recycled the name) • Tullsen 2000 • F# 1.1, 2006 • Odersky & Emir 2006 • Jambon 2007 • Peyton-Jones proposal for Haskell, 2007 • ...

18. Active Patterns in F#

19. Active Patterns in F# These tags are “active recognizer labels” The whole function is an “active recognizer”. Two views on complex numbers let (|Rect|) (x:complex) = (x.Real, x.Imaginary) let (|Polar|) (x:complex) = (x.Magnitude , x.Phase) They are just ordinary functions with “banana names” letmulViaRect c1 c2 = match c1,c2 with | Rect(ar,ai), Rect(br,bi) -> CreateRect(ar*br - ai*bi, ai*br + bi*ar) letmulViaPolar c1 c2 = match c1,c2 with | Polar(r1,th1),Polar(r2,th2) -> CreatePolar(r1*r2, th1+th2) letmulViaRect (Rect(ar,ai)) (Rect(br,bi)) = CreateRect(ar*br - ai*bi, ai*br + bi*ar) letmulViaPolar (Polar(r1,th1)) (Polar(r2,th2)) = CreatePolar(r1*r2, th1+th2) The use of active recognizer labels implicitly select and apply the function (|Rect|) (|Polar|)

20. Complete Discriminations let (|Named|Array|ByRef|Ptr|Param|) (typ : System.Type) = iftyp.IsGenericTypethen Named(typ.GetGenericTypeDefinition(), typ.GetGenericArguments()) elif not typ.HasElementTypethen Named(typ, [| |]) eliftyp.IsArraythen Array(typ.GetElementType(), typ.GetArrayRank()) eliftyp.IsByRefthenByRef(typ.GetElementType()) eliftyp.IsPointerthenPtr(typ.GetElementType()) eliftyp.IsGenericParameterthenParam(typ.GenericParameterPosition, typ.GetGenericParameterConstraints()) elsefailwith"MSDN says this can't happen" type System.Type≈ | Named of System.Type * System.Type[] | Array of System.Type * int // rank | ByRef of System.Type | Ptr of System.Type | GenericParam of int * System.Type[] // constraints Multiple tags = View = Total Recognizer Tags can be used on RHS Much simpler and shorter than the if/then/else code letrectoStringtyp = matchtypwith | Named (con, args) ->"(" + con.Name + " ...)" | Array (arg, rank) ->"(Array " + " " + toStringarg + ")" | ByRefarg->"(ByRef " + toStringarg + ")" | Ptrarg->"(Ptr " + toStringarg + ")" | Param(pos,cxs) ->"Param" Note we maintain completeness and redundancy checking

21. Complete Discriminations (|Named|Array|ByRef|Ptr|Param|)

22. Partial Recognizers • Total recognizers never fail (except through exceptions and incomplete matches) • Partial recognizers intentionally leave failing cases Option values used to indicate success/failure let (|MulThree|_|) inp = if inp % 3 = 0 then Some(inp/3) else None let (|MulSeven|_|) inp = if inp % 7 = 0 then Some(inp/7) else None match 28 with | MulThree(residue) ->printf"residue = %d!\n" residue | MulSeven(residue) ->printf"residue = %d!\n" residue | _ ->printf"no match!\n"

23. Partial Recognizers • Partial Recognizers are most useful on “heterogeneous” or “very general” types • Strings • XML • Term structures • Abstracting adhoc queries on other types

24. Parameterized Partial Recognizers • It’s very useful if partial recognizers can take parameters • “Split a string at character N” • “Match any attribute A on an XmlNode” • “Match any LINQ Expression Tree involving a call to method M” let (|MulN|_|) n inp = if inp % n = 0 then Some(inp/n) else None match 28 with | MulN 3 residue ->printf"residue = %d!\n" residue | MulN 7 residue ->printf"residue = %d!\n" residue | _ ->printf"no match!\n"

25. Example: XML matching typeGlyphInfo = { bitmapID : int originX : int originY : int width : int height : int } <?xmlversion="1.0" encoding="utf-8" ?> <fontbase="20" height="26"> <bitmaps> <bitmapid="0" name="comic-0.png" size="256x256" /> </bitmaps> <glyphs> <glyphch=" " code="0020" bm="0" origin="0,0" size="1x27" aw="5" lsb="0" /> </glyphs> </font> let (|GlyphElem|_|) inp = matchinpwith | Elem "glyph" (Attributes (Attr"ch" (Char ch) & Attr"code" (NumHex code) & Attr"bm" (Num bm) & Attr"origin" (Pair (Num ox,Numoy)) & Attr"size" (PairX (Num sw,Numsh)) & Attr"aw" (Num aw) & Attr"lsb" (Num lsb))) -> Some {bitmapID = bm; originX = ox; originY = oy; width = sw; height = sh; advanceWidth = aw; leftSideBearing = lsb}

26. “Both” patterns (pat, ..., pat) -- tuple pattern [pat, ..., pat] -- list pattern [| pat, ..., pat |] -- array pattern { id= pat, ..., id=pat } -- array pattern Point(pat, pat) -- data pattern pat | pat -- “either” pattern pat & pat -- “both” pattern _ -- wild pattern x -- variable binding 36 -- constant pattern "36" -- constant pattern System.DayOfWeek.Monday -- constant pattern :? type -- type test pattern :? typeas id -- type test pattern null -- null test pattern

27. Example: XML matching // nb using a -?> b == (a -> b option) val ( |Child|_| ) : string -> #XmlNode -?> XmlElement val ( |Elem|_| ) : string -> #XmlNode -?> XmlNode val ( |Attr|_| ) : string ->XmlAttributeCollection -?> string val ( |Num|_| ) : string -?> int32 val ( |Float|_| ) : string -?> float val ( |NumHex|_| ) : string -?> int val ( |Char|_| ) : string -?> char val ( |Pair|_| ) : string -?> (int32 * int32) val ( |PairX|_| ) : string -?> (int32 * int32) valSelectChildren : (XmlNode -?> 'a) -> #XmlNode-> 'a list val ( |Attributes| ) : #XmlNode->XmlAttributeCollection val ( |ChildNodes| ) : #XmlNode->XmlNodeList

28. Example: Term Structures • Given type Expr = | ConstExpr of ExprConstInfo | VarExpr of ExprVarName | LambdaExpr of ExprVar * Expr | AppExpr of Expr * Expr

29. Example: Term Structures • Some Typical Active Patterns: let (|App1|_|) = functionAppExpr(ConstExpr(k),x) -> Some(k,x) | _ -> None let (|App2|_|) = functionAppExpr(App1(k,x1),x2) -> Some(k,x1,x2) | _ -> None let (|App3|_|) = functionAppExpr(App2(k,x1,x2),x3) -> Some(k,x1,x2,x3) | _ -> None let (|AppN|_|) = letrecqueryAcc e acc = match e with | AppExpr(f,x) ->queryAcc f (x::acc) | ConstExpr (k) -> Some(k,acc) | _ -> None in fun e ->queryAcc e []

30. Example: Term Structures • And more: let (|Cond|_|) = function App3(CondOp,e1,e2,e3) -> Some(e1,e2,e3) | _ -> None let (|Tuple|_|) = functionAppN(TupleMkOp(ty),e) -> Some(ty,e) | _ -> None let (|Equality|_|) = function App2(EqualityOp,e1,e2) -> Some(e1,e2) | _ -> None let (|Lambda|_|) = functionLambdaExpr(a,b) -> Some (a,b) | _ -> None let (|App|_|) = functionAppExpr(a,b) -> Some (a,b) | _ -> None let (|Lambdas|) e = qZeroOrMoreRLinear (|Lambda|_|) e let (|Apps|) e = qZeroOrMoreLLinear (|App|_|) e

31. Example: Term Structures • And more: /// Recognise the compiled form of “a && b” let (|LazyAnd|_|) x = match x with | Cond(Equality(Bool(true),x),y,Bool(false)) -> Some(x,y) | _ -> None let (|LazyOr|_|) x = match x with | Cond(Equality(Bool(true),x),Bool(true),y) -> Some(x,y) | _ -> None let (|BetaReducible|_|) x = match x with | Let((v,e),b) -> Some((v,e),b) | App(Lambda(v,b),e) -> Some((v,e),b) | _ -> None

32. Other examples • Matching on other term structures • LINQ Expressions (in progress) • Phoenix Compiler Expression Trees (in progress) • Lazy Lists • Mutation, but idempotent • Standard examples from the “views” literature • Join lists, Unzip, etc. etc.

33. Issues & Possible Extensions

34. Issues: Syntax/Resolution • Syntax • esp. for parameterized partial patterns • Name resolution • e.g. can you have (|A|B|) and (|A|C|) in scope? • (Yes, but lcan’t mix ‘n match) • Mixing total recognizers • e.g. can you mix A and C from (|A|B|) and (|C|D|)? • (No in the current implementation, but reconsidering this.) • Type checking and inference • Type inference: • Generalization: Recognizers are functions, so Hindley-Milner generalization applies as normal

35. Issues: Semantics • Naive Semantics • Run each rule separately, on failure go to next rule • Select active recognizers based on single labels • Assume “all active recognizers are idempotent” • Assume all active recognizers give equivalent results on equivalent inputs • Assume no side effects on subsequent executions against equivalent inputs • Hence can optimize • Places a semantic burden on the library designer

36. Issues: Possible Extensions • Multiple subsets in partial patterns? • Existentials • GADTs • Monadic Generalization let (|A|B|C|_|) inp = ... Reasonable, but NYI – perhaps never will be

37. Issue: Encoding shows through in types • F# type of total recognizer • Ocaml variants would be useful here: val (|Cons|Nil|) : 'a llist-> Choice<('a * 'a llist),unit> type Choice<'a,'b> = | Choice2_1 of 'a | Choice2_2 of 'b type Choice<'a,'b,'c> = | Choice3_1 of 'a | Choice3_2 of 'b | Choice3_3 of 'c etc. val (|Cons|Nil|) : 'a llist -> [ `Cons of ('a * 'a llist) | `Nil ]

38. Possible Extensions: Existentials? GADTs? • Existentials are a natural extension to pattern matching in languages with subtyping & generics • But what of active patterns? The natural encoding is to permit anonymous existentials on the right of active recognizers: • But what of GADTs? The natural encoding is to permit anonymous constrainedexistentials on the right of active recognizers: matchobjwith | <'a> :? List<'a> as l -> ... | <'a> :? 'a[] asarr-> ... | <'k,'v> :? Dictionary<'key,'value> -> ... match obj with | <'a> AnyList(l : 'a list) -> ... | <'a> AnyArray(arr : 'a[]) -> ... | <'k,'v> AnyDictionary(dict: Dictionary<'k,'v>) -> ... val (|AnyList|_|) : obj -?> ('a. 'a list) val (|Lambda|_|) : Expr<'a> -?> ('b 'c. ('a = 'b -> 'c) => Var<'b> * Expr<'c>)

39. Possible Extensions: Monadic Generalization • Generalize types of pattern functions using monads type Pattern<'a,'b> = 'a -> 'b option type Pattern<'M,'a,'b> = 'a -> 'M<'b> when M :> MonadPlus

40. Possible Extensions: Monadic Generalization • MonadPlus – the pattern matching monad trait MonadPlus<M> ~=~ { static member Return : 'a -> M<'a> static member Bind : M<'a> -> ('a -> M<'b>) -> M<'b> static member Zero : M<'a> static member Plus : M<'a> -> M<'a> -> M<'a> } instance MonadPlus<option> // deterministic evaluation instance MonadPlus<list> // backtracking evaluation instance MonadPlus<STM> // transactional evaluation

41. Possible Extensions: Monadic Generalization • Transactional pattern matching of lock-free data structures let f q = atomically $ match<STM>q with | stmCons (x, stmCons (y, ys)) ->stmQueue (x + y) ys | stmCons (_, xs) ->return xs implicit retry f q = atomically $ do { (x, xs) <- stmCons q; (y, ys) <- stmCons xs; stmQueue (x+y) ys } `orElse` do { (_, xs) <- stmCons q; return xs } desugaring

42. Implementation • F# uses Ramsay’s Generalized Pattern Match Algorithm with a simplistic Left-to-right Heuristic • Dumb but effective • Modification is straight-forward • Choose “first N relevant” edges, instead of “all” edges • Leave remaining edges unexplored • If any partial patterns are used we revert to rule-by-rule to avoid exponential blow up • Non left-to-right heuristics can still be used for matching on concrete data or if purity & termination is guaranteed

43. Performance • Recognizers are functions: many optimizations apply immediately • Currently an allocation on each non-total active pattern invocation • Using .NET structs for Choice and Option would mostly solve this • And no, we don’t yet use these in the F# compiler (which x-compiles with OCaml) • But they feel adequate for our target purposes

44. Summary

45. Summary • Pattern matching is simultaneous discrimination and decomposition • Active Patterns are in F# 1.9 (heading towards an “F# 2.0” in ~August) • It feels like we’ve reached a fairly stable design point • The feature has many good uses, but new active recognizers are best designed by experienced and trained programmers