Symbolic Finite State Transducers: Algorithms and Applications

2. Symbolic Finite State Transducers:Algorithms and Applications MargusVeanes Pieter Hooimeijer Benjamin Livshits David Molnar NikolajBjørner

3. Symbolic Finite State Transducers:Algorithms and Applications MargusVeanes Pieter Hooimeijer Benjamin Livshits David Molnar NikolajBjørner

4. Formal languages are well-studied.

5. a*b+ a b b ✔ ✘ aaaa abb

6. POPL (2001–2011) Number of papers “automata”

7. What about transformation?

8. http://en.wikipedia.org/wiki/Osborne_1

10. Compute image: • Check properties: • Equivalence • Composition ✔ abb{baa} ✘ aaaa b/a a/b b/a

11. POPL (2001–2011) Number of papers “automata” “transducers”

12. Talk Outline Background Approach Case Studies

13. Background “Fast and Precise Sanitizer Analysis with Bek” Idea: Develop a language for commonly-used string transformations. Prove properties about those transfor-mations.

14. t := iter(cins)[b:= false;] {case (!b&&cin"['\"\\]"):b:= false;yield('\\', c);case (c=='\\'):b:= !b;yield(c); case (true): b:= false; yield(c); }; Gap b/a a/b b/a FSTs Code

15. domain-specific languages t := iter(cins)[b:= false;] {case (!b&&cin"['\"\\]"):b:= false;yield('\\', c);case (c=='\\'):b:= !b;yield(c); case (true): b:= false; yield(c); }; 1 b/a a/b b/a FSTs Code

16. domain-specific languages t := iter(cins)[b:= false;] {case (!b&&cin"['\"\\]"):b:= false;yield('\\', c);case (c=='\\'):b:= !b;yield(c); case (true): b:= false; yield(c); }; 1 2 b/a a/b more expressivetransducers b/a FSTs Code

17. domain-specific languages t := iter(cins)[b:= false;] {case (!b&&cin"['\"\\]"):b:= false;yield('\\', c);case (c=='\\'):b:= !b;yield(c); case (true): b:= false; yield(c); }; 1 2 b/a a/b more expressivetransducers b/a FSTs Code

18. Talk Outline Background Approach Case Studies

20. Symbolic Finite State Transducers Idea: • Equip transitions with formulae • Allow the use of any decidable theory

21. Definition Symbolic Finite State Transducer (SFT):

22. Symbolic Finite State Transducer (SFT): - states - start state - final states

23. Symbolic Finite State Transducer (SFT): - states - start state - final states

24. predicates output Symbolic Finite State Transducer (SFT): - states - start state - final states

25. Symbolic Finite State Transducer (SFT): • Background Theory: • predicates • label theory - states - start state - final states - transition

26. Example

29. Closure under composition SFT A  B SFT A SFT B in in out out Requirement:

30. Single-valued equivalence Definition: 1

31. Algorithm: • Construct 2-outputproduct transducer • Find conflicts (dft): • output length • output value Complexity: complexity of decision procedure number of rules

32. Key restriction: single-valuedness • Transducer A is single-valued if, for all inputs, A has at most one out-put. 1

33. Transducer A is single-valued if, for all inputs, A has at most one out-put. 1 Note: This definition permits non-determinism, e.g.: b/[] ... ... ... b/[]

36. idempotence subsumption equivalence commutativity ...

37. Talk Outline Background Approach Case Studies

39. Case Studies "b"'b' Location Privacy HTMLdecode MalwareFingerprinting Image Blurring

40. "b"'b' Location Privacy HTMLdecode MalwareFingerprinting Image Blurring

41. HTMLdecode "<" "&lt;" "&#60;" "&#0060;" Decode

42. The Task: Prove that HTMLdecodeis not idempotent The Metric: Running time "<" "&lt;" "&#60;" "&#0060;" Decode

43. "<" "&lt;" "&#60;" "&#0060;" Decode The Problem: Unicode defines 1,114,112 code points.

44. Three Participating Representations C# C# C# +REG +REG SFT (Eager) SFT+Registers (Eager) SFT+Registers (Lazy)

45. Transducer size () 6.6M maximum number of digits

46. C# C# C# +REG +REG SFT (Eager) SFT+Registers (Eager) SFT+Registers (Lazy)

47. Transducer size () 6.6M SFT SFT + Symbolic State Space 51 maximum number of digits

48. Idempotence Checking: Time SFT + REG(eager) SFT + REG(lazy) SFT 2 3 4 5 6 maximum number of digits

49. Talk Outline Background Approach Case Studies

50. Conclusion • Introduced Symbolic Finite State Transducersover any decidable background theory • Presented decidability and complexity results • Comes with a scalable and robust* implementation