More Flexible Software By Favoring Implicit Calls and Implicit Communication

1. More Flexible Software By Favoring Implicit Callsand Implicit Communication by Karl Lieberherr Northeastern University, CCIS/PRL Joint work with Bryan Chadwick, Ahmed Abdelmeged and Therapon Skotiniotis Bern CHOOSE 2007

2. Overview • Why is traversal abstraction important? • generalized data abstraction: What is AP? • expression problem • AP-F and AP-P by example • Container capacity checking • Translation tasks • Implementation: Demeter-F for AP-F and Demeter-P for AP-P • Conclusions Bern CHOOSE 2007

3. Important Flexible Software Topics To Which We Contribute with Demeter • Generalizing the Data Abstraction Problem • Contributing to Solving the Expression Problem • Better Abstractions for Traversals Bern CHOOSE 2007

4. Generalizing Data Abstraction well-accepted • Data Abstraction Principle: the implementation of objects can be changed without affecting clients provided the interface holds. • Adaptive Programming (AP) Principle: the interface of objects can be changed without affecting clients provided the Demeter interface holds. Bern CHOOSE 2007

5. Expression Problem • Expression Problem (named by Phil Wadler), which separates structure from computations performed on that structure, while still enabling modular extensions to both the structure and the computations. • We address the Expression Problem by practicing structure-shy programming: loosely couple structure and computations. • Supports modular extensions to both structure and computations. Bern CHOOSE 2007

6. Traversal Abstraction • Traversal of an object is fundamental to any computation on the object. • Express WhereToGo using a domain-specific language for navigation control (a-la XPath). • But traversals perform diverse computations: Interpretation and Translation. Bern CHOOSE 2007

7. Lack of Traversal Abstraction • leads to tight coupling between structure and computations. • hinders using the adaptive programming principle • hinders solving the expression problem Bern CHOOSE 2007

8. Traversal Abstraction • A new way of thinking about traversal abstraction. • Explain with Law of Demeter: Talk only to your friends. • New: Listen only to your friends. Your friends satisfy all your needs! Two ways: • AP-F: F for Functional: communicate through arguments. • AP-P: P for interPosition: communicate through variables. Bern CHOOSE 2007

9. What happens if you don’t use traversal abstraction? • You write a lot of boiler plate code! Related work: Scrap Your Boilerplate (SYB) started by Laemmel and Peyton-Jones. • You tightly couple structure (changes frequently) and computation (more stable). That is against common sense. • You always deal with the worst-case scenario of AP-F or AP-P. Bern CHOOSE 2007

10. Related work in Bern • Magritte Project (Lukas Renggli (Diplomarbeit), Stephane Ducasse, Adrian Kuhn) • Magritte supports end-user customization of meta models. Facilitates creation of infrastructure for changed meta models (editors, views, persistency tools). • AP supports customization of meta models including the application-specific code. Bern CHOOSE 2007

11. A Quick Intro to AP-F • An AP-F program is defined by three sets of functions which adapt behavior of a predefined traversal with a multiple dispatch function. The three sets of functions: • transformers • builders • augmentors Bern CHOOSE 2007

12. Intro to AP-F • Think of object computation as moving information down (augmentors) and pushing up (builders and transformers). • Allow transformation at each node (transformers). • Send information down as needed (augmentors). • Default behavior: copy object. Bern CHOOSE 2007

13. Motto of AP-F • Law of Demeter: Talk only to your friends. • Law of AP-F: Listen only to your friends and selectively receive information from your superiors. • You might not need all the information you get from your friends. You only take what you need and what the communication protocol requires. Bern CHOOSE 2007

14. Motto of AP-F augmentors A object graph Listen to C and D Receive from A B builders/ transformers Does NOT Receive from A C Receive from A D E Receive from A Bern CHOOSE 2007

15. DemeterF implements AP-F • Terminology map: AP-F to Java • transformers -- apply methods • take one argument • default: identity • builders – combine methods • several arguments (from your friends) • default: copy • augmentors – update methods • two arguments (where, what to pass down) • default: identity (return second argument) Bern CHOOSE 2007

16. * Item items Container cap: int Element weight: int Class DiagramNested Container Capacity Checking (CCC) Bern CHOOSE 2007

17. Illustration of combinefor capacity constraint violation :C (w1+w2+w3+w4,2) :Cons (w1+w2+w3+w4,1) :E (w4,0) (w1+w2+w3,1) :Cons :C (w2+w3,1) :Cons (w1,0) (w2+w3,0) :Cons (0,0) :E (w1,0) :Empty :E (w3,0) :Cons (w2,0) both containers (C) violate capacity constraints (w2,0) :E :Empty (0,0) Bern CHOOSE 2007

18. Illustration of combinefor capacity constraint violation 19a 19 :C (w1+w2+w3+w4,2) :Cons (w1+w2+w3+w4,1) :E (w4,0) 1 20 2 18a 3 18 3a 13a (w1+w2+w3,1) :Cons after blue arrow combine is active (like after) 17a :C (w2+w3,1) 4 12a 17 :Cons (w1,0) 5 13 14 12 16a (w2+w3,0) :Cons 15a 6 (0,0) :E (w1,0) :Empty 11a 7a 15 16 11 :E (w3,0) :Cons (w2,0) both containers (C) violate capacity constraints 8 7 10a 9a (w2,0) :E :Empty (0,0) 9 Bern CHOOSE 2007 10

19. AP-F: Listen through arguments | AP-P: Listen through variables Container-specific code for CCC ManualTraversal Pair check(){ Pair p = items.check(); return p.add(0,(p.w > cap)?1:0); } Pair combine(Container c, Integer i, Pair wv) { return wv.add(0,(wv.w > c.cap)?1:0); } void after(Container c) { myWeight += c.myWeight; totalViolations+= ((c.cap<c.myWeight)?1:0); } TALK TO YOUR FRIENDS AP-F AP-P LISTEN TO YOURFRIENDS Bern CHOOSE 2007

20. AP-F for CCC in DemeterF class Check extends IDb{ // generic combines for passing up pairs Pair combine(Object o, Pair f, Pair r) // pair addition {return f.add(r.w, r.v);} Pair combine(Object o, Pair f) {return f;} // pass it up Pair combine(Object o) { return Pair.make(0,0); } // init // container capacity checking specific combines Pair combine(Element e) {return Pair.make(e.weight,0);} Pair combine(Container c, Integer i, Pair wv) { return wv.add(0, (wv.w>c.cap)?1:0); } } Bern CHOOSE 2007

21. Calling the Function Object Pair check(Item i) { return new Traversal(new Check()).traverse(i); } Bern CHOOSE 2007

22. items * Item noise5 Noise3 noise4 Noise1 Noise2 OptItem noise3 noise0 noise1 noise2 Element weight: int Container cap: int Noise4 EmptyItem Class Diagram with Noise for CCC Noise is very common because the objects are used for several purposes. 0 outgoing has-a edges 1 outgoing has-a edge 2 outgoing has-a edges Bern CHOOSE 2007

23. * Item items Container cap: int items Element weight: int * Item noise5 Noise3 noise4 Noise1 Noise2 OptItem noise3 noise0 noise1 noise2 Element weight: int Container cap: int Noise4 EmptyItem Any program change? class Check extends IDb{ // generic combines for passing up pairs Pair combine(Object o, Pair f, Pair r) // pair addition {return f.add(r.w, r.v);} Pair combine(Object o, Pair f) {return f;} // pass it up Pair combine(Object o) { return Pair.make(0,0); } // container capacity checking specific combines Pair combine(Element e) {return Pair.make(e.weight,0);} Pair combine(Container c, Integer i, Pair wv) { return wv.add(0, (wv.w>c.cap)?1:0); } } Bern CHOOSE 2007

24. items * Item noise5 Noise3 noise4 Noise1 Noise2 OptItem noise3 noise0 noise1 noise2 Element weight: int Container cap: int Noise4 EmptyItem Any Change needed to CCC program? default operation: 1: send up: Pair combine(Object o, Pair f) { return f; } 2: add: Pair combine(Object o, Pair f, Pair r) { return f.add(r.w, r.v); } 3: init: Pair combine(Object o) { return Pair.make(0,0); } 4: problem specific Bern CHOOSE 2007

25. Benefit of Traversal Abstraction NO CHANGE! although meta model changed significantly. Bern CHOOSE 2007

26. Quick Intro to AP-P • AP-P is concerned about achieving the most structure-shy yet reusable form of communication among different pieces of traversal advice. • The approach proposed by AP-P for structuring communication is to assign a set of interposition variables in the visitor class to store information related to a specific type of objects. Bern CHOOSE 2007

27. Quick Intro to AP-P • Pieces of traversal advice communicate through interposition variables which give access to the innermost enclosing object of a given class. • An important benefit: communication can take place even across different visitors. • An interposition variable is a context sensitive variable. Its context is defined by the traversal. Bern CHOOSE 2007

28. Intro to AP-P • Think of computation as traversing the object and collecting information in a visitor object. • Use both regular visitor variables as well as interposition variables. • Interposition variables facilitate implicit communication. • Default behavior: traverse object. Bern CHOOSE 2007

29. CCC for AP-P in DemeterP class Checker extends Visitor{ public int totalViolations = 0; @Interposition (className = "Container") public int myWeight = 0; public void after(Element e) { myWeight += e.weight; } public void after(Container c) { myWeight += c.myWeight; totalViolations+=((c.cap<c.myWeight)?1:0); } public Pair getReturnValue() {return new Pair(myWeight, totalViolations);} } Pair check(Item i) { return cg.traverse(i, "from Container to Element",new Checker());} Bern CHOOSE 2007

30. Current Traversal Abstraction Restriction • Abstract traversal code • non-cyclic objects • explicit and implicit cyclic objects cause problems Bern CHOOSE 2007

31. Module = Interface + Implementation + DemeterInterface uses (imports) module M2 module M1 faceM2 mentM2 AdapConM1 faceM1 mentM1 uses (imports) faceM1 mentM1’ module M1 faceM1’ mentM12 faceM1 mentM1’’ faceM1’’ mentM13 change implementation change interface DemeterInterface(M1) Bern CHOOSE 2007

32. What Hinders Creation of Flexible Software • Manually following rules like: Follow the structure, follow the grammar. • Actively call traversal methods (explicit traversal problem). • Also leads to manual passing of arguments (explicit argument problem). Bern CHOOSE 2007

33. Type Unifying: information flow T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D* T apply(D d) E* T apply(E e) Bern CHOOSE 2007

34. Type Unifying: information flow T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) E T apply(E e) Bern CHOOSE 2007

35. Type Unifying: information flow T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) E T apply(E e) Bern CHOOSE 2007

36. Type Unifying: information flow T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) E T apply(E e) Bern CHOOSE 2007

37. Type Unifying: information flow T combine(A a, T t) A object graph T apply(T t) T combine(B b, T t1, T t2) B T apply(T t) T combine(C c, T t1) C D T apply(D d) E T apply(E e) Bern CHOOSE 2007

38. semantics: apply-combine expression Tree = <n> Node [<left> Tree] [<right> Tree]. Node = [IdentityApply] <o> Object. apply(combine(this, apply(combine(left,…)), apply(combine(right,…)) )) Translate tree object into apply-combine expression. Bern CHOOSE 2007

39. Still follow the grammar? • Might have to write customized methods for every node. Extreme case. • Encode local information about structure in personalized methods. Bern CHOOSE 2007

40. Simple applicationProgram transformation • Old • E : Num | Var | Op | Call … • Op : Plus | Equals. • Equals = “=“. • New • E : … | Bool. • Bool : True | False. class BoolTrans extends IDf { static E newtrue = Call.parse(“(= 1 1)”), static E newfalse= Call.parse(“(= 1 0) “); E apply(True t) {return newtrue; } E apply(False t) {return newfalse; } } apply for transformation of result returned by builder Bern CHOOSE 2007

41. for later de Bruijn indices • Old • Var : Sym. • Sym = Ident. • New • Var : Sym | Addr. • Addr = Integer. class AddrTrans extends IDf { Var apply(Var var, SymList senv) { return new Addr(senv.lookup(var));} } class SymExtender extends IDa { SymList update(Lambda l, SymList senv) { return senv.push(l.formals); } } Bern CHOOSE 2007

42. Related Work • Modularity First: A Case for Mixing AOP and Attribute Grammars, Pavel Avgustinov, Torbjorn Ekman, Julian Tibble, Programming Tools Group, University of Oxford, AOSD 2008 • SYB home page • AP home page Bern CHOOSE 2007

43. Conclusions • Rely more on your friends. Listen to them on the agreed upon communication channels like your Facebook wall. • Don’t tightly couple structure (volatile) and computation (more stable). • Use AP-F and AP-P ideas as your design notations. To execute your designs: DemeterF home page. Bern CHOOSE 2007

44. Operation of AP-F Bern CHOOSE 2007

45. AP-F for SDG • AP-F: • think of generic propagation first • add specific processing Bern CHOOSE 2007

46. generic propagation in SDG • push down predicate, retrieve an object satisfying predicate. • push down assignment, combine integer or formula. • push down predicate. Bern CHOOSE 2007

47. Design space for Demeter-F programs ID use builders and transformers Bern CHOOSE 2007

48. managing software development • Learn by evolving a software product: an algorithmic financial derivative trading game • Learn by working in a small team using pair programming • Learn by integrating software from the teams Bern CHOOSE 2007

49. SDG Features • outcome determined by owner/buyer • type • outcome • raw material (owner) • finished product (buyer) • pay (quality) • outcome determined by environment • type • outcome • pay Bern CHOOSE 2007

50. SDG Features • outcome determined by owner/buyer • type T • set of relations of rank k • exactly rank k, at most rank k. • size of set, e.g., 2. • use a predicate to constrain permitted relations, e.g. only OR relations -> CNF • outcome • raw material (owner) • CSP(T) formula • finished product (buyer) • assignment to variables of CSP(T) formula. • pay (quality) • satisfaction_ratio • all-or-nothing: satisfaction_ratio > price:1:0 • outcome determined by environment • type • outcome • pay Bern CHOOSE 2007