1. Automated Testing of Classes Alessandro Orso Politecnico di Milano - Georgia Institute of Technology http://www.cc.gatech.edu/~orso Ugo Buy University of Illinois at Chicago Mauro Pezzè Politecnico di Milano – Università di Milano Bicocca

2. Breaking of the encapsulation Breaking of the encapsulation Generation ofmessage sequences Generation ofmessage sequences Formal specs Formal specs Scripting Scripting Source code Source code Automated class testing Problems with class testing • Object = data + operations • Behavior of objects highly depends on their state • Encapsulation and Information hiding  the state is hidden • The overall goal is the automated, code-based generation of sequences of messages to exercise an object (i.e., an instance of a class)

3. Class and state:The CoinBox example void CoinBox::returnQtrs() { curQtrs=0; } void CoinBox::addQtr() { curQtr=curQtr+1; if (curQtr>1) allowVend=1; } void CoinBox::vend() { if (allowVend) { total=total+curQtrs; curQtrs=0; allowVend=0; } } allowVend=0; Class CoinBox { int total; int curQtrs; int allowVend; public: CoinBox() { total=0; allowVend=0; curQtrs=0; } void addQtr(); void returnQtrs(); void vend(); }; Fault:method returnQtrs does not reset variable allowVend Possiblefailure: CoinBox() addQtr() addQtr() returnQtr() vend()  free drink!

4. Steps of the technique • Data-flow analysis to identify “paths of interest” within the class • Symbolic execution to express the semantics of the class in terms of pre- and post-conditions • Automated deduction techniques to generate sequences of messages for the class that exercise those paths

5. 11 curQtrs=0 DEF 20a t=total+curQtrs USE Step I: Data flow analysis Step I Class CoinBox 18 vend() 18 vend() 13 addQtr() 1 CoinBox() 10 returnQtrs() 19 if(allowVend) 19 if(allowVend) 14a t=curQtrs+1 2 total=0 11 curQtrs=0 F T 14b curQtrs=t 20a t=total+curQtrs 3 allowVend=0 12 exit 12 exit 20b total=tmp 20b total=t 15 if(curQtrs>1) 4 curQtrs=0 T 16 allowVend=1 21 curQtrs=0 21 curQtrs=0 5 exit F 17 exit 22 allowVend=0 22 allowVend=0 Virtual Node Virtual Node 23 exit 23 exit Vend()

6. DU associations Step I DUA# variabiledefinitionuse • curQtrs CoinBox (4) addQtr (14a) • curQtrs CoinBox (4) vend (20a) • allowVend CoinBox (3) vend (19) • total CoinBox (2) vend (20a) • curQtrs returnQtrs (11) addQtr (14a) • allowVend vend (22) vend (19) • curQtrs returnQtrs (11) vend (20a) • curQtrs addQtr (14b) addQtr (14a) • curQtrs addQtr (14b) addQtr (15) • curQtrs addQtr (14b) vend (20a) • allowVend addQtr (16) vend (19) • total vend (20b) vend (20a) • curQtrs vend (21) addQtr (14a)

7. Step II: Symbolic execution Step II For each path in each method of the class: • Execution conditions:( curQtrs>0 ) • Relation between input and output variables:allowVend'=1, curQtrs'=curQtrs+1 • Set of defined attributesdef={curQtrs,allowVend} Limitations: • Approximations required for loops • Restrictions on the program constructs that symbolic execution can handle

8. (curQtrs>0) (curQtrs == 0) addQtr() addQtr() addQtr() curQtrs=curQtrs+1 curQtrs=curQtrs+1 curQtrs=curQtrs+1 if(curQtrs>1) if(curQtrs>1) if(curQtrs>1) T T T allowVend=1 allowVend=1 allowVend=1 F F F exit exit exit allowVend'=1, curQtrs'=curQtrs+1 curQtrs'=1 Conditions for the CoinBox example Step II addQtr (curQtrs>0)  allowVend'=1, curQtrs'=curQtrs+1 def={curQtrs,allowVend} (curQtrs==0)  curQtrs’=1 def={curQtrs}

9. Step III: Automated deduction Step III For each DU association <def,use, v>, identified with data flow analysis, build the method sequence: <constructor,...,mi,..., mD,...,mj,...,mU> Method that contains the definition of v Method that contains the use of v Def-clear path w.r.t. v

10. m (a>0) && (b>=3) m() m’ m’ if(a>0) T (b>=3) && (c > d) (a>0) && (c==d) F if(b<3) ... F T F ... exit The method invocation tree Step III • node m = a path in m and the associated pre-condition cond(m) • successorsm’ (of m) = all paths pjwhose post-conditions do not contradict cond(m) • cond(m’) is the conjunction of • cond(m), without clauses implied by thepost condition of pi • precondition ofpi • node m= a path in m and the associated pre-condition cond(m) m’ (c > d)  (a’ = 1) (c == d)  (b’ = 9) (c < d)  (b’ = 2)

11. Building the tree Step III • root = a path in mu that traverses the use • If possible, add mdto the tree and explore only the sub-tree rooted at md, else start exploring the paths that do not imply the condition of their predecessor • Once md has been added, the next goal is to add a constructor • Stop when: • Both md and a constructor have been added: test case • No more nodes can be added: infeasible DUA • Tree depth reaches a user-provided threshold : fail

12. use vend def (AllowVend!=0) returnQtrs addQtr addQtr returnQtrs (AllowVend!=0) (curQtrs == 0) &&(AllowVend!=0) addQtr addQtr vend (curQtrs > 0) &&(AllowVend == 0) (curQtrs>0) (curQtrs = 0) CoinBox constructor (true) Building the deduction tree for DUA #7 Step III var: curQtrs def: returnQtr node 11 use: vend node 20a (AllowVend!=0) returnQtrs (true)  curQtrs'=0 def={curQtrs} (curQtrs>0) addQtrs (curQtrs>0)  allowVend'=1 def={curQtrs,allowVend} curQtrs'=curQtrs+1 (curQtrs==0)  total'=total def={curQtrs} curQtrs'=1 returnQtrs (true)  curQtrs'=0 def={curQtrs} Test case:CoinBox(),addQtr(), addQtr(),returnQtr(),vend()

13. Characteristics of the technique • Focuses on problems related to the state(instance variables) • Generates test cases from the source code(no specification required) • Uses a stepwise automated approach(partial results can be used)

14. The experimental setting EDG C++Code C++ Parser IL Analyzer IL DataFlowAnalyzer Abstract Graph DF Tool Symbolic Executor DUAs DUAs +Pre/Post Conditions SICStus Message Sequences Automated Deduction

15. Summary and future work • Technique for automated generation of test data for classes • Combination of existing techniques in a new context: data-flow analysis, symbolic execution, and automated deduction • Useful intermediate and partial results • DUAs • Pre/Post • Subset of test cases • Results of the preliminary experiments are encouraging • A tool is under development (first prototype due by the end of the year) • Future work • Experimentation • Extension to non-primitive attributes

16. Questions?