Data Model Property Inference and Repair

1. Data Model Property Inference and Repair JaideepNijjar and Tevfik Bultan {jaideepnijjar, bultan}@cs.ucsb.edu Verification Lab Department of Computer Science University of California, Santa Barbara ISSTA 2013

2. Motivation • Web applications influence every aspect of our lives • Our dependence on web applications keep increasing • It would be nice if we can improve the dependability of web applications

3. Acknowledgement: NSF Support

4. Three-Tier Architecture Browser Web Server Backend Database

5. Three-Tier Arch. + MVC Pattern Browser Web Server Controller Views Model Backend Database

6. Model-View-Controller Pattern DB • Benefits of the MVC pattern: • Separation of concerns • Modularity • Abstraction Model View • MVC has become the standard way to structure web applications • Ruby on Rails • Zend for PHP • CakePHP • Struts for Java • Django for Python • … Controller User Response User Request

7. Data Model • Data model is the heart of the web application • It specifies the set of objects and the associations (i.e., relations) between them • Using an Object-Relational-Mapping the data model is mapped to the back-end datastore • Any error in the data model can have serious consequences for the dependability of the application

8. Our Data Model Analysis Approach Active Records Data Model (Schema + Constraints) Data Model + Inferred Properties Failing Properties Repair Generation Verification Model Extraction Property Inference Data Model Schema Inferred Properties Transitive Relations Orphan Prevention Delete Propagation

9. Outline • Motivation • Overview of Our Approach • Rails Data Models • Basic Relations • Options to Extend Relations • Formalization of Semantics • Verification Techniques • Property Inference • Property Repair • Experiments • Conclusions and Future Work

10. A Rails Data Model Example Role * class User < ActiveRecord::Base has_and_belongs_to_many :roles has_one :profile, :dependent => :destroy has_many :photos, :through => :profile end class Role < ActiveRecord::Base has_and_belongs_to_many :users end class Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroy has_many :videos, :dependent => :destroy, :conditions => "format='mp4'" end class Tag < ActiveRecord::Base belongs_to :taggable, :polymorphic => true end class Video < ActiveRecord::Base belongs_to :profile has_many :tags, :as => :taggable end class Photo < ActiveRecord::Base ... * User 1 1 * 0..1 1 * Profile Photo 1 1 format=.‘mp4’ * 1 Taggable Video * Tag

11. Rails Data Models • Data model analysis verification: Analyzing the relations between the data objects • Specified in Rails using association declarations inside the ActiveRecord files • Three basic relations • One-to-one • One-to-many • Many-to-many • Extensions to the basic relationships using Options • :through, :conditions, :polymorphic, :dependent

12. Three Basic Relations in Rails class User < ActiveRecord::Base has_one:profile end. class Profile < ActiveRecord::Base belongs_to:user end User • One-to-One . • One-to-Many 1 0..1 Profile class Profile < ActiveRecord::Base has_many:videos end. class Video < ActiveRecord::Base belongs_to:profile end Profile 1 * Video

13. Three Basic Relations in Rails class User < ActiveRecord::Base has_and_belongs_to_many:users end class Role < ActiveRecord::Base has_and_belongs_to_many:roles end • Many-to-Many User * * Role

14. Extensions to the Basic Relations • :through Option • To express transitive relations • :conditions Option • To relate a subset of objects to another class • :polymorphic Option • To express polymorphic relationships • :dependent Option • On delete, this option expresses whether to delete the associated objects or not

15. The :through Option class User < ActiveRecord::Base has_one :profile has_many :photos, :through => :profile end class Profile < ActiveRecord::Base belongs_to :user has_many :photos end class Photo < ActiveRecord::Base belongs_to :profile end Profile 0..1 1 * 1 Photo User 1 *

16. The :dependent Option class User < ActiveRecord::Base has_one :profile,:dependent => :destroy end class Profile < ActiveRecord::Base belongs_to :user has_many :photos,:dependent => :destroy end • :delete directly delete the associated objects without looking at its dependencies • :destroy first checks whether the associated objects themselves have associations with the :dependent option set and recursively propagates the delete to the associated objects User Profile Photo 1 1 0..1 *

17. Formalizing Data Model Semantics Formal data model: M = <S, C, D> • S: Data model schema • The sets and relations of the data model, e.g., { Photo, Profile, Role, Tag, Video, User} and the relations between them • C: Constraints on the relations • Cardinality constraints, transitive relations, conditional relations, polymorphic relations • D: Dependency constraints about deletions • Express conditions on two consecutive instances of a relation such that deletion of an object from the first instance leads to the other instance

18. Outline • Motivation • Overview of Our Approach • Rails Data Models • Verification Techniques • Bounded Verification • Unbounded Verification • Property Inference • Property Repair • Experiments • Conclusions and Future Work

19. Verification Overview Active Records Model Extraction Bound Properties Data model + properties bound Bounded Verification Unbounded Verification AlloyTranslator SMT-LIB Translator formula formula Alloy Analyzer SMT Solver instance or unsat or unknown Results Interpreter Results Interpreter instance or unsat Property Verified Property Failed + Counterexample Unknown

20. Sample Translation to Alloy class User < ActiveRecord::Base has_one :profile end class Profile < ActiveRecord::Base belongs_to :user end sig Profile {} sig User {} one sig State { profiles: set Profile, users: set User, relation: Profile lone -> one User }

21. Sample Translation to SMT-LIB class User < ActiveRecord::Base has_one :profile end class Profile < ActiveRecord::Base belongs_to :user end (declare-sort User) (declare-sort Profile) (declare-fun relation (Profile) User) (assert (forall ((p1 Profile)(p2 Profile)) (=> (not (= p1 p2)) (not (= (relation p1) (relation p2) )) ) ))

22. Property Inference: Motivation • Verification techniques require properties as input • Effectiveness depends on quality of properties written • Manually writing properties is time-consuming, error-prone, lacks thoroughness • Requires familiarity with the modeling language • We propose techniques for automatically inferring properties about the data model of a web application • Inference is based on the data model schema • A directed, annotated graph that represents the relations

23. Data Model Schema Example

24. Outline • Motivation • Overview of Our Approach • Rails Data Models • Verification Techniques • Property Inference • Orphan Prevention Pattern • Transitive Relation Pattern • Delete Propagation Pattern • Property Repair • Experiments • Conclusions and Future Work

25. Property Inference: Overview • Identify patterns in the data model schema graph that indicates that certain property should hold in the data model • Extract the data model schema from the ActiveRecords declarations • Search for the identified patterns in the data model schema graph • If a match is found, report the corresponding property

26. Orphan Prevention Pattern • For objects of a class that has only one relation: when the object it is related to is deleted but the object itself is not, such an object becomes orphaned • Orphan chains can also occur • Heuristic looks at all one-to-many or one-to-one relations to identify all potential orphans or orphan chains • Infers that deleting an object does not create orphans 0 1 . . . n . . .

27. Transitive Relation Pattern • Looks at one-to-one or one-to-many relations in schema • Finds paths of relations that are of length > 1 • If there is a direct edge between the first and last node of the path, infer that this edge should be transitive, i.e. the composition of the others 1 2 . . . n

28. Delete Propagation Pattern • Look at schema with all relations removed that are many-to-many or transitive • Remove cycles in graph by collapsing strongly connected components to a single node • Assign levels to all nodes indicating its depth in the graph • Root node(s), those with no incoming edges, are at level 0 • Remaining nodes are at level 1 more than the maximum of their predecessors • Propagate deletes if levels between nodes is one 1 1 level=0 2 2 3 level=1 c 4 level=2

29. Repair Generation • Check the inferred properties on the formal data model • If a property fails we can point out the option that needs to be set in the data model to make sure that the inferred property holds • For delete propagates and orphan prevention patterns: Set the dependency option accordingly to propagate the deletes • For transitive property: Set the through option accordingly to make a relation composition of two other relations

30. Repair Examples 1 class User < ActiveRecord::Base 2 has_one:preference, :conditions => "is_active=true”, 3 :dependent => :destroy 4 has_many:contexts, :dependent => :destroy 5 has_many:todos, :through => :contexts 6 end 7 class Preference < ActiveRecord::Base 8 belongs_to:user 9 end 10 class Context < ActiveRecord::Base 11 belongs_to:user 12 has_many:todos, :dependent => :delete 13 end 14 class Todo < ActiveRecord::Base 15 belongs_to:context 16 # belongs_to: user 17 has_and_belongs_to_many:tags 18 end 19 class Tag < ActiveRecord::Base 20 has_and_belongs_to_many:todos 21 end

31. Outline • Motivation • Overview of Our Approach • Rails Data Models • Verification Techniques • Property Inference • Property Repair • Experiments • Conclusions and Future Work

32. Experiments on Five Applications

33. A Social Networking Application • LovdByLess: A social networking application • Users can write blog entries • Users can comment on a friend’s blog entry • Friend deletes blog entry

34. A Social Networking Application • A friend writes a blog entry • User comments on the friend’s blog entry • Friend deletes the blog entry

35. A Failing Inferred Property • deletePropagatesproperty inferred for LovdByLess delete should propagate Blog Comment

36. A Todo List Application • Tracks: A todo list application • Todos can be organized by Contexts • Users can also create Recurring Todos • Delete the Context. Then edit the Recurring Todo.

37. A Failing Inferred Property • Data Model and Application Error:deletePropagates property inferred for Tracks delete should propagate Context RecurringTodo

38. False Positives • deletePropagates property inferred for FatFreeCRM • But in FatFreeCRM it is valid to have a contact not associated with any account delete should propagate Account Contact

39. False Positives • transitive property inferred for LovdByLess • Just not a transitive relation due to semantics of the application ForumTopic User ForumPost

40. Experiment Results

42. Conclusions and Future Work • It is possible to extract formal specifications from MVC-based data models and analyze them • We can automatically infer properties and find errors in real-world web applications • Most of the errors come from the fact that developers are not using the ActiveRecords extensions properly • This breaks the modularity, separation of concerns and abstraction principles provided by the MVC pattern • We are working on analyzing actions that update the data store • We are also investigating verifiable-model-driven development for data models

43. Related Work • Automated Discovery of Likely Program Invariants • Daikon [Ernst et al, ICSE 1999] discovers likely invariants by observing the runtime behaviors of a program • [Guo et al, ISSTA 2006]extends this style of analysis and applies it to the inference of abstract data types • We analyze the static structure of an extracted data model to infer properties • Static Verification of Inferred Properties • [Nimmer and Ernst, 2001] integrate Daikon with ESC/Java, a static verification tool for Java programs • We focus on data model verification in web applications

44. Related Work • Verification of Web Applications • [Krishnamurti et al, Springer 2006 ] focuses on correct handling of the control flow given the unique characteristics of web apps • Works such as [Hallé et al, ASE 2010] and [Han et al, MoDELS 2007] use state machines to formally model navigation behavior • In contrast to these works, we focus on analyzing the data model • Formal Modeling of Web Applications • WebAlloy [Chang, 2009]: user specifies the data model and access control policies; implementation automatically synthesized • WebML [Ceri et al, Computer Networks 2000]: a modeling language developed specifically for modeling web applications; no verification • In contrast, we perform model extraction (reverse engineering)

45. Related Work • Verification of Ruby on Rails applications • Rubicon [Near et al, FSE 2012] verifies the Controller whereas we verify the Data Model • Requires manual specification of application behavior, whereas we verify manually written properties • Limited to bounded verification • Data Model Verification using Alloy • [Cunha and Pacheco, SEFM 2009] maps relational database schemas to Alloy; not automated • [Wang et al, ASWEC 2006] translates ORA-SS specifications to Alloy, and uses the Analyzer to produces instances of the data model to show consistency • [Borbar et al, Trends 2005] uses Alloy to discover bugs in browser and business logic interactions

46. Related Work • Unbounded Verification of Alloy Specifications using SMT Solvers • [Ghazi et al, FM 2011], approach not implemented • More challenging domain since Alloy language contains constructs such as transitive closures • Specification and Analysis of Conceptual Data Models • [Smaragdakis et al ASE 2009, McGill et al ISSTA 2011] use Object Role Modeling to express data model and constraints • Focus is on checking consistency and producing test cases efficiently

47. Questions?