Analyzing Goal Models – Different Approaches and How to Choose Among Them

1. Analyzing Goal Models – Different Approaches and How to Choose Among Them Jennifer Horkoff1 Eric Yu2 1Department of Computer Science2Faculty of Information University of Toronto, Canada SAC’11 RE Track

2. Goal-Oriented Requirements Engineering (GORE) • GORE has received much attention in RE research as a means of: • Understanding the motivations for system requirements • Helping to ensure that the right system is built • Generally, GORE frameworks allow for: • Representation of stakeholder goals • Goals may be assigned to an agent (stakeholder or system) • Goals may have relationships to other goals, often describing achievement • Several goal modeling frameworks • KAOS, GBRAM, AGORA, NFR, i*, Tropos, GRL, … Analyzing Goal Models, Horkoff & Yu

3. Goal-Oriented Requirements Engineering (GORE) Example: Counseling Organization i* Model (Horkoff & Yu, 2009) We can analyze the contents of goal models systematically… Analyzing Goal Models, Horkoff & Yu

4. Example: Qualitative, Interactive Forward Satisfaction Analysis of Goal-Oriented Models What is the effect of using a Cybercafe/ Portal/ Chat Room?” Full Satisfaction Full Denial Horkoff, Yu: Evaluating Goal Achievement in Enterprise Modeling Analyzing Goal Models, Horkoff & Yu

5. Models and Analysis become Complex Analyzing Goal Models, Horkoff & Yu

6. Goal Model Analysis • Many different analysis techniques for goal models have been introduced: • Propagate satisfaction values through the model • Measure metrics over the model • Apply planningtechniques • Run simulations • Perform checks over models • Abundance of approaches is encouraging from a research perspective, but… • From a user or practitioner perspective can be confusing • What are the differences? • When would I use one and not another? • Limits adoption Analyzing Goal Models, Horkoff & Yu

7. Motivating Questions • Survey of methods • What methods are available? • What types of analysis questions can these methods answer? • What types of goal modeling constructs do the procedures support? • What information is needed in order to use the methods? • Analysis benefits • What are some of the potential benefits of goal model analysis in the requirements process? • Mapping and Selection • What available methods can be applied to achieve which kinds of usage objectives? • How can we use this information to advise on selection? Analyzing Goal Models, Horkoff & Yu

8. Survey of Goal Model Analysis Procedures Summary Dimensions Procedures Analyzing Goal Models, Horkoff & Yu

9. A Survey of GORE Analysis Techniques • Summarize results over the following points: • Algorithm Approach • Satisfaction Forwards, Backwards, Human Intervention, Metrics, Planning, Simulation, and Model Checking • Format of analysis results • Qualitative ( ), quantitative (0.37), binary (T/F) • Goal-oriented concepts supported (beyond AND/OR) • Dependencies, softgoals, contribution links • Additional information required beyond typical goal model constructs • e.g., priority, probabilities, events, delegations, and trust. Analyzing Goal Models, Horkoff & Yu

10. Satisfaction Analysis • Example Analysis Questions: • What is the effect of this alternative? • Can this goal be satisfied? • Evaluates the satisfaction or denial of goals given a functional or design alternative • Values are propagated forward or backward throughout the model • Qualitative or quantitative approaches • Techniques take different approaches to resolving multiple values for incoming goals: • Adding evidence, combine using probabilistic rules, separate evidence, fixed rules, human judgment Analyzing Goal Models, Horkoff & Yu

11. Satisfaction Analysis Analyzing Goal Models, Horkoff & Yu

12. Metrics • Example Analysis Questions: • How secure is the system represented by the model? • How risky is a particular alternative for a stakeholder? • Structural properties of the model and construct classifications are used to calculate metrics • Example: counts of dependency classifications (instance, model, duplicate, hidden) in a Strategic Dependency (SD) • Metrics often represent non-functional requirements • Examples: predictability, security, privacy, accuracy, etc. • They can also represent model properties: • Examples: completeness, consistency and correctness • Metrics can be local or global Analyzing Goal Models, Horkoff & Yu

13. Metrics Analyzing Goal Models, Horkoff & Yu

14. Planning • Example Analysis Questions: • What actions must be taken to satisfy goals? • What are the best plans according to certain criteria? • Work has applied AI-type planning to find satisfactory sequences of actions in models • Requires definition of axioms that express possible goal decompositions and delegations • Expresses the capabilities of actors in a model • A planner finds a delegation of goals to actors which fulfills model goals • Plans are evaluated by some criteria Analyzing Goal Models, Horkoff & Yu

15. Planning Analyzing Goal Models, Horkoff & Yu

16. Simulation • Example Analysis Questions: • What happens when an alternative is selected? • Are there unexpected properties in a simulation? • Adds temporal information including pre- and post- conditions to models • Translated to ConGolog (situation calculus) programs for simulation • Extensions simulate confidence, trust and distrust Analyzing Goal Models, Horkoff & Yu

17. Simulation Analyzing Goal Models, Horkoff & Yu

18. Model Checking • Example Analysis Questions: • Is it possible to achieve a particular goal? • Is the model consistent? • Models are expanded/converted to a temporal formalism • Includes expressions of creation, fulfillment and invariant properties • First order temporal logic statements are used to represent desired constraints • Model checker is used to validate properties and check for consistency • Further work adds in checks for security and trust Analyzing Goal Models, Horkoff & Yu

19. Model Checking Analyzing Goal Models, Horkoff & Yu

20. Tabular Summary Analyzing Goal Models, Horkoff & Yu

21. Information Required by each Procedure Analyzing Goal Models, Horkoff & Yu

22. Goal Model Analysis Objectives • Using capabilities of techniques in our survey, as well as our own experience in modeling and analysis, we list categories of objectives for goal model analysis • List is likely not complete • Objective Categories (goal model analysis can help…): • Understand the domain • Communicate • Improve the model • Make scoping decisions • Prompt requirements elicitation • Improve requirements • Design a system Analyzing Goal Models, Horkoff & Yu

23. Mapping Procedures to Objectives • We have made suggestions concerning what procedures may map to what objectives • Each mapping can be considered as a hypothesis • We have included guiding questions with each objective to help motivate the mapping and guide users • Example mappings: • Downloadable interactive mapping table: www.cs.utoronto.ca/~jenhork/GOREAnalysisSelectionTable.zip Analyzing Goal Models, Horkoff & Yu

24. Mapping of Procedures to Objectives Objectives Procedures Analyzing Goal Models, Horkoff & Yu

25. Guideline Usage Examples • Example: Online Counseling Domain (Horkoff & Yu, 2009) • Online counseling alternatives: text messaging or chat room? • Apply guiding questions… • High degree of social interaction (QU1) • Do no yet understand details, not yet confident in the accuracy and completeness of models (Qu2, QM1) • Communication is important, scoping is challenging (QS1, QU2) • Etc… • Recommendations: • Interactive, agent-oriented techniques for forward satisfaction analysis supporting softgoals • Analysis for anonymity or privacy with the same techniques or with GRL Satisfaction Analysis, and/or i* Metrics • If the required detailed information is available, apply planning and/or simulation techniques Analyzing Goal Models, Horkoff & Yu

26. Conclusions • First step towards making goal model analysis techniques more accessible to modelers • Enable potential users to user their knowledge of the domain and analysis objectives to select one or more procedures • We have attempted to be neutral in our analysis • Each procedure has unique abilities • Future work is needed to undertake studies to validate or refute the claims made by our guidelines • Hope that guidelines will be expanded and refined as more application experiences are available Analyzing Goal Models, Horkoff & Yu

27. Thank you • www.cs.utoronto.ca/~jenhork • jenhork@cs.utoronto.ca • www.cs.utoronto.ca/~eric • yu@ischool.utoronto.ca Analyzing Goal Models, Horkoff & Yu

28. Outline Goal Models Goal Model Analysis Motivation: Abundance of Approaches Survey of Goal Model Analysis Approaches Survey Results Summary Objectives of Goal Model Analysis Mapping of Procedures to Objectives Example Selection Conclusions & Future Work Analyzing Goal Models, Horkoff & Yu

29. Survey of GORE Analysis Techniques: Selection • Article selection: • Started with a set of known relevant papers • Linked work through references • Stopped with picture of breadth was captured (24 papers) • Alternative selection methods: • Search for specific key words… • … in specific journals, conferences, portals • … during specific time periods • Challenge: work in goal model analysis appears in a range of venues with a range of keywords • Venues: Books, RE, REJ, Agent-related conferences, CAiSE, AI related journal, FSE, PoEM, Journal of Information Systems, Trust-related Conference, ASE, etc… • Keywords: agent-oriented software development, goal-oriented requirements analysis, early requirements analysis, multi-agent systems, agent-oriented software engineering, agent-oriented methodologies, risk analysis, countermeasure identification, goal modeling, goal-oriented analysis, quality metrics, etc…. Analyzing Goal Models, Horkoff & Yu

30. Guideline Usage Examples • Example: Wireless service from Amyot et al. 2010 • New wireless service added to existing network • Where should the data and service be located? • Apply guiding questions… • Domain contains some interacting systems (QU1), no emphasis on communication (QC1) • Aware of alternatives, but need to select one (QD1, QD3) • Do not yet understand details, detailed alternatives, don’t have access to specific information, don’t want to find processes or perform simulations (QU2, QR2, QD2, QD5, QD6) • Domain is well understood, scope is clear, models are sufficiently complete (QE1, QS1, QM1) • Must consider non-functional requirements (QE3), data privacy (QR1) • Recommendations: • Agent-oriented approaches supporting softgoals to consider social and non-functional nature of the problem • Satisfaction analysis or metrics to chose between alternatives Analyzing Goal Models, Horkoff & Yu

31. Guideline Usage Examples • Example: Online Counseling Domain from Horkoff & Yu, 2009 • Online counseling alternatives: text messaging or chat room? • Apply guiding questions… • High degree of social interaction (QU1) • Do no yet understand details, not yet confident in the accuracy and completeness of models (Qu2, QM1) • Communication is important, scoping is challenging (QS1, QU2) • Consider many non-functional requirements, privacy and security especially, capture assumptions (QE3, QE4, QR1) • Need to find and evaluate alternatives (QD1, QD3) • Could be useful to find the most successful process (plan) for counseling or simulate throughput (QD4, QD5) • Recommendations: • Interactive, agent-oriented techniques for forward satisfaction analysis supporting softgoals • Analysis for anonymity or privacy with the same techniques or with GRL Satisfaction Analysis, and/or i* Metrics • If the required detailed information is available, apply planning and/or simulation techniques Analyzing Goal Models, Horkoff & Yu