1. GENERALLY ACCESSIBLE Model-Based Testing Forces and Solutions Prof. Walter Kriha, Hochschule der Medien Stuttgart, Computer Science and Media Faculty September 16, 2005

2. SECTION 1 Problem View

3. Example:Internet Service When „search“ returns passwords... User Registry Credent. Vault. Author. Server Identity Server Authent Server client Reverse Proxy App. Server App. Server Host Internet CSIv2 CSIv2 CSIv2 WS-S WS-S External TTP App. Server Other Company App. Server Domain Bridge (TTP) Deploying an application into this environment can take month after month of laborious testing. But how can you be sure that core security concepts (like trust zones, end-to-end security, secrecy etc.) are met and maintained by the software? Automation of tests is a key requirement! Execution of tests must be fully traced.

4. Challenges Technical and Social • multi-tier architectures • complex infrastructures (J2EE/.NET) • long deployment chains (Dev.Test/Mit/AIT/PTE/Production) • Cross-cutting concerns (security, performance) • Multiple technologies (descriptive, imperative) mixed • Social factors (roles, skill sets, limitations, change) It will take advanced tools for developers to perform continuous and early testing of those complex systems

5. Example: Browser Bugs An endless story? Same Origin Concept site A site B A B Script Engine Every new feature in a browser (frames, pop-up windows, tabs, bookmarks etc.) seems to violate this simple principle of protection. Take a look at the mozilla security buglist. Why is that so?

6. Challenges Concepts and Implementations • The Code does not show „same origin“ • Browser models do not contain the concept • How can we test for such high-level concepts? Is it possible to capture advanced security concepts in models and use them for (automatic) testing? Another example that needs much better testing is the implementation of dynamic Role-based-access-control (RBAC) systems which rely on static data, rules and environment values to reach a verdict.

7. SECTION 2 Testing Approaches

8. Overview From no concept to a representation of test concepts No test concept Logbook textual input, manual tests, automated reporting: What do they want me to test? I wonder what this change will do? Model based model and test language supported by tools. The model itself contains concepts through profiles. Scripts Another application change, need to fix test scripts too The concepts need to be caught in models and used to drive test engines, simulators or generate code against the production software

9. Model-Based Testing Architecture Test Tooling Test libraries Profiles Test Specification Test Generator specification model SUT behavioral test models Test Model Interpreter State chart Activity Diagram MSC Control Flow Data Flow Abstract Concrete This is only one example of many different ways to use MBT. With UML 2.0 activity diagrams are an effective way to represent „unit-of-work“ like concepts which appear naturally in testing.

10. Is it Better? Where are the benefits? • Just building the test model uncovers lots of bugs in the requirements (model) • Number and quality of bugs found: A strong tendency towards better results with MDT • Manually deriving tests from a model still benefits from the model. According to Pretschner et.al. testing without a model gives the worst results.

11. Who can do it? EAI example: How to avoid the textual step Execute instructions in EAI server Business creates Transformation Rules with specialized UML editor Create Meta-data and Actions for Interpreter/Server Server (UML VM) Tool A successful example how model driven development enables even end-users to create precise specifications of data transformations. Shouldn't this be possible with testing as well?

12. Technologies and Tool Support UML • In non-technical environments UML is the prevailing modeling language • Behavioral specifications are expressed as state charts or activity diagrams • Tools should provide „canned know-how“ for tests of non-functional requirements • Model checking and executable UML are possible options for the future. They bridge the gap between abstract models and implementations by bringing more concreteness (abstract syntax of actions e.g.) into the models. For an example of the use of UML for model checking take a look at UMLsec.

13. MDD vs. MBT Different Benefits • Different abstraction levels: MDD needs to express more general concepts and less implementation allow different implementations • Requirement models in MDD need to be generalized to create flexible systems. In MBT requirement models need to be made more concrete. • MBT can use concrete implementation know-how from the SUT • MBT tools should be able to re-use models from MDD • MBT tools need to cater to a different class of users Model-Based Testing is not the same as Model-Driven Development.

14. The Future Exept and HDM • Constraint modeling and transformation into tests • Process modeling, semantics • Representation of non-functional requirements Based on a common understanding of concept based learning and development HDM and Exept will explore the above topics in projects and courses at HDM.

15. Resources • W.Prenninger, A.Pretschner, Abstractions for Model-Based Testing http://www4.informatik.tu-muenchen.de/~prenning/publications/tacos04.pdf (shows that developement and test models are different. Good introduction to MBT) • Pretschner, Prenninger, Baumgartner, Stauner, One Evaluation of Model-Based Testing and its Automation. http://www.inf.ethz.ch/personal/pretscha/papers/icse05.pdf (empirical results on a comparison of MBT vs. manual testing and mixed forms) • M.Friske, H.Schlingloff, Von Use Cases zu Test Cases: Eine systematische Vorgehensweise http://www.informatik.hu-berlin.de/~hs/Publikationen/2004_MBEES_Schlingloff-Friske_Von-Use-Cases-zu-Test-Cases.pdf (shows requirements translation into activity diagrams) • H.Störrle, LMU Munich, Towards a formal Semantics of UML 2.0 Activities http://www.pst.informatik.uni-muenchen.de/personen/stoerrle/V/AD-11-Limits.pdf (a denotational semantics for UML Activities based on extended Petri Nets) • Workgroups: GI-FG TAV (Test and Verification of Software) • Workgroups: Dagstuhl Seminar 2004: MBT for reactive systems http://www.it.uu.se/research/project/motres/subjects.html • Hartmann, Nagin, The AGEDIS Tools for Model Based Testing • Bernhard Merkle, Designermodelle, IX 2005/5. Good Introduction to Model-Driven Architecture and Development Tools and Processes. • Jan Jürjens, Secure Systems Development with UML, Springer 2005. Uses a UML Profile for security to validate security protocols. Uses Model Checkers and shows how to test SAP3 permission settings against a model. • T.Stahl, M. Völter, Modellgetriebene Software-Entwicklung. Dpunkt Verlag 2005. Covers meta-modelling, Domain Languages, split-level development and generative patterns.