ASTA Aspect Software Testing Assistant

1. ASTAAspect Software Testing Assistant Juha Gustafsson, Juha Taina, Jukka Viljamaa University of Helsinki

2. Security Logging Cross-cutting concerns clutter the main class model Persistence MotivationCross-Cutting Concerns in Software Implementation modules

3. Logger out: OutputStream log(s: String)

4. An ExampleMoney Transfer: Logical Level void transfer(Account from, Account to, int amount) { if (from.getBalance() < amount) { throw new InsufficientFundsException(); } from.withdraw(amount); to.deposit(amount); }

5. Security Persistence Logging An ExampleMoney Transfer: Implementation Details void transfer(Account from, Account to, int amount) { if (!getCurrentUser().canPerform(OP_TRANSFER)) { throw new SecurityException(); } Transaction tx = database.newTransaction(); if (from.getBalance() < amount) { tx.rollback(); throw new InsufficientFundsException(); } from.withdraw(amount); to.deposit(amount); systemLog.logOperation(OP_TRANSFER, from, to, amount); tx.commit(); }

6. SolutionAspect-Oriented Programming (AOP) • Separating cross-cutting concerns from thecore concerns (i.e. from the main class model) • Aspect = a definition of a cross-cutting concern • pointcut = a pattern that selects join points(i.e. elements that appear in method bodies) • advice = a method-like construct that is executedin the selected join points (before, after, around) • weaving = the process of attaching advice to the appropriate join points

7. An example AOP programHelloWorld in the AspectJ language • Go to http://www.eclipse.org/ajdt/ … • … and watch the demo!

8. A new question arisesHow to test AOP? • Do we test what we intend to test? • How to prioritise our tests? • How to pick a good testing strategy? • When should we stop testing? • In AOP these are especially difficult questions because • the execution context of an aspect depends on the (internal implementation of the) class it is woven to • relating a failure to the fault that causes it is difficult • the control and data flows are not explicit • a particular aspect weave order can cause problems

9. A new question arisesHow to test AOP? • Visualising AOP testing is not straightforward • what kinds of test coverage criteria to use and visualise (is it sufficient to show that we have visited an aspect once, how to show the weaving context)? • current control flow graphs are insufficient to visualise the test execution paths in the presence of aspects • high-level coverage visualisation for AOP testing is a completely open question • Yet, visualisation of AOP testing is (at least) as important as visualisation of traditional (OO) testing

10. Testing AOPFault Sources • Fault in the core concern code • Fault in the (context independent) aspect code • Fault created by interactions between the aspect and the core concern code • Fault created when more than one aspect is woven into the core concern code

11. Testing AOP [Alexander, Bieman, Andrews, 2004]Fault Model & Additional Testing Required • Incorrect strength in pointcut patterns • test the aspect • Incorrect aspect precedence • test all weave orders • Incorrect focus of control flow • use a form of condition coverage of pointcut designators • Failure to establish expected postconditions • use the original test set to re-test all methods that have the aspect • Failure to preserve state invariants • same as in previous • Incorrect changes in control dependencies • same as in previous

12. AOP Testing Theory • Coverage criteria • Coupling metrics • Control flow formalism • …

13. Tool Support • Tool support is essential for systematic testing • Not many tools exist for AOP testing • coverage tools • visualisation tools • scripting languages & environments to automate AOP testing

14. Research Background • Due to project RITA (fRamework Intgration and Testing Application) we have a strong background in white-box test execution and analysis • RITA is intended for product family testing, but similar ideas and issues arise in AOP testing • RITA as a prototype is not suitable for large-scale testing but its ideas are relevant

15. RITA Functionality • Rita supports the following: • Java program testing • Product family or single application testing • JUnit and private test execution • Full support for data flow diagrams • Visualisation and white-box coverages on four levels: • Method level: traditional coverages • Object level: object and class coverages • Package level: object coupling, reference coverages • Application level: multiple application coverages

16. Lessons learned from RITA • A White-Box Testing Software (WBTS) must support traditional code • RITA: not all applications are in product families • ASPECTS: not all applications use aspects • WBTS should be integrated to a software development environment • While stand-alone software can have similar functionality than integrated software, usability issues favor the latter approach • Code coverage is not sufficient in WBTS • The higher is the abstraction of testing, the more complex coverages are needed. • Even at code level code coverage gives weak results.

17. Aspect-Oriented Testing Tool • Based on our experiences in RITA we propose the following requirements for an Aspect-Oriented Testing Tool (AOTT): • It has to support testing of traditional software • It must be integrated to a well-known and actively updated programming environment such as Eclipse • It has to support several levels of white-box testing including program-level, package-level, object-level, aspect-level, and method-level • Aspect-level: testing and visualisation are inside a single aspect.

18. AOTT Requirements Continue • More AOTT requirements: • It has to calculate static mesurements of the software: • Lines of code (LOC) • Aspects of code (AOC) • Coupling between objects • Depth of inheritance tree • Depth of aspect tree (aspects in aspects) • Coupling between code and aspects etc. • It has to support integration between dynamic test execution results and static measurements • How? This is an open research question • Something else? Definitely! This is a work-in-progress list

19. Research directions [Alexander, Bieman, Andrews, 2004]Open Questions • Can we measure test coverage after weaving? • also visualisation • How do we test aspects that interact with the core concern code? • coupling between an aspect and core concern code as a measure of testability of the aspect • How do we test aspects that interfere with each other? • inter-aspect coupling • How do we test aspects whose effects must span more than one concern? • Are there ways to test aspects on their own? • probably not… ? • (Can we reverse engineer the weave process?) • how does this relate to testing

20. Validation • How to validate that • Theory is correct? • Application is effecient? • Alternatives: • Formalism • Empirical controlled experiments • Practical experiments in industrial settings • All alternatives are • Relevant • Needed.

21. Summary • To be written...