An Approach to Testing Distributed Systems

1. An Approach to Testing Distributed Systems Ina Schieferdecker, Mang Li, Stephan Pietsch, Theofanis Vassiliou-Gioles, Andreas Hoffmann GMD FOKUS, Berlin

2. Contents • Introduction • Testing of Distributed Systems • Test Case Development • Test Validation: TSsim • Test Implementation: TCgate • Test Execution: TTman • An Example: TINA Retailer Reference Point • Conclusions

3. Introduction • Distributed systems are complex, heteregeneous, and concurrent systems • Validated properties of the system specification might not be fulfilled by a system implementation • CTMF is a well-established and widely used test method for conformance and interoperability testing of OSI systems (including TTCN as a test notation) • Goal: • Application of TTCN to distributed systems • Automated CORBA based distributed test execution

4. Local Testing Distributed Testing Test Manager Test Manager TC TC TC TC TC TC IUT IUT • One Testing Device • Common Time • Coordination Message Exchange is Local • Many Testing Devices • Distributed Time • Coordination Message Exchange crosses Boundaries Local and Distributed Testing

5. TS OperationsTest StepsTimers TTCN Test Cases ODL / IDL SDL Adaptor CoDec TTCN ETS TCgate SUT ORB ORB ORB The Overall Approach

6. Test Case Development • Declarations part: • TTCN types: • Translation from ODL types is supported by mapping rules. • Mapping rules can be used for manual or semi-automatic test development. • Mapping rules support generic encoding/decoding. • Others: • Operations, parameters, constants, variables, timers.

7. Test Case Development (cont‘d) • Dynamic part, constraints part : • Currently manually developed, based on interpretation of textual description. • Alternative: semi-automatic, based on SDL specificationI.e. with TTCNLink, AutoLink or TGV.

8. ODL-to-TTCN Mapping • Mapping for an ODL interface declaration: • Interface inheritance is flattened. • An interface is comparable to a SAP. • Requested and provided interfaces are distinct  two PCO types: • PCOtoSUTServer • PCOtoSUTClient • Multiple PCOs derived from the same interface declaration can be used in a test (multi-server or multi-client).

9. ODL-to-TTCN Mapping (cont‘d) • Mapping for a typical ODL operation declaration: • Operations are mapped to ASPs. • Three ASP types are derived from an operation declaration: • A request-related ASP type. • A reply-related ASP type. • A raise-related (exception-related) ASP type.

10. PA Consumer Domain Retailer Domain IA requestNamedAccess() UAP i_RetailerInitial UAP: User ApplicationPA: Provider AgentIA: Initial Agent Example:Test for Service Access Session Retailer Reference Point

11. Example: IDL Specification • module TINARetRetailerInitial { • interface i_RetailerInitial { • voidrequestNamedAccess ( • in TINACommonTypes::t_UserId userId, • in TINACommonTypes::t_UserProperties userProperties, • out Object namedAccessIR, • out TINAAccessCommonTypes::t_AccessSessionSecretId asSecretId, • out TINAAccessCommonTypes::t_AccessSessionId asId • ); • }; • };

12. Example: Simplified TTCN Test Case

13. Example: Test Case for Service Access Session

14. Distributed TestExecution Distributed System TestValidation Test Specification TTman SUT ITEX TSsim TCgate The Test Tool Chain

15. Motivation for a TTCN Simulator • Testing the test problem • Validation of test suites is essential • Complexity of Concurrent TTCN test cases • Analyser tools provided by TTCN editors provide only syntax and static semantic checks • Manual dynamic semantic check by experts inefficient and error-prone TSsim

16. Test Suite Simulator TSsim • Basis for test suite validation • Generic test suite simulator • Supports Concurrent TTCN • Interactive step by step execution of test cases (PDUs from the IUTs, Timeouts) • Verification of dynamic aspects (e.g. test coordination procedures, timer settings) • Validation of logical correctness, deadlocks TSsim

17. Test Suite Simulator GUI TSsim

18. Simulation Validated TTCNATS Simulation Results Correction Simulation Report Validation Process with TSsim TSsim

19. Error Detection with TSsim • Timer Errors deadlock, if no receive event is evaluated to true • Test Component Coordination Errors timeout, if a test component waits for a CM • Indentation Errors unexpected dynamic behavior • Constraint Errors non-matching of correct PDUs or CMs TSsim

20. Distributed TestExecution Distributed System TestValidation Test Specification TTman SUT ITEX TSsim TCgate The Test Tool Chain TCgate

21. Motivation for a TTCN/CORBA GW • Acceptance of TTCN • Support of parallel test components via Concurrent TTCN • Test access to CORBA-based systems. • Uniform interface to TTCN-based test systems • Separated from TTCN-based test systems TTCN compiler-independent. • Problem: TTCN OSI paradigm vs. CORBA OO paradigm TCgate

22. TTCN/CORBA GW: TCgate • Architecture of the gateway: • Supports testing of interfaces provided by the SUT. • Supports emulation of interfaces required by the SUT. • Is a generic request-level bridge. • Uses dynamic interfaces of ORB  type-independent. • Is a CORBA-compliant application. • Uses public interfaces of ORB  easily portable. TCgate

23. ETS GatewayServer Supervisory functions TTCN run-time behaviour Help functions gwAdaptor gwCoDec GatewayMain TTCN / CORBA Gateway Interface Repository IDLSkeleton SUT IDLStubs GatewayClient DII DSI DSI DII BOA BOA ORB ORB Test Interface for CORBA Based Systems TCgate

24. 0100011000101101011111010101000100011101 ITEX-GCI Generic CoDec TTCN TTCN/CORBA-Gateway Adaptor TTCN ATS C++ ETS ETS Generator Test Suite Adaption to TCgate • Is based on GCI interface. • Has generic encoding/decoding for all IDL types. • Is reusable for other TINA and CORBA applications. TCgate

25. PA ETS Retailer Reference Point Consumer Domain Retailer Domain SUT requestNamedAccess() TINA Server TINA Client UAP i_RetailerInitial UAP: User ApplicationETS: Executable Test SuiteSUT: System Under Test Example:Test for Service Access Session

26. ETS TTCN / CORBA Gateway IUT / SUT GatewayMain GatewayServer GatewayClient IDLSkeleton IDLStubs DII DSI ORB ORB Example: Test Execution 1. Test Request from ETS to IUT 2. IUT Parameter Request to ETS 3. ETS Parameter Reply to IUT 4. IUT Test Reply to ETS 5. Test Result: Verdict Assignment => PASS / FAIL TCgate

27. Distributed TestExecution Distributed System TestValidation Test Specification SUT TTman ITEX TSsim TCgate The Test Tool Chain TTman

28. Test Sychronizations Protocol 1 • Defines Procedures for • Test Setup, • Setting Up Configuration • Distributing Parameters, etc. • Test Execution • Start, Stop and Cancel • Routing of Coordination Messages • Test Reporting TTman

29. CMI TSP1 FrontEnd1 TC TC TC TC System Supervisor FrontEnd2 IUT CMI - Campaign Managment Interfacce TSP1 - Test Synchronization Protocol 1 TC - Test Component TSP1 Architecture TTman

30. The Test Manager TTman • Portable Library for Implementing • The System Supervisor • The Front End • State Driven TCL/TK GUI at the CMI TTman

31. Test Execution • Generation of MSC logs: • Uses a library supplied by Telelogic. • Display selected test events in an MSC editor. • Benefits: visualization, problem location.

32. Test Execution (cont‘d)

33. Conclusions • Testing of CORBA based systems • ETS Generator for transformation of abstract test suites into executable ones, validation with TSsim • TCgate and TTman for automated distributed test execution • Testing within the development methodology for distributed systems • Future work • Enhanced coordination features for parallel test components • Investigation of real-time aspects of the TTCN/CORBA Gateway • TTCN extensions to better address the TINA object model