30 likes | 189 Vues
Reference Toolkit. What is a “reference” toolkit? What is the status of the generation 2 Reference Toolkit? How Big? Tracking it. We need to understand test execution (the process of creating tools mapped to the written test artifacts) on several dimensions. Architecture Philosophy.
E N D
Reference Toolkit • What is a “reference” toolkit? • What is the status of the generation 2 Reference Toolkit? • How Big? • Tracking it. We need to understand test execution (the process of creating tools mapped to the written test artifacts) on several dimensions.
Architecture Philosophy • Very Small/Independent & Light Coupling • Big bang ‘one ring to rule them all’ systems haven’t worked well for the ONC Test Team. • Each module is meant to accomplish one thing, and modules are set into types. • We are researching a lightweight orchestration, but all modules can be accessed separately. • As each sprint is completed, we release into the toolkit. The test coverage (represented in the written packages) stays very consistent, but the toolkit keeps expanding/maturing. • The overall community and new CTB is meant to keep pushing an open Reference Toolset.
Testing Toolkit System Under Test (SUT) Core Modules Test Data SUT Gateway Backend Systems Message Results A Tester Can perform a number of functions: • Defines basic connectivity information (endpoints, etc) directly with their systems, toolkit, modules or through Web/Orchestration layer. • Triggers messages (Initiator test cases) to the test tool. • Triggers simulators directly with modules or (potentially) through Web-Orchestration layer from the test tool back to the SUT (Responder test cases). • Retrieves results from received messages (Responder test cases) and passes them to the Content Checkers (or performs with written checklists). Web & Orchestration Layers Exploring Cucumber as a Middle TDD-focused Layer in this stack Test Data Message DB & Log/Message Pullers Results DB & Worksheets Results DB User/Permissions DB