Common Test Project

1. Common Test Project CMU Presentation Jason Weighley Manager Test Engineering

2. Background • Bombardier Transportation – RCS Division – Provides CBTC signaling for Metro and People Mover applications • Signaling system consists of several sub systems (Comprised of Hardware / Software) which require changes from project to project • All sub systems are tested in a factory environment (Separate then integrated) Tested procedures and cases based on functional requirements and prepared by a test engineering group. • Safety sub systems are also tested / inspected by independent group (VNV). This group also prepares its own test cases for functional / safety requirements.

3. Why do we need COMMON tests? • Sharing a common test suite (tools) is the first requirement towards improving schedule, budget, and quality – and is already being addressed via the effort of the test group. • Common tests will give us a clearly defined quality gate that must be cleared before code can be released to FAT, V&V and/or the field. Stable test environment must be achieved before developers begin to use. • As each new “issue/bug” is discovered, additional test can be added to the test cases thereby ensuring that each new issue/bug will continue to be tested – despite the fact that people may forget about them over time. • If we have (increased) common test coverage, we will be able to provide faster turnaround time with less effort. • Quality and Repeatability will increase. Cost and schedule turnaround will decrease.

4. Why do we need SHARED test cases? • Maintaining 2-3 sets of test cases is costing us 2-3 times as much budget – this is a cost issue. • Sharing test cases will ensure that the expertise and knowledge of each engineer will go into the process to be shared by all others. • Quality and Repeatability will increase. Cost and schedule turnaround will decrease.

5. All of the tests are really the same! Most of our effort goes here We do NOT do enough of this!

6. What exactly does “shared tests” mean?

7. Current Approach

8. Proposed Approach • Management tool to track progress • Built-in Quality Gates (V&V and Factory test cannot start unless the automatic scripts pass) • No duplication of effort results in 2/3 reduction of total test time • Better consistency • Automated test yields faster turn-around time after changes • Higher confidence that all requirements were implemented and tested Scripts must be continuously updated as the software is debugged.

9. Summary Statements • Improving the efficiency of the project testing is the key towards reducing the product cost! • Finding defects earlier (moving to the left-side of the “V” diagram) • Reducing the test overlap between the FAT, Safety, Field, and Development groups • Reducing the turn-around time for regression testing (Development, FAT, Safety)

10. Plan to implement and Work Required • Record VNV and Test Engineering test cases used for vehicle and wayside development on Metro Madrid into DOORS (cases to be provided by cognizant engineers) • Update links to functional requirements for ATP related tests to VNV and Test Engineering test cases • Review Test Case linking with Cognizant engineers • Based on linking Identify common test cases from Metro Madrid (Current base project for several new China projects) • Review Common Test Cases with Cognizant Engineers • Repeat above for Shenzhen based on Project requirements and add/modify common tests • Update / Generate automated testing for China Projects based on Common Test • Test automated test scripts to verify script is working correctly.