Download
test and verification solutions n.
Skip this Video
Loading SlideShow in 5 Seconds..
Test and Verification Solutions PowerPoint Presentation
Download Presentation
Test and Verification Solutions

Test and Verification Solutions

104 Views Download Presentation
Download Presentation

Test and Verification Solutions

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Improved time to market through automated software testing Mike Bartley, TVS Test and Verification Solutions

  2. Agenda • The potential benefits & drawbacks of automation • Compare with HW verif • Can automation reduce TTM? • Automating static & dynamic testing • Deciding what to automate • Choosing your automation tools • Case study

  3. The potential benefits of automation • Reduced costs • Improved quality • Improved employee motivation • Improved TTM

  4. The potential drawbacks of automation • Shelf-ware! • Maintenance • Skills base • Missing failures

  5. HW verification The mechanics of complex test bench Test Scoreboard Functional Coverage Coverage constraint addr data Monitor Design Under Test Assertions Stimulusgenerator Driver Coverage assert

  6. The mechanics of finding a bug Stimulate …..01010101 Propagate Design Under Test 01100101….. …..01001101 11110101….. …..10011010 00010101….. …..01001101 Observe Actual Results Expected Results Compare

  7. Can automation reduce TTM? • Getting test results sooner • Catching bugs earlier • Stabilising the software sooner • Assessing the stability of the software

  8. Automating static testing • Types of bug found • A quick win • Beware false failures • Introduce filters slowly • Developers not testers (check in checks, IDE) • No testing of intent

  9. Automating dynamic testing • Test plan • Test design specification • Test case specification • Test procedure specification • Test execution • Test log • Test incident report • Test summary report

  10. Deciding what to automate • Complexity of the test environment • The test level • Frequency of execution • Ease of automating pass/fail • Test stability and repeatability

  11. Choosing your automation tools • Who writes test test? • Matching skill set • Separate orthogonal concerns • Portability • Ease of adding/banning tests • Integration into configuration/build system • Cost of the tool?

  12. Case Study – Background • Includes chips, accelerator cards, and software • All work with industry-standard x86-based systems

  13. Case Study – Background • Low level host driver code • Code on host computer communicates with the chip on the board in the PCI slot • Libraries of code • Runtime library • Mathematics acceleration libraries (ACML, MKL, …) • Application code • Linpack, Montecarlo, …. • Code running on the chip • This had parallel C extensions not supported by our chosen tool • Software models of the hardware • In-house compiler for the code running on the chip • Mainly C and C++, some assembler

  14. Case study – automation details • Integrated into configuration/build system • Automated mostly lower level tests • Developed internally by SW Engineers • Offshored manual tests • Offshored some automated test development • Integration of static analysis • Test maintenance

  15. Case study - results of automation • Improved quality • Shortened release schedules • Release schedules hit • Reduced TTM • ROI?

  16. Questions? • Mike Bartley • mike@tandvsolns.co.uk • LinkedIn • 07796 307958