1 / 18

Improving Effectiveness of Regression Testing of Telecommunications Systems Software

Improving Effectiveness of Regression Testing of Telecommunications Systems Software. Sami Torniainen 19.2.2008 Supervisor: Professor Raimo Kantola. Outline. Goal Background How to improve cost-effectiveness? Methods Implementation Evaluation Further research. Goal.

voverton
Télécharger la présentation

Improving Effectiveness of Regression Testing of Telecommunications Systems Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Effectiveness of Regression Testing of Telecommunications Systems Software Sami Torniainen 19.2.2008 Supervisor: Professor Raimo Kantola

  2. Outline • Goal • Background • How to improve cost-effectiveness? • Methods • Implementation • Evaluation • Further research

  3. Goal • Goal: improve cost-effectiveness of regression testing at the target company • Three sub-objectives: • explore present regression testing practices • implement methods • evaluate empirically

  4. Background - Target Company • Finnish telecommunications systems developer • Located in Espoo • Main product: IP/MPLS routers • Customers: global mobile and internet service providers

  5. Background – V-model

  6. Background – Regression testing • “Regression testing is selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified requirements.” (IEEE) • Can occur on any level of the V-model • This thesis focuses on regression testing on integration testing level

  7. How to improve cost-effectiveness? • Costs accumulate from time and resources consumed in executing tests • Two approaches mainly utilized in literature • test selection techniques • test automation

  8. Test selection techniques • Reduce the number of tests • Mainly based on code exploration • unit testing • Also exists risk-based method • integration testing level

  9. Test automation • Automating manual working phases • creating executable test programs i.e. scripts • Popular method • Expensive to utilize

  10. Implementation • Limited scope (10 test suites) • Risk-based test selection technique • evaluating test suites based on risk-analysis • executing only five test suites of ten • Test automation • automating the control of Adtech AX-4000 • Tcl API • no manual intervention needed during test execution

  11. Influence on regression test process

  12. Evaluation – testing effort

  13. Evaluation – detected errors

  14. Evaluation – deploying effort

  15. Evaluation - Summary • Test automation reduced testing effort significantly • but was expensive to implement (94%) • Risk-based test selection method also reduced testing effort • but will likely miss errors

  16. Conclusions • Despite the great implementation costs, test automation is recommended • break-even ”only” after 34 test rounds • Benefits • reduced testing effort • safer • reduce the overall testing elapsed time • potentially earlier time to market

  17. Further research • Develop Tcl API • reloading of GUI-based test setups • would reduce scripting effort • Optimizing test coverage of risk-based test selection technique • threshold • minimizing probability to miss errors Vs. reducing the number of test suites

  18. Thank You! Questions?

More Related