1 / 12

Improving Testability

Improving Testability. Dave Catlett Test Architect, Microsoft davidcat@microsoft.com. Agenda. What is testability? Why is testability important? Background: What is a test? Determining testability: SOCK Improving testability Q&A. What is testability?.

aran
Télécharger la présentation

Improving Testability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Testability Dave Catlett Test Architect, Microsoft davidcat@microsoft.com

  2. Agenda • What is testability? • Why is testability important? • Background: What is a test? • Determining testability: SOCK • Improving testability • Q&A

  3. What is testability? • The degree to which components and systems are designed and implemented to make it easier for tests to achieve complete and repeatable code path coverage and simulate all usage situations in a cost efficient manner. • In other words: How easy is it to test?

  4. Why is testability important? • Helps enforce good design principles. • Reduces the time to diagnose unexpected behavior. • Improves manageability and supportability. • Provides the building blocks for self-diagnosing and self-correcting software. • Speeds test development while increasing the depth and quality of automated tests.

  5. Background: What is a test? • Making sure the component works as expected and has the expected effect on the system as a whole in a repeatable and reliable way. • Anatomy of a test case: SEARCH • Setup • Execution • Analysis • Reporting • Cleanup • Help

  6. Determining testability: SOCK • Simplicity • Reducing cost of testing at each phase of SEARCH • Design • Observability • Exposing state (aka Visibility) • Control • Exercising every nook & cranny as efficiently as possible. • Knowledge of expected results • Is observed behavior is correct?

  7. Improving Simplicity • Make setup and configuration easy • Componentize • Allow multiple installs with varying configs on same machine • Use good design principles • Strong cohesion • Loose coupling • Low redundancy

  8. Improving Observability • Expose component state • Magic numbers (Any value that controls code path execution.) • Resource usage • State transition information • API in/out parameters • Tracing • Good tracing library • Observer or Visitor patterns can help maintain good design • Be careful of exposing security info!

  9. Improving Control • Wrap interfaces you have no control over • Forces behavior you need to test your component • Facilitates fault injection tools • Windows: Holodeck, AppVerifier • Expose Magic Numbers to become configurable at runtime. • Be mindful of security

  10. Improving Knowledge of expected results • Code contracts in source code • Document expected behavior in all situations • Document end-to-end user scenarios • Future: Use modeling (machine readable specifications)

  11. Q&A

  12. Resources • Design Patterns Explained: A new perspective on object-oriented design. Alan Shalloway, James Trott. Addison Wesley, 2002. • Observer pattern (tracing option for new classes) • Visitor pattern (tracing option for existing classes) • Adapter pattern (wrapping for fault injection) • Refactoring: Improving the design of existing code. Martin Fowler, et al. Addison-Wesley, 1999. • Refactoring to improve design helps testability at the same time. • www.netobjectives.com • Emergent Design, other materials • Fault injection tools for Windows • Holodeck: http://www.sisecure.com fault injection tool. • AppVerifier: http://www.microsoft.com/technet/prodtechnol/winxppro/deploy/appvrfr.mspx • Google: CCCC cyclomatic complexity tool • Brett Pettichord, Industry Testability Guru • Design for Testability: http://www.io.com/~wazmo/papers/design_for_testability_PNSQC.pdf • QA-related papers: http://www.io.com/~wazmo/qa • Other: http://www.pettichord.com

More Related