1 / 13

Tools for Automating Verification and Validation

Tools for Automating Verification and Validation. Validation: checking: Are we building the right system? Verification: checking: Are we building the system right? Model dependent size dependent platform dependent Purpose: improve the S/W development process less error-prone

yuri
Télécharger la présentation

Tools for Automating Verification and Validation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tools for Automating Verification and Validation Validation: checking: Are we building the right system? Verification: checking: Are we building the system right? • Model dependent • size dependent • platform dependent Purpose: improve the S/W development process • less error-prone • more standard • better defined • more predictable • faster (time-to-market) • better managed process • assist in better understanding product (helping to manage complexity of product) - avoid delays, reworks

  2. Automating V & V in the S/W Development Process Must be at SEI CMM Level 3 or higher. (- process is defined) [pre-condition for use of tools] BUT only automate where necessary e.g. Reqts. High Level Design Detailed Design Release & Support Codes Inspections & Unit Test Regression Performance ... Build & System Test Level 4 CMM - metrics to help in process management

  3.  taking the process view for V & V tool assessment, 1) Where in the process does the tool apply? 2) What is the entry criteria to the use of the tool(set)? 3) What is the exit criteria (for stopping the use of the tool?)

  4. Ed Kit’s Tool Classifications Tools for reviews and inspections - assist in reviews, walkthroughs and inspections of requirements, functional design, internal design, and code. - Complexity analysis - “20% of code causes 80%of the problems” (Pare to principle) [not good predicates of bad code, easy to collect $] - Code comprehension - help to understand unfamiliar code (take an unbiased look) - Syntax and semantic analysis - extensive error checking to find errors a compiler would miss - flag potential defects Tools for Test Planning - define the scope, approach, resources and schedule of testing activities - tools required are: - templates for test plan documentation - test schedule and staffing estimates - complexity analyzer # of issues R8 R3 R1 R2 R4 R10 R5 R6 R7 R9

  5. Kit’s Tool Classifications (2) Tools for Test Design and Development • detailing the overall test approach specified in the test plan for software features or combinations of features, and identifying and prioritizing the associated test cases. • Not much tool support • types of tools required: • test data generator • requirements-based test design tool • capture/playback • coverage analysis Test Execution and Evaluation Tools • process of executing test cases and evaluating the results • includes selecting test cases for execution, setting up the environment, running the selected tests, recording the execution activities, analyzing potential product failures, and measuring the effectiveness of the effort. • Types of tools required: • capture/playback • coverage analysis • memory testing • simulators and performance • regression testing - necessary

  6. Kit’s Tool Classifications (3) Software Testing Support Tools • tools that lend overall support to the overall test process • when these tools are of poor quality or non-existent the professional tester suffers • types of tools required: • Problem management • defect tracking tools, etc., used to record, track, and assist with the management of defects and enhancements throughout the lifecycle • Configuration management • manages, controls and coordinates changes to documents and other artifacts Questions before tool acquisition • How do the tools fit into and support our test process? • Do we know how to plan and design tests? • Who will be responsible for making sure we get the proper training on our new tool? • Who will promote and support tool use within the organization on an ongoing basis?

  7. Beizer’s Test Tool Classifications • Fundamental Tools • coverage certifiers • flowgraph generators and analyzers (e.g. MSC generator) • metrics support • instrumentation support (probe insertion & monitoring) • Test Execution Automation • capture/replay • test editor • smart comparator • local and distributed • single and multi-thread • exception reports • stubs and drivers • test case initialization • input simulation • outcome comparison • path instrumentation and verification • reporting • passage to next test • debugging support • test environments • support for testing in a CASE environment

  8. Beizer’s Tool Classifications (2) • Test Design Automation • structural generators • data-flow test generators • functional generators • FSM (EFXM) based generators • random test data generators • Test Result Analysis Automation

  9. Load and Performance Testing Current Listings Benchmark Factory Chariot CYRANO Test ITF Load Runner Product Family Pegasus PLATINUM Final Exam Internet Load Test pre Vue-C/S pre Vue-X QALoad SQA LoadTest Teleprocessing Network Simulator TestMaster WebLoad Silk Performer

  10. Non-GUI Drivers and Test Suite Managers Current Listings Acqua QES/EZ AcquaNavio SNA Development Test Facility AdaTEST SQA Manager ANVL SQA Suite ARTT STAR BenchWorks STEPMaster Cantata STW/Regression CobRetest STW/Regression/MSW CTB TALC2000 DejaGnu TBGEN ESIM Testing Environment TCS for Embedded Software TestDirector Internetworking Test Facility TestExpert MYNAH TestMate OTF - An Object Testing Framework TestPlan Pilot TMS Pre Vue TMX QA Director TETware QES/Architect VPRO-G

  11. Test Evaluation Tools Current Listings Branch validator C-Cover CodeTEST CoverTest CTC++ DeepCover for Java DeepCover for C/C++ GCT Hindsight/TCA Hindsight/TPA Insure++ LDRA Testbed Logiscope McCabe Visual Testing Toolset ObjectCoverage Panorama-C/C++ pf77 PureCoverage safec SofInst STW/Coverage TCMON TCAT for Java TestCenter VisionSoft/TEST

  12. GUI Test Drivers Current Listings ATF (Automated Test Facility) B-Tree Validor Family CAPBAK/X, CAPBAK/MSW CYRANO TEST Evaluator Ferret QARun Panorama-2 PLATINUM’s Final Exam C/S-Test PLATINUM Final Exam Internet Test QA Partner QC/Replay QES/EZ for GUI ReplayXt RSW Web Analyst SET (Solution Evaluation Tool) Smalltalk Test Mentor Vermont High Test Plus SQA Robot Visual Test 4.0 STW/Web WinRunner Test Edge Win Vue Test Now WITT for OS/2, WITT/Windows, X/WITT TestRunner xrc - X Remote Control TestWeb XRunner

  13. Marick’s CornerSite Hosted by ST Labshttp://www.stlabs.com/marick/faqs/t-design.htm CALIBER: www.tbi.com

More Related