1 / 46

Validating Requirements

Validating Requirements . Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009. Overview. Validation Purpose and Definitions A Correct and Complete SRM, and the Three Questions SRM Correlation Mapping

wayne
Télécharger la présentation

Validating Requirements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validating Requirements Determining Completeness and Correctness of Requirements Using the System Reference Model IV&V Workshop 16 September 2009

  2. Overview • Validation Purpose and Definitions • A Correct and Complete SRM, and the Three Questions • SRM Correlation Mapping • Analysis of Correlation Results • Correct, Complete, Incorrect, and Incomplete Examples • Best practices, lessons learned, and challenges

  3. IV&V’s Validation Definition • The process of evaluating artifacts to ensure that the right behaviors have been defined in the artifacts. • The right behaviors are those that adequately describe • what the system is supposed to do • what the system is not supposed to do, and • what the system is supposed to do under adverse conditions. • Validation ensures that the software system performs to the user’s needs under operational conditions. • Contains the desired capabilities to accomplish the mission goals • Does not contain unspecified limitation that impedes the capabilities

  4. Validation Goal • To ensure that • The right behaviors have been defined • Adequately describe • What the system is supposed to do • What the system is not supposed to do • What the system is supposed to do under adverse conditions • Correct and Complete • The requirement specifications are of high quality • Unambiguous, Consistent, and Verifiable

  5. Correct (IVV 09-1) • Applicable requirement(s) meet all or part of the goals and behaviors of the system • Note: not all requirements can be evaluated in isolation; it may require a set of requirements to be evaluated together in order to determine that a particular goal or behavior is being met). • The requirements are an accurate elaboration of the defined objectives or goals • e.g., the use of temporal modal operators like “next”, “until”, “always”, and “eventually”, are appropriately used to reflect the desired behavior • The requirements adequately refine the higher-level requirements • Design or implementation-specific information is specified as constraints to the behaviors captured in the requirements

  6. Complete (IVV 09-1) • All the needed information to completely specify a desired behavior is identified (i.e., all preconditions, postconditions, and invariants are specified for the described behavior). • Threads of behavior are represented by more than one requirement, versus one compound requirement that attempts to capture the entire thread (i.e., that each requirement specifies only one “thing”). • The use of conjunctions (e.g., “and”, “or”) are restricted to preconditions, postconditions, and invariants.

  7. How? “This goal is achieved through the development and application of a system reference model (SRM) that will include a formal specification. The SRM can then be used to show (e.g., validate) that the right system behaviors are specified and the associated requirements are unambiguous, correct, complete, consistent, and verifiable. The SRM can also be used to validate (or develop) a test design that will demonstrate that the software products meet the specification and the operational need.” – IVV 09-1

  8. The System Reference Model • Includes sets of Modeling Artifacts • Use cases • Activity Diagrams, Sequences Diagrams • Statecharts • Domain Models (Class Diagrams, Communication Diagrams) • Statechart Assertions • JUnit Test Cases • A concise description of the IV&V team’s understanding of the problem • Analysis tool • Communication tool • Captures expected system behaviors • 3 Questions

  9. What’s In the SRM? Validation and Verification involves answering the following three questions: 1. Will the System do what it is supposed to do? 2. Will the System not do what it is not supposed to do? 3. Will the System maintain operations under adverse conditions? Note, in order to answer these questions, we must first have an independent understanding of: What the system is supposed to do What the system is not supposed to do What the system is supposed to do to maintain operations under adverse conditions This information can be found within different areas of our model.

  10. SRM Product Dependencies

  11. System Goals

  12. System Goals

  13. Constraints, Actors, and Environment

  14. Text Use Cases

  15. Use Case Example What the system is supposed to do All parts within the Main Success Scenarios describe the actions that must take place to accomplish the goal(s). Adverse Conditions Extension Scenarios show how the system should react to adverse conditions to get back on the success path or transition to safe mode.

  16. Activity Diagram Example What the system is supposed to do All parts within the Main Success Scenarios describe the actions that must take place to accomplish the goal(s). <<Main Success Scenario>> Precess to Earth Point <<Extension Scenarios>> Precess to Earth Point Fails Flight System Flight System Turn on IMUs [IMU Does not turn on or malfunctions] Turn on Precession catalyst bed heaters Return Errors [Cat bed heaters do not turn on] Turn on X-band Transmitters [not turning to earth point] [turning to earth point] Goal:Flight System precesses and damps nutation to point the High Gain Antenna at earth for communication and GRAV science Select Forward Low Gain Antenna Select High Gain Antenna Use Other Thrusters Turn off cat bed heaters Adverse Conditions Extension Scenarios show how the system should react to adverse conditions to get back on the success path or transition to safe mode. Pulse RCS Thrusters [thrusters do not fire] Precondition: Engineering Instruments are calibrated Turn off IMU Passive Nutation Damping

  17. Sequence Diagrams Example What the system is supposed to do The interactions of the Sequence Diagram describe the steps involved to accomplish the goal(s). Adverse Conditions A sequence diagram can also be developed to show how the system should react to adverse condition. 18

  18. SRM Validation Scenarios AssertTrue() AssertTrue() AssertFalse() estimateStateVector() estimateStateVector() getEstimatedStateVector() getEstimatedStateVector() getEstimatedStateVector() What the system is NOT supposed to do This information can be found in our Assertions and the Validation scenarios we create to test against them. • public void testScenario2() { • st.getEstimatedStateVector(); • st.estimateStateVector(); • assertTrue (st.isSuccess()); • st.estimateStateVector(); • assertTrue (st.isSuccess()); • st.getEstimatedStateVector(); • st.getEstimatedStateVector(); • assertFalse (st.isSuccess()); • }

  19. Validation WBS (IVV 09-1) 1.0 Validation 1.1 Obtain/Develop a System Reference Model (SRM) 1.2 Validate System Requirements 1.3 Validate Test Design

  20. Validate System Requirements • For each level of system decomposition, we need to determine • Sufficiency of the requirements • Is there a corresponding requirement for every SRM behavior and Statechart assertion at that level? • Quality of the requirements • Assess the quality of each requirement that has a corresponding SRM behavior or Statechart assertion at that level • Sufficiency of the SRM • Is there any mission-critical, safety-critical requirement not covered by an SRM behavior or Statechart assertion at that level? • A “correlation map” is built to capture these relationships

  21. Validated Requirements SRM Correct? SRM Correct? Unambiguous, Requirements Requirements In Correct, Complete? Scope and Consistent, & Valid? Verifiable Requirements Validation Possibilities SRM Behaviors Requirements

  22. An Example Behavior Preconditions Goal Extensions – Q2 & Q3 Main Success Scenario – Q1 & Q2 Constraints References Post-conditions

  23. Requirement Proxies

  24. Requirement Proxies

  25. Requirement Proxies

  26. Requirement Proxies

  27. Other Analysis Tools Subject Requirement Validation Findings Parent Requirements Correlation Mapping Child Requirements

  28. Requirement Data Subject Requirement Subject Requirement Validation Findings Parent Requirements Correlation Mapping Child Requirements

  29. Parent Traces Subject Requirement Validation Findings Parent Requirements Parent Requirements Correlation Mapping Child Requirements

  30. Child Traces Subject Requirement Validation Findings Parent Requirements Correlation Mapping Child Requirements

  31. Correlation Mapping Subject Requirement Validation Findings Parent Requirements Correlation Mapping Child Requirements

  32. Validation Findings Subject Requirement Validation Findings Parent Requirements Correlation Mapping Child Requirements

  33. Requirement Proxies

  34. Correlation Mapping & Requirement Evaluation

  35. Correlation Map

  36. Correlation Map Model Elements Findings/Issues Requirement Mapping

  37. Correct (IVV 09-1) • Applicable requirement(s) meet all or part of the goals and behaviors of the system • Note: not all requirements can be evaluated in isolation; it may require a set of requirements to be evaluated together in order to determine that a particular goal or behavior is being met). • The requirements are an accurate elaboration of the defined objectives or goals • e.g., the use of temporal modal operators like “next”, “until”, “always”, and “eventually”, are appropriately used to reflect the desired behavior • The requirements adequately refine the higher-level requirements • Design or implementation-specific information is specified as constraints to the behaviors captured in the requirements Coverage of Model Consistency with Model Separation of Information

  38. Complete (IVV 09-1) • All the needed information to completely specify a desired behavior is identified (i.e., all preconditions, postconditions, and invariants are specified for the described behavior). • Threads of behavior are represented by more than one requirement, versus one compound requirement that attempts to capture the entire thread (i.e., that each requirement specifies only one “thing”). • The use of conjunctions (e.g., “and”, “or”) are restricted to preconditions, postconditions, and invariants. Coverage of Model

  39. Correlation Mapping & Requirement Evaluation

  40. Correlation Mapping & Requirement Evaluation

  41. Identifying Issues • L2 Rqmnt - The Project shall generate, route, transport, store and execute a sequence containing any of the following types of time-tagged commands:  absolute time, time relative to a sequence-external time value stored on-board, time relative to the execution of the previous command in the sequence. • L3 Rqmnt - The sequence time tags shall be either an absolute execution time, time relative to a sequence external time value, or a time relative to the execution of the previous command or command file. • L4 Rqmnt - The flight software shall provide the means for running onboard relative and absolute time, relative to a sequence external time value,  tagged sequences.

  42. Findings Requirements Model Identifying Issues Model Elements Findings/Issues Requirement Mapping

  43. Lessons Learned & Best Practices

  44. Challenges • Varying levels of detail between the SRM and requirements being validated • What vs. How – “Requirements Model” vs. “Design Model” • Terminology differences between SRM behaviors and requirements • Lack of adequate tools, work instructions, product descriptions/templates – particularly MKS Artifact Mapping capability

  45. Questions?

More Related