1 / 24

Migrating to a Software Assurance Standard

Migrating to a Software Assurance Standard. 2008 ADF Software Symposium FLTLT Patrick Redmond SCI-DGTA. Overview. The Problem with Legacy Software Migrating to a Software Assurance Standard The Potential Challenges Bounding the Change Low-Level Requirements Traceability

mathilda
Télécharger la présentation

Migrating to a Software Assurance Standard

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Migrating to a Software Assurance Standard 2008 ADF Software Symposium FLTLT Patrick Redmond SCI-DGTA

  2. Overview • The Problem with Legacy Software • Migrating to a Software Assurance Standard • The Potential Challenges • Bounding the Change • Low-Level Requirements • Traceability • Structural Coverage • Case Study • MIL-STD-498 to DO-178B • Considered throughout.

  3. The Problem with Legacy Software • Numerous ADF platforms acquired where no software assurance standard has been explicitly applied. • Only MIL-STD-498 or DOD-STD-2167A. • Development standards do not define how well software has to be constructed. • During development, somebody made the decision to stop testing. • Why? • During development, somebody made the decision to review source code. • Why? • To what extent should the ADF rely on this software? • To what extent does the ADF rely on this software?

  4. Migrating to a Software Assurance Standard • DGTA requires that a software assurance standard be applied to legacy software systems. • All development methods apply a level of assurance, but is that level acceptable? • Applying a software assurance standard to each modification of in-service software progressively increases the integrity. • Applies ‘Cancer Theory’ to Legacy Software • Two methods: • Service History Argument and Application of Recognised Standard • e.g. RTCA DO-178B • Software Assurance Task Matrix • Negotiate a custom software assurance “standard” with DGTA

  5. How does migration work? DGTA considers the software to be compliant with the applied software assurance standard from this point forward. ? ? ? ? ? ? ? ? ? ? Legacy Software of Unknown Integrity Build y Software of Known(?) Integrity ? ? ? Area Impacted by Build x ? Known Integrity ? ? ? ? ? Build x

  6. Service History Argument / Recognised Standard • Establish a Service History Argument that demonstrates that the current software is acceptably safe. • Need to consider, among others: • Configuration Management • Problem Reporting • Change Control • Relevance of Product Service History • Operating Environments • Safety Related Problems • Design and Code Errors • Error Rates • No need to create a service history argument for in-service software, ADF has already determined that it is acceptably safe. • Only applies to in-service, legacy software. Does not apply to acquisitions. • Does not apply if there is a substantial change of context in which the software is used. • Apply a recognised software assurance standard to the next modification. • DGTA then consider the software to be compliant with that standard.

  7. Software Assurance Task Matrix 178B Negotiate with SCI Low-level requirements are defined. High-level requirements coverage. High-level requirements are defined. Source code complies with low-level reqs. Statement coverage. Source code is verifiable. Decision coverage. Decision coverage. Statement coverage plus untested code analysis.

  8. The Potential Challenges • Bounding the Change • How much “re-assurance” needs to be done? • Traceability • Low-Level Requirements • Structural Coverage

  9. The Easy Ones • Planning • Additional Considerations • Tool Qualification • Development of High-Level Requirements, Source Code and Executable Object Code • Verification of High-Level Requirements • Configuration Management • Quality Assurance

  10. Bounding the Change • Software assurance standard only needs to be applied to the scope of the modification. • What is the scope of the modification? • The things that are changed, and • The things affected by the things that are changed. • Change Impact Analysis: • Traceability • Memory Margin • Timing Margin • Data Flow • Control Flow • Input/Output • Development Environment and Process • Operational Characteristics • Certification Maintenance Requirements • Partitioning Analysis

  11. Traceability • Full traceability data was not recorded for many legacy software systems. • Software assurance standards generally require traceability to source code. • When applying a software assurance standard to a legacy software system, how much traceability data needs to be generated? • DGTA Position: Trace down and up once. • Trace down from new or modified high-level requirements to affected and new low-level requirements, to affected and new code. • Trace up from affected and new code to low-level requirements and to high-level requirements.

  12. Example – MIL-STD-498 to DO-178B • This example will be considered a number of times. • A legacy software system developed to MIL-STD-498 with typical artefacts. • Can cause or contribute to Major hazards. • Migrating to DO-178B Level C objectives.

  13. Traceability HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 HLR1 New High-Level Requirement HLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR1 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR2 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 LLR3 New Low-Level Requirement LLR1 LLR2 LLR3 Modified Low-Level Requirement SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM1 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM2 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM3 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM4 SCM1 SCM2 SCM3 SCM4 HLR2 LLR4 New Source Code Modified Source Code SCM5

  14. Low-Level Requirements • Some legacy systems have no low-level requirements. • Others have design descriptions that are not refined enough to be low-level requirements. • Others have design descriptions that do not drive source code development (source code is developed from requirements). • How many low-level requirements need to be defined or redefined? • DGTA Position: Each affected low-level requirement and each low-level requirement identified by the down and up trace.

  15. Low-Level Requirements HLR1 HLR2 HLR2 LLR1 LLR1 LLR4 LLR4 LLR2 LLR3 LLR3 SCM3 SCM3 SCM4 SCM4 SCM5 SCM5 SCM1 SCM2 SCM2

  16. Low-Level Requirements • Verify that low-level requirements LLR1, LLR3 and LLR4: • Comply with high-level requirements. • Are accurate and consistent. • Conform to standards. • Are traceable to high-level requirements. • Ensure that low-level requirements LLR1, LLR3 and LLR4 are sufficiently refined to be directly translatable to source code. • May need to further refine one low-level requirement into several. • Leave LLR2 as is.

  17. Structural Coverage • Software assurance standards generally require assessment of structural coverage in order to demonstrate that testing is complete. • Purpose of Structural Coverage Objectives is to: • Identify shortcoming in requirements based test cases or procedures, • Identify inadequacies in software requirements, • Identify dead code, and • Identify deactivated code. • How do these measures apply to modification of legacy software systems? • DGTA Position: • For requirements based measures: each new, modified or affected requirement needs to be tested. • For structure based measures: each new or modified source code module.

  18. Structural Coverage Normal Range and Robustness Tests Normal Range and Robustness Tests Statement Coverage HLR1 HLR1 HLR2 HLR2 LLR1 LLR1 LLR4 LLR4 LLR2 LLR3 LLR3 SCM3 SCM3 SCM4 SCM4 SCM5 SCM5 SCM1 SCM2 SCM2

  19. Structural Coverage • Normal Range and Robustness Tests • High-level requirements that have been added or modified. • High-level requirements where a dependent low-level requirement has been added or modified. • High-level requirements where the implementation (source code) of a dependent low-level requirement has been added or modified. • Low-level requirements that have been added or modified. • Low-level requirements where the implementation (source code) has been added or modified. • Structural Coverage Objectives: • Structural coverage of all new or modified source code modules.

  20. Additional Testing • Also need to consider: • Data Dependencies • Has the change impacted data that other functions rely upon? • Control Flow Dependencies • Has the change inadvertently/adversely disrupted control flow or coupling? • Timing Dependencies • Has the change violated a timing constraint? • Memory Space Dependencies • Has the change violated memory constraints or used memory space assigned to other functions?

  21. What about bug fixes? • A large part of in-service support is the rectification of bugs. • Bug fixes may not commence at the requirements level, may start with an identified fault in the source code. • To what extent should software assurance standards be applied to bug fixes? • DGTA Position: Trace up from the modified source code.

  22. What about bug fixes? Low-Level Requirements Normal Range and Robustness Tests Additional traceability data Structural Coverage HLR3 HLR3 HLR4 HLR4 LLR5 LLR5 LLR8 LLR6 LLR7 LLR7 SCM8 SCM8 SCM9 SCM10 SCM6 SCM7

  23. Conclusions • DGTA requires the application of a software assurance standard to legacy software systems. • All development methods provide a level of assurance, writing it down can identify gaps. • A software assurance standard can be applied either by: • A Product Service History Argument and application of a recognised software assurance standard, or • A Software Assurance Task Matrix. • The software assurance standard need only be applied to the current modification. • Determine extent through change impact analysis. • For legacy software systems, a large number of assurance objectives are probably already being achieved. • Meeting a recognised software assurance standard will probably require additional effort in the areas of: • Traceability, • Low-Level Requirements, and • Structural Coverage

  24. Questions?

More Related