70 likes | 79 Vues
Injecting Faults for Error Evaluation. NASA Glenn Research Center Kalynnda Berens, SAIC Richard Plastow, SAIC. The Problems:. Unknowns in software systems Acquired software (COTS, open source) Used more often in systems Lack of documentation No or little insight into the software
E N D
Injecting Faults for Error Evaluation NASA Glenn Research Center Kalynnda Berens, SAIC Richard Plastow, SAIC
The Problems: • Unknowns in software systems • Acquired software (COTS, open source) • Used more often in systems • Lack of documentation • No or little insight into the software • Interfaces to hardware/other systems • Need a way to test and verify the functionality and reliability • Goal: Find errors on the ground…before it flies!
Our approach • Gather system information • Determine interfaces/COTS • What can go wrong? • Common errors • Standards • Requirements • Create test cases • Execute tests (aka Break the System) • Feedback errors to developer (and duck)
Importance and Benefits • Standardized approach to test systems with COTS and hardware interfaces • Roadmap of where to look to determine what to test • In latest pilot project, found 3 critical errors • Software crash with reboot • Invalid telemetry • Incorrect fan speed control
Relevance to NASA • Allows use of COTS with improved reliability and a lower error rate • Can be used with all types of software • Two NASA pilot projects
Accomplishments • Found additional errors in two diverse projects • Projects took our input and fixed the problems • Draft of Fault Injection Methodology document
Next Steps • In-depth case study (ISS flight payload) • Update Fault Injection Methodology document • Record all the details – problems as well as successes • Compare results to other defect detection mechanisms • Written for those who want to try the technique • Release FI Methodology and Case Study