Program Model Checking: Case Studies and Practitioner’s Guide

1. Program Model Checking: Case Studies and Practitioner’s Guide John Penix, ARC Howard Hu, JSC

2. Problem • Research has shown that program model checking and be effective at detecting critical software errors that are difficult to detect via testing. • However, applying model checking remains a “black art”. Best practices are only just emerging and remain a relatively ad-hoc combination of methods. Program Model Checking Case Studies and Practitioner’s Guide

3. Approach • Assemble the emerging best practices in program model checking • Demonstrate and validate their use in several case studies • Document the results in a Practitioner’s Guide for Program Model Checking Program Model Checking Case Studies and Practitioner’s Guide

4. Importance and Benefits The guidebook will provide: • Methods for formalizing requirements and identifying critical properties – may help a test team develop verification goals for V&V or IV&V • Test driver development and test coverage assessment methods to support testing or model checking • Methods for improving the verifiability of designs • Guidance in configuring model checking options and organizing and validating model checking results Program Model Checking Case Studies and Practitioner’s Guide

5. Importance and Benefits Testing covers one path Model checking searches all paths Improves testing: • Ability to control thread scheduling and environment responses • Stress test critical software states • Directed search for specific errors: deadlock, race conditions, assertion violations, … Program Model Checking Case Studies and Practitioner’s Guide

6. Relevance to NASA Initial Case Study Application Shuttle Abort Flight Manager (SAFM) Provides onboard abort performance assessment during powered flight and landing site evaluation and monitoring during glided flight in Cockpit Avionics Upgrade. 30KLOC in C++. Program Model Checking Case Studies and Practitioner’s Guide

7. Accomplishments • Kick-off meeting at JSC with overview of SAFM • Delivery of requirements document, design document, source code and test infrastructure from JSC to ARC • Evaluation of SAFM source code and requirements for applicability to model checking & identify critical issues • Hosted SAFM test lead at ARC for a week and to elicit requirements and design properties that are currently unchecked. • Identified Sequencer as a critical subsystem • Obtained the SAFM test system, requirements simulator, and test data from the SAFM development team. • Set up SAFM build & test environment at ARC • Gathered data on existing test coverage Program Model Checking Case Studies and Practitioner’s Guide

8. Next Steps • Assessment of critical SAFM properties and current test coverage • Evaluate use of property patterns to formalize critical SAFM requirements Program Model Checking Case Studies and Practitioner’s Guide