Optimizing NASA IV&V Benefits Using Simulation

1. Optimizing NASA IV&V Benefits Using Simulation Grant Number: NAG5-12736 David M. Raffo, Ph.D College of Engineering and Computer Science School of Business Administration Portland State University davidr@sba.pdx.edu

2. Problem • There is a critical need allocate scarce IV&V resources in the most effective manner possible • Questions • How can the economic benefit of IV&V technologies be optimized? • What is the best strategy for applying IV&V technologies? • What is the benefit of applying one combination of IV&V techniques to a given process vs another? • The goal of this project is to provide NASA with a flexible decision support tool that can be applied to NASA projects and NASA IV&V to assess the benefits of various quality assurance and IV&V policies on a given NASA project.

3. Relevance to NASA • Contributes to mission assurance & improved project management • More cost effective management of IV&V • Potential to be applied to multiple NASA projects at multiple NASA Centers (started project with KSC) • Develop method for rapidly tailoring simulation models in the NASA environment • Comparison of effectiveness of DES models with COCOMO & SD models • Augment DES models with SD factors

4. Importance/ Benefits • What if a NASA project canceled due to software delays and cost overruns had this tool: • They could have assessed the benefits vs. costs of IV&V early on. They could have let IV&V in earlier or had a broader scope. • They could have had better estimates, validated those estimates, and quantified the impacts of scope and budget changes. • They could have validated processes they had or designed better processes. • They could have assessed the impacts of developer and project manager experience on productivity and its impact on the project

5. Approach • This tool is based on Software Process Simulation techniques and the Process Tradeoff Analysis Method (PTAM) based on work originally done at the SEI. • Key Advantages of using a Simulation Based Approach: • Compare alternative process scenarios and make tradeoffs • Forward looking at what is “possible” not just what has been • Extensive “What if” analyses to explore alternatives desired by management • Compute a ROI and develop a quantitative business case • Update initial plans; choose best corrective action

6. Better Process Decisions Software Development Process Process Performance Cost, Quality, Schedule SW Process Simulation Model Project Data Process and Product Approach

7. Process Analysis Tradeoff Tool • Large-Scale software project planning support • Rapidly deployable models dramatically reduce time to develop and modify • Platform for analyzing process tradeoffs and impact of QA, IV&V or other process technologies (tools) • NASA Project studying the impact of IV&V • IV&V can be applied at one or more phases over the entire software development lifecycle • Alternative IV&V application configurations can be compared (ROI)

8. Corrective Action Actual Data Current Trajectory Basis for Project Management Original Plan X

9. Accomplishments • Delivered prototype version of simulation tool to NASA. This prototype contains: • IEEE 12207 Software Development Process Lifecycle Model that can be adapted to a variety of NASA projects • Nine potential IV&V techniques (using new IV&V WBS) • Increased PF to 7 - Researchers using NASA data (2 projects) • Reported work at ProSim’04

10. Next Steps • Creating new software process life cycle models • Adding more IV&V Techniques • Continuing NASA data analysis with goal of developing business case scenarios • Continuing tool development • Joint research with KSC • Continuing research in Optimization Theory • Assessing variety of techniques including: Treatment Learning,DEA, and Financial Measures

11. The End