Process Improvement in the Aerospace Industry CMMI and Lean Six Sigma

1. Process Improvement in the Aerospace IndustryCMMI and Lean Six Sigma Warren Scheinin USC CS510

2. Agenda • Current Challenges Facing the Aerospace Industry • Current Industry Approaches • Capability Maturity Model Integrated • Lean Six Sigma • Northrop Grumman Approach

3. NDIA Top 5 Systems Engineering Issues (2006) • Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle. • Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development. • Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs. • The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry. • Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels. Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera

4. Agenda • Current Challenges Facing the Aerospace Industry • Current Industry Approaches • Capability Maturity Model Integrated • Lean Six Sigma • Agile • Northrop Grumman Approach

5. Heritage of Standards for Systems Engineering 2002 2002 ISO/IEC 15504 ISO/IEC 19760 (FDIS) (PDTR) 1998 1994 2002 EIA 632 ISO/IEC 15288 EIA / IS 632 (Full Std) 1994 (FDIS) (Interim Standard) Mil-Std-499B 1974 1998 1994 Mil-Std-499A 1969 EIA/IS 731 SE CM IEEE 1220 1998 Mil-Std-499 (Not Released) IEEE 1220 (Trial Use) (Interim Standard) (Full Std) 2002 CMMI- SE/SW/IPPD Legend Supersedes Source for Standards for Systems Engineering, Jerry Lake, 2002

6. The Frameworks QuagmireSarah A. Sheard, Software Productivity Consortium http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp

7. Two Complimentary Approaches to Process Improvement Data-Driven (e.g., Lean Six Sigma) • Clarify what your customer wants (Voice of Customer) • Critical to Quality (CTQs) • Determine what your processes can do (Voice of Process) • Statistical Process Control • Identify and prioritize improvement opportunities • Causal analysis of data • Anticipate your customers/ competitors (Voice of Business) • Design for Six Sigma Model-Driven (e.g., CMMI) • Determine the industry best practice • Benchmarking, models • Compare your current practices to the model • Appraisal, education • Identify and prioritize improvement opportunities • Implementation • Institutionalization • Look for ways to optimize the processes

8. Agenda • Current Challenges Facing the Aerospace Industry • Current Industry Approaches • Capability Maturity Model Integrated • Lean Six Sigma • Northrop Grumman Approach

9. What is the Capability Maturity Model Integrated? • The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management • Developed under the sponsorship of DoD • Consistent with DoD and commercial standards Three Constellations sharing common components and structure • CMMI for Development - used by engineering organizations • CMMI for Acquisition - used by buyers (e.g., govt. agencies) • CMMI for Services - used by service providers (e.g., help desk)

10. Basic Building Blocks – 22 Process Areas Implemented by the organization Implemented by each project Project Management • Project Planning • Project Monitoring and Control • Supplier Agreement Management • Integrated Project Management) • Risk Management • Quantitative Project Management • Process Management • Organizational Process Focus • Organizational Process Definition • Organizational Training • Organizational Process Performance • Organizational Performance Management • Engineering • Requirements Development • Requirements Management • Technical Solution • Product Integration • Verification • Validation • Support • Configuration Management • Process and Product Quality Assurance • Measurement and Analysis • Decision Analysis and Resolution • Causal Analysis and Resolution

11. SG 1 Establish Estimates SP 1.1 Estimate the Scope of the Project SP 1.2 Establish Estimates of Work Product and Task Attributes SP 1.3 Define Project Lifecycle Phases SP 1.4 Estimate Effort and Cost SG 2 Develop a Project Plan SP 2.1 Establish the Budget and Schedule SP 2.2 Identify Project Risks SP 2.3 Plan Data Management SP 2.4 Plan the Project’s Resources SP 2.5 Plan Needed Knowledge and Skills SP 2.6 Plan Stakeholder Involvement SP 2.7 Establish the Project Plan SG 3 Obtain Commitment to the Plan SP 3.1 Review Plans That Affect the Project SP 3.2 Reconcile Work and Resource Levels SP 3.3 Obtain Plan Commitment GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational Policy GP 2.2 Plan the Process GP 2.3 Provide Resources GP 2.4 Assign Responsibility GP 2.5 Train People GP 2.6 Manage Configurations GP 2.7 Identify and Involve Relevant Stakeholders GP 2.8 Monitor and Control the Process GP 2.9 Objectively Evaluate Adherence GP 2.10 Review Status with Higher Level Management GG 3 Institutionalize a Defined Process GP 3.1 Establish a Defined Process GP 3.2 Collect Improvement Information Expected Practices Provide Guidancefor Implementation & Institutionalization Project Planning – Implementation Project Planning - Institutionalization

12. Practice Ratings for the Organization/Projects

13. How is the CMMI Used for Process Improvement? IDEAL Model www.sei.cmu.edu/ideal/

14. Typical CMMI Benefits Cited in Literature • Reduced costs • 33% decrease in the average cost to fix a defect (Boeing) • 20% reduction in unit software costs (Lockheed Martin) • Faster Schedules • 50% reduction in release turnaround time (Boeing) • 60% reduction in re-work following test (Boeing) • Greater Productivity • 25-30% increase in productivity within 3 years (Lockheed Martin, Harris, Siemens) • Higher Quality • 50% reduction of software defects (Lockheed Martin) • Customer Satisfaction • 55% increase in award fees (Lockheed Martin)

15. Agenda • Current Challenges Facing the Aerospace Industry • Current Industry Approaches • Capability Maturity Model Integrated • Lean Six Sigma • Northrop Grumman Approach

16. What is Lean Six Sigma (LSS)? • Lean Six Sigma is a powerful approach to improving the work we do • LSS improvement projects are performed by teams • Teams use a set of tools and techniques to understand problems and find solutions • Lean Six Sigma integrates tools and techniques from two proven process improvement methods +

17. Tooearly Too late Too late Too early Defects Defects Reduce variation Delivery Time Delivery Time Spread of variation too wide compared to specifications Spread of variation narrow compared to specifications Six Sigma • A management philosophy based on meeting business objectives by reducing variation • A disciplined, data-driven methodology for decision making and process improvement • To increase process performance, you have to decrease variation • Greater predictability in the process • Less waste and rework, which lowers costs • Products and services that perform better and last longer • Happier customers

18. Define project scope Explore data Define control method Define Measure Analyze Improve Control Establish formal project Summarize& baseline data DMAIC Roadmap Identify possible solutions Identify needed data Characterize process & problem Implement Obtain data set Select solution Document Implement (pilot as needed) Evaluate data quality Update improvement project scope & scale Evaluate [Hallowell-Siviy 05]

19. Define DMAIC Toolkit Measure Improve Control Analyze Benchmark Contract/Charter Kano Model Voice of the Customer Voice of the Business Quality Function Deployment GQIM and Indicator Templates Data Collection Methods Measurement System Evaluation Cause & Effect Diagrams/ Matrix Failure Modes & Effects Analysis Statistical Inference Reliability Analysis Root Cause Analysis, including 5 Whys Hypothesis Test Design of Experiments Modeling ANOVA Tolerancing Robust Design Systems Thinking Decision & Risk Analysis PSM Perform Analysis Model Statistical Controls: Control Charts Time Series methods Non-Statistical Controls: Procedural adherence Performance Mgmt Preventive measures

20. Define project scope Explore data Evaluate pilot Define Measure Analyze Design Verify Quantify CTQs Design for Six Sigma (e.g., DMADV) Develop detailed design Identify customers Scale-up design Research VOC Design solution Refine predicted performance Establish formal project Document Benchmark Predict performance Develop pilot

21. Lean • Series of tools and techniques refined by Toyota and called the “Toyota Production System” • Called “Lean” by Womack, Jones and Roos in The Machine That Changed the World • Focused on increasing efficiency by eliminating non-value added process steps and wasteful practices • Being adopted world-wide by both manufacturing and transactional based organizations • Utilizes tools like “Value Stream Mapping,” “Just in Time” and “Kaizen” LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME

22. Wastes in Production CORRECTION MOTION Repair or Rework WAITING Any wasted motion to pick up parts or stack parts. Also wasted walking Any non-work time waiting for tools, supplies, parts, etc.. Types of Waste PROCESSING OVERPRODUCTION Producing more than is needed before it is needed Doing more work than is necessary INVENTORY CONVEYANCE Maintaining excess inventory of raw mat’ls, parts in process, or finished goods. Wasted effort to transport materials, parts, or finished goods into or out of storage, or between processes.

23. Organizational Adoption:Roles & Responsibilities • Champions – Facilitate the leadership, implementation, and deployment • Sponsors – Provide resources • Process Owners – Responsible for the processes being improved • Master Black Belts – Serve as mentors for Black Belts • Black Belts – Lead major Six Sigma projects • Typically requires 4 weeks of training • Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt • Typically requires 2 weeks of training

24. A Typical Lean Six Sigma Project in Aerospace • The organization notes that systems integration has been problematic on past projects (budget/schedule overruns) • A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s) • The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test • Procedures and criteria for better peer reviews are written, using best practices from past projects • A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem • The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects

25. Agenda • Current Challenges Facing the Aerospace Industry • Current Industry Approaches • Capability Maturity Model Integrated • Lean Six Sigma • Northrop Grumman Approach

26. Northrop Grumman Approach:Mission Success Requires Multiple Approaches Risk Management Systems Engineering Independent Reviews & Cost Estimates Training, Tools, & Templates Dashboards for Enterprise-Wide Measurement Communications & Best-Practice Sharing Robust Governance Model (Policies, Processes, Procedures) Program Effectiveness MissionAssurance & Enterprise Excellence Operations Process Effectiveness Effectiveness CMMI Level 5 for Software, Systems, and Services ISO 9001 and AS-9100 Certification Six Sigma

27. Organizational Infrastructure Required for CMMI Level 3 Policies, Processes,Templates & Tools Process Group Training Program Process Improvement Measurement Repositories Predictive Modeling Best-Practice Libraries Audits & Appraisals Communications Developing and maintaining mature processes requires significant time and investment in infrastructure

28. Northrop Grumman Approach:Institutionalizing Our Improvements We systematically analyze quality and process data and trends to determine how to improve our processes We improve our process assets based on internal and external best practices Deployed to programs Disposition Analysis Information ISO/AS9100 Findings • Systems/ Software Engineering Process Group • QMS Working Group • Program Management Advisory Board Industry Standards Policy Configuration Control Board CMMI Appraisal Findings InternalBest Practices Process Customer Comments Six Sigma Projects Procedures Independent Audits ExternalBest Practices Checklists and Guides Lessons Learned & Metrics Templates and Examples Tools msCAS PAL eToolkit Increasing program efficiency StartIt! My MS Portal Workbench PCDB

29. Northrop Grumman Approach:Lessons Learned • Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a level” • Reinforces that change (improvement) is a way of life • Benefits results from institutionalizing local improvements across the wider organization • CMMI establishes the needed mechanisms • CMMI and Lean Six Sigma compliment each other • CMMI can yield behaviors without benefits • Lean Six Sigma improvements based solely on data may miss innovative improvements (assumes a local optimum) • Training over half the staff as Lean Six Sigma Green Belts has resulted in a change of language and culture • Voice of Customer, data-driven decisions, causal analysis, etc. • Better to understand/use tools in everyday work than to adopt the “religion”