IS 553 Advanced Systems Development Practices

IS 553Advanced Systems Development Practices James Nowotarski 11 May 2004

Course Map Underpinnings . Introduction . Essentials Content . Rational Unified Process . Agile Implementation . Metrics . CMM . Distributed development . Tools & training Briefings (Term Papers) 1 2 3 4 5 6 7 8 9 10 11 Week Assignments Quizzes (Distr. Dev.) (RUP) (Agile) (CMM)

Today’s Objectives • Understand Capability Maturity Model • Objectives • Components • Relationship to Other Frameworks • Results to Date • Latest Thinking

Today’s Agenda Topic Duration • CMM Overview 30 minutes • CMM Activity 30 minutes *** Break 15 minutes • Current Event Reports 20 minutes • CMM Overview (cont.) 20 minutes • CMM Jeopardy 40 minutes • CMMI 10 minutes • Term Paper - Open 30 minutes

What is CMM Overview • CMM = Capability Maturity Model • Developed in1991 by Software Engineering Institute (SEI) to assess the software engineering capability of government contractors • A framework for software process improvement (SPI) that has gained wide acceptance in the industry • A roadmap of effective practices that build on one another in a logical progression • coherent • ordered set of incremental improvements

What is CMM Benefits • Practical • Structured • Proven • Reputation • Quantitative benefits: • productivity: 9-67% • time to market: 15-23% • post-release defects: 10-94% • business value ratio: 4.0-8.8:1

What is CMM Overview • Different capability maturity models • Software CMM (SW-CMM) • People CMM • Systems Engineering CMM • Software Acquisition CMM • Integrated Product Development CMM • Team Software Process • Personal Software Process

What is SEI Overview • SEI = Software Engineering Institute • Federally funded research & development center • Sponsored by Department of Defense • Affiliated with Carnegie Mellon University in Pittsburgh • Established in 1984 • Research and publications oriented • Mission is to improve the state of the practice of software engineering

Optimized (5) Managed (4) Defined (3) Repeatable (2) CMM Levels Initial (1)

Indicate Contain Process capability Key process areas Achieve Contain Goals Key practices Key process areas (KPAs) Maturity levels

Key process areas (KPAs) • Key Process Areas • Identify the issues that must be addressed to achieve a maturity level • Key Practice • Activities and infrastructure that contribute the most to the effective implementation of the KPA

Key Process Area Process capability Disciplined process Software project planning Key Practice Goal A plan is developed that appropriately and realistically covers the software activities and commitments Estimates for the size of software products are derived according to a documented procedure Key process areas (KPAs) Maturity level Level 2: Repeatable

Maturity Questionnaire Response Analysis 3 2 KPA Profile Findings based on the CMM On-site visit Interviews & document reviews 5 4 6 CMM Appraisal Method Team Selection 1

Appraisal Methods • Software Process Assessments (SPA) • Performed in open, collaborative environment • Focuses on improving the organization’s software process • Now called CMM-Based Appraisal for Internal Process Improvement (CBA-IPI) • Software Capability Evaluations (SCE) • Performed in a more audit-oriented environment • Focuses on identifying risks associated with a contractor • Team’s recommendation will help select contractors or set fees

Level 1: Initial • Instability • Dependence on “heroes” • Inability to meet targets • Key process areas: • none

Class Activity • Summarize and explain to the rest of the class: • Maturity levels 2-5 • The 18 key process areas

Level 2: Repeatable

Level 3: Defined

Level 4: Managed

Level 5: Optimizing

Level 2: Repeatable • Track project results, which form basis for future project plans • Basic processes in place for: • requirements management • supplier management • standards • Key process areas • requirements management • software project planning • software project tracking and oversight • software subcontract management • software quality assurance • software configuration management

Level 3: Defined • Software engineering and management processes are defined • Software engineering process group (SEPG) exists • Organization-wide training • Key process areas: • organizational process focus • organizational process definition • training program • integrated software management (i.e., tailoring) • software product engineering • intergroup coordination • peer reviews

Level 4: Managed • Stable and measured processes • Measurements are captured across the organization • productivity • quality • Processes are predictable • Key process areas: • quantitative process management • software quality management

Level 5: Optimizing • Continuous process improvement • Defect prevention • root cause analysis • Processes are predictable • Key process areas: • process change management • technology change management • defect prevention

As maturity increases • Less prone to miss targets • Less variation around the target goals • Better, faster, cheaper development

CMM Issues in the Real-World • “Level envy” • Areas not addressed: • Business strategy and linkage to IT • Operations, help desk, support • Management of the IT human resource • Application portfolio • Tools • Many question whether it is worth the effort to pursue levels 4 and 5

19.3% Initial CMM Maturity Profile 1998 thru August 2002 100% 90% 80% 70% 60% 43.2% 50% % of Organizations 40% 23.4% 30% 20% 7.3% 6.8% 10% 0% Repeatable Defined Managed Optimized Based on assessments from 1998-2002 of 1124 organizations

CMM Maturity Profile April 2002 – October 2003 100% 90% 80% 70% 60% 50% % of Organizations 33.3% 40% 26.4% 23.0% 30% 20% 11.5% 5.7% 10% 0% Repeatable Defined Managed Optimized Initial Based on assessments from 2002-3 of 87 organizations

USA and Offshore Profiles 1998 thru August 2002 100% 90% 80% 70% 60% 50% 47.4% % of Organizations 37.6% 40% 25.3% 30% 23.3% 22.0% 13.2% 14.0% 20% 10.0% 10% 5.3% 2.0% 0% Initial Repeatable Defined Managed Optimized Offshore USA Based on 645 U.S. organizations and 479 offshore organizations

USA and Offshore Profiles April 2002 – October 2003 100% 90% 80% 70% 54.5% 60% 50% % of Organizations 40% 29.6% 27.8% 30% 21.2% 20.4% 15.2% 14.8% 20% 7.4% 6.1% 10% 3.0% 0% Initial Repeatable Defined Managed Optimized Offshore USA Based on 33 U.S. organizations and 54 offshore organizations

Time to Move Up # of months to move to next level 75 50 Largest observed value that is not an outlier 28 75th percentile Recommended time between appraisals (18-30 mos) 23 22 25 17 Median (50th percentile) 25th percentile Smallest observed value that is not an outlier 0 1 to 2 4 to 5 2 to 3 3 to 4

CMM Market Pressure As of late 2002, approximately 50 of the 70 or so publicly-acknowledged Level 5 CMM-certified organizations were in India • Marketing tool to win clients, who are based predominantly in US and Europe • Clients using Indian service providers should have certain key processes in place: • service level agreements • identifying business requirements • scoping requirements • managing changes

CMM-based Software Process Improvement (SPI) • Time and cost often exceed expectations • 18-24 months to advance 1 level • Can cost $2K per software engineer per year • 1-2% full-time resources (e.g., 5-10 in a 500-person organization) • 2-4% of rest of organization’s time • Difficult KPAs • Planning and tracking • Key success factors • Senior management is engaged • Participation and buy-in at all levels, including middle management and technical staff • Clearly stated, well understood SPI goals • Clear assignment of responsibility • SEPG staffed by highly respected people

Software Process Improvement Models A number of models enable software development organizations to compare their practices to a set of “best practices” • ISO 15504 • ISO 9000-3 • TickIT IT specific models General models • Total Quality Management (TQM) • Six Sigma

Software Process Improvement Models ISO 15504 • International collaborative effort (including SEI) • Sparked by an investigative study sponsored by the U.K. Ministry of Defense (MOD) • Objective: To develop a standard in the area of software process assessment • establish a common framework for expressing the process capability ratings resulting from a 15504-conformant assessment • provide a migration path for existing assessment models and methods wishing to become 15504-conformant

Software Process Improvement Models The Integrated CMM (CMMI) has emerged as the latest thinking from SEI • Over time, proliferation of CMMs: • Capability Maturity Model for Software (SW-CMM®) • Systems Engineering Capability Model (SECM) (may or may not include software) • Integrated Product Development Capability Maturity Model (IPD-CMM) • Software acquisition • Many organizations wish to integrate improvement efforts across disciplines • Differences among these multiple models made integration difficult • SEI developed common framework to support integration of current and future discipline-specific maturity models • The common framework is called the Integrated CMM (CMMI) • “Each CMMI model is designed to be used in concert with other CMMI models, making it easier for organizations to pursue enterprise-wide process improvement at their own pace”

Integrated CMM (CMMI) • Incorporates and extends: • Capability Maturity Model for Software (SW-CMM®) • Systems Engineering Capability Model (SECM) • Integrated Product Development Capability Maturity Model (IPD-CMM) • Supplier sourcing • CMMI-SW model released August 2002 • SW-CMM® now sunsetted CMMI integrates process improvement models for product and service development and maintenance

CMMI • What’s different about CMMI-SW: • New process areas • Modern best practices added • Generic goal added to each process area • Continuous and staged representations • Stronger linkage to business objectives and customer needs • Greater alignment with relevant ISO standards • Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) V1.1 as a replacement for CMM-Based Appraisal for Internal ProcessImprovement (CBA IPI) and Software Capability Evaluation (SCESM)

CMMI • What’s different about CMMI-SW (cont.) • Still outside scope of CMMI: • People CMM (P-CMM) • Software Acquisition CMM (SA-CMM) • New names for maturity levels

CMMI • New names for maturity levels:

Defect Prevention Causal Analysis & Resolution Technology Change Management Organizational Innovation & Deployment Process Change Management Level 5 Optimized Level 4 Managed Quantitative Process Management Organizational Process Performance Software Quality Management Quantitative Project Management Quantitatively Organization Process Focus Organization Process Focus Organization Process Definition Organization Process Definition Training Program Organizational Training Integrated Software Management Integrated Project Management Risk Management Software Product Engineering Requirements Development Technical Solution Product Integration Intergroup Coordination Verification Peer ReviewsValidation Decision Analysis & Resolution Level 3 Defined Requirements Management Requirements Management Software Project Planning Project Planning Software Project Tracking & Oversight Project Monitoring & Control Software Subcontractor Management Supplier Agreement Management Software Quality Assurance Product & Process Quality Assurance Software Configuration Management Configuration Management Measurement & Analysis Level 2 Repeatable Managed SW-CMM v1.1 vs. CMMI Process Areas

For more information CMM • http://www.sei.cmu.edu/cmm/cmm.articles.html • http://www.sei.cmu.edu/publications/documents/02.reports/02tr029.html (600+ page pdf) CMMI

Today’s Agenda Topic Duration • CMM Overview 30 minutes • CMM Activity 30 minutes *** Break 15 minutes • Current Event Reports 20 minutes • CMM Overview (cont.) 20 minutes • CMM Jeopardy 40 minutes • CMMI 10 minutes • Term Paper – Open 30 minutes

IS 553 Advanced Systems Development Practices