ICS 52: Introduction to Software Engineering

ICS 52: Introduction to Software Engineering Lecture Notes for Summer Quarter, 2003 Michele Rousseau Topic 14

Process Improvement • Capability Maturity Model • ISO 9000 • PSP

Process improvement • Understanding existing processes • Introducing process changes to achieve organisational objectives • usually focused on: • quality improvement • cost reduction • schedule acceleration • Most work so far has focused on • defect reduction to improve Quality • Testing can’t do it all

Process improvement stages • Process analysis • Model and analyse (quantitatively if possible) existing processes • Improvement identification • Identify quality, cost or schedule bottlenecks • Process change introduction • Modify the process to remove identified bottlenecks • Process change training • Train staff involved in new process proposals • Change tuning • Evolve and improve process improvements

The process improvement process

Process and product quality • Process quality  product quality • These are closely related • A good process is usually required to produce a good product • For manufactured goods, process is the principal quality determinant • For design-based activity, other factors are also involved especially the capabilities of the designers

Principal product quality factors

What is CMM? • Capability Maturity Model • Developed by the software community in 1986 with leadership from the SEI. • Has become a de facto standard for assessing and improving processes related to software development • Has evolved into a process maturity framework • Provides guidance for measuring software process maturity • Helps establish process improvement programs

What is the Software CMM? • “A common-sense application of process management and quality improvement concepts to the software development and maintenance” • A model for organizational improvement

Software Capability Maturity Model. Maturity Levels Process Capability Indicate Contain Key Process Areas Goals Organized by Achieve Common Features Implementation Address Contain Key Practices Activities Describe CMU/SEI-93-TR-24 p. 29

What makes up the CMM? • The CMM is organized into five maturity levels: • Initial • Repeatable • Defined • Manageable • Optimizing • Except for Level 1, each maturity level decomposes into several key process areas that indicate the areas an organization should focus on to improve its software process.

Each Maturity Level • 1. Initial : • ad hoc process. Success depends on individual effort. • 2. Repeatable : • Basic management processes: cost, schedule and functionality • 3. Defined : • Activities are documented, standardized and integrated into an organization-wide software process. • 4. Managed : • Detailed measures are collected: software and product quality. • 5.Optimizing : • Continuous process improvement: quantitative feedback from the process and from testing new ideas and technologies.

Five Levels of Software Process Maturity Continually improving process Optimizing Predictable process Focus on process improvement Managed Process measured and controlled Level 5 Standard, consistent process Defined Process characterized, fairly well understood Level 4 Disciplined process Repeatable Level 3 Can repeat previously mastered tasks Initial Unpredictable and poorly controlled Level 2 Level 1

Key Process Areas • KPAs associated with each maturity level describe functions that must be present to satisfy good practice at a particular level. (Except Level 1) • Each KPA is described by: • Goals– Overall objectives • Commitments – Requirements that must be met to achieve the goals. • Abilities – Things that must be in place to enable the organization to meet the commitment. • Activities – Specific tasks required to achieve the KPA function • Methods for monitoring implementation • Methods for verifying implementation • Each KPA is defined by a set of key practices that contribute to satisfying its goals. (ie policies, procedures, and activities)

Level 2 KPAs - Purpose * Paraphrased

Level 2 KPAs - Purpose (Cont.) Changes in commitments are agreed to by affected groups All activities are planned and tracked * Paraphrased

Level 3 KPAs - Purpose (Cont.) * Paraphrased

Interesting CMM Facts • The number of companies using CMM to assess their software management practices more than doubles every five years (since 1987). • Software Quality Assurance is the biggest obstacle for organizations trying to move from level 1 to level 2. • Organization Process Definition is one of the biggest obstacles for organization trying to move from level 2 to level 3.

… more interesting facts • On average, it takes an organization: • 25 months to move from level 1 to 2 • 22 months to move from level 2 to 3 • 36.5 months to move from level 3 to 4 • About a third of companies engaged in CMM are located overseas (primarily India), and are 3 times more likely to reach CMM level 4 or 5 than US organizations. • Only about 23% of organizations surveyed eventually move from level 2 to level 3 or higher.

What improved maturity provides Based on data from 1300 applications, average 200,000 LOC Source: Master Systems, Inc.

Characteristics of Immaturity • Software process improvised during the course of a project. • Even if process is specified, it is not rigorously followed or enforced. • Reactionary, focus on solving immediate crises. • Hard deadlines often mean a compromise in functionality and/or quality. • No objective basis for judging product quality or for solving process problems. • Quality is difficult if not impossible to predict.

Characteristics of Maturity • Able to manage software development and maintenance organization/project wide. • There is a prescribed, mandated, and enforced process. • Process is consistent with the way that work actually gets done. • Process is updated and improved as necessary. • Roles and responsibilities within the process are clear. • Quality is measured and monitored, and an objective basis for judgment exists. • The necessary infrastructure for supporting the process exists. • Workers see the value in the process.

Greater Maturity Can Bear Fruit • SE division of Hughes aircraft spent @$500K over a three year period for assessment and improvement programs. By the end of the three year period, assessed at CMM Level 3. Estimated savings of @$2M annually as a result (less overtime, less rework, greater productivity, etc.) • Equipment Division of Raytheon rise to CMM Level 3, at an estimated cost of @$580K resulted in 2-fold increase in productivity along with savings of @$15.8M in rework costs. • Motorola GED (CMM Level 4) documented significant • reduction in cycle time • reduction in defect rates • increase in productivity

ISO 9000 - Background • International set of standards for quality management (ISO 9000:2000, ISO 9001:2000, ISO 9004:2000, etc.) • ISO is name adopted by the International Organization for Standardization (not an acronym) – comes from isos “equal” • ISO 9000 is the most popular quality standard in the world • Over 13,000 standards issued since 1946 • Made up of representative bodies from over 140 countries • It applies to almost all types of organizations regardless of their function or product.

From “The Dilbert Principle”

ISO 9000 – What is it? • ISO 9000 is primarily concerned with "quality management". • ISO 9001:2000 specifies requirements for a quality management system for any organization that needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements and aims to enhance customer satisfaction, in all business sectors • It involves the development of a quality system that meets the quality requirements of the ISO standards.

Contents of Software Quality Planfrom ISO9000 Management responsibility Quality system Contract review Design control Quality control Purchasing Customer supplied info Configuration management Process control Inspection and testing Inspection and testing equipment Control of non-conforming product Corrective action Handling, storage, packing and delivery Quality records Internal quality audits Training Software maintenance Statistical techniques Control of the development environment

ISO 9000 series of standards: • represents an international consensus on good management practices • guidelines on what constitutes an effective quality management system • serves as framework for continuous improvement

ISO 9000 and quality management

ISO 9000 - Certification • Quality standards and procedures should be documented in an organisational quality manual • External body may certify that an organisation’s quality manual conforms to ISO 9000 standards (namely ISO 9001) • Customers are, increasingly, demanding that suppliers are ISO 9000 certified

ISO vs CMM • CMM and the ISO 9000 series of standards share common concerns with quality and process management. • CMM emphasizes continuous improvement • ISO deals with minimum criteria of quality systems • There is a clear correlation between the key processes in the CMM and the quality management processes in ISO 9000 • ISO 9000 has little explicit support for continuous improvement

ISO vs CMM (2) • The CMM is more detailed and prescriptive and includes a framework for improvement • An ISO 9001-compliant organization would not necessarily satisfy all of the CMM level 2 key process areas (it would satisfy most of the level 2 goals and many level 3 goals. • Organisations rated as level 2 in the CMM are likely to be ISO 9000 compliant

ISO9000 and CMM compared CMMISO 9001 • Specific to software development Intended for most industries • Used in USA, less widely Recognised and accepted in most elsewhere countries • Provides detailed and specific Specifies concepts, principles and definition of what is required safeguards that should be in place for given levels • Assesses on 5 levels Establishes one acceptable level • CMM Level 2 - 3  ISO 9000 • Relevant to Stabilises the customer - supplier s/w development process relationship • No time limit on certification Certification valid for three years • No ongoing audit Auditors may return for spot checks during the lifetime of the certificate

Personal Software Process (PSP) • Developed by SEI in 1994 • A measurement and analyses framework to help you characterize your process • A defined procedure to help you to improve your performance • PSP principles • System quality depends on the quality of its worst components • Component quality depends on individual developers

Overview of CMM and PSP • CMM sets out the principal practices for managing the processes in large-scale software development • PSP sets out the principal practices for defining, measuring and analysing an individual’s own processes

PSP • PSP applies a CMM-like assessment for individual work • Measurement & analysis framework to help you characterize your process • Self-assessment and self-monitoring • Prescribes a personal process for developing software • defined steps • Forms • Standards • Assumes individual scale & complexity • Well-defined individual tasks of short duration

PSP Overview • The PSP is introduced in 7 upward compatible steps (4 levels) • Write 1 or 2 small programs at each step • Assume that you know the programming language • Gather and analyze data on your work • Many standard forms & spreadsheet templates • Use these analyses to improve your work • Note patterns in your work

PSP Evolution PSP3 Cyclic development Cyclic Personal Process PSP2.1 Design templates PSP2 Code reviews Design reviews Personal Quality Management PSP1.1 Task planning Schedule planning PSP1 Size estimating Test report Personal Planning Process PSP0.1 Coding standard Size measurement Process improvement proposal (PIP) Baseline Personal Process PSP0 Current process Time recording Defect recording Defect type standard

PSP Evaluation • Humphrey has used in SE courses • Improvements in time-to-compile, quality and productivity • Patchy, but promising use in industry • E.g. Nortel (Atlanta) • Still immature • Requires large overhead for data gathering • Not clear that you should use permanently or continually

PSP/TSP/CMM

ICS 52: Introduction to Software Engineering