1. PM00.10Project Management Preparation for Success * Manage Metrics *

2. Manage Metrics Why do we measure? • Process improvement • Improved performance on this project • Improved performance on future projects • Improved organization performance • Declaring success • Validate duration, cost, effort and quality objectives for the project

3. Manage Metrics • Can be the hardest of the project management processes • Often ignored • Must collect and analyze metrics to conduct a good quality management process. • Beginning state of quality • Tell whether quality is increasing or decreasing

4. Manage Metrics • Some value from basic metrics like cost, effort and duration • More value for medium / large projects • Most value on an organization wide basis

5. Manage Metrics • Understand the cost of the metrics (and the value) • If you collect metrics, use them • Link team performance with individual performance • Beware unintended consequences

6. Types of Metrics • Cost • Effort • Duration • Productivity • Quality of deliverables • Customer satisfaction with the deliverables produced • Project team performance • Business value delivered

7. Types of Metrics Cost • Actual cost vs. budget (variance) for project, phase, activity, etc. • Total support costs for X months after solution is completed • Total labor costs vs. non labor (vs. budget) • Total cost of employees vs. contract vs. consultant (vs. budget) • Material costs (vs. budget) • Cost associated with building components for reuse • Total cost per transaction • Ideas for cost reductions implemented, and cost savings realized

8. Types of Metrics Effort • Actual effort vs. budget (variance) • Amount of project manager time vs. overall effort hours Duration • Actual duration vs. budget (variance)

9. Types of Metrics Productivity - difficult to measure accurately • Function point counts • Effort hours per unit of work/function point • Work units/function points produced per effort hour • Effort hours savings from using standard processes • Effort hours saved through reuse of previous deliverables, models, components, etc. • Number of process improvement ideas implemented • Number of hours/dollars saved from process improvements

10. Types of Metrics Quality of deliverables - direct • Defect rate • Percent within tolerances • Return rate • Scrap • Price premium • Repeat buyers

11. Types of Metrics Quality of deliverables - indirect • Percentage of deliverables going through quality reviews • Percentage of deliverable reviews resulting in acceptance the first time • Number of defects discovered after initial acceptance • Percentage of deliverables that comply 100% with organization standards • Percentage of deliverables that comply with organization architectural standards • Number of customer change requests to revise scope • Number of hours of rework to previously completed deliverables • Number of best practices identified and applied on the project • Number of risks that were successfully mitigated

12. Types of Metrics • Customer satisfaction with deliverables • Survey - overall customer satisfaction with deliverables in terms of: • Reliability • Minimal defects • Usability • Response time • Ease of use • Availability • Intuitiveness • Security • Meets customer needs • Flexibility • Easy to understand messages • User documentation • Application response time (calculated by the system) • Number of approved business requirements completed by the project

13. Types of Metrics • Project team performance • Overall customer satisfaction with the project team in terms of: (survey) • Responsiveness • Competence • Accessible • Courteous • Good communication • Knowledge of the customer • Reliable / follows through on commitments • Professionalism • Training provided • Overall customer satisfaction • Response time to customer queries and problems • Average time required to resolve problems • Would customer do business with you again (if theyhad a choice)?

14. Manage Metrics Business value delivered • Look at original value proposition / business case • Measure the results versus expectations • May have to measure over time • Measurement is responsibility of organization proposing the value received

15. Manage Metrics Difficulty in measuring business value • Incremental improvements • Soft benefits • Value spread across many organizations (infrastructure projects) • Value of increased information • Value of making better decisions • Value of making decisions more quickly • Small time savings will be absorbed by other work

16. Creating Balanced Scorecard • Identify criteria for success • Internal – project management and execution • External – project objectives / deliverables • Assign potential metrics • Hard • Soft (survey)

17. Creating Balanced Scorecard • Look for a balance • Cost • Effort • Duration • Productivity • Quality of deliverables • Customer satisfaction with the deliverables produced • Project team performance • Business value delivered

18. Creating Balanced Scorecard • Prioritize the balanced list of metrics • Least cost to collect / most value to the project • Set targets • Some are obvious • Some targets are set after an initial baseline • Some targets are set as improvements from initial baseline

19. Creating Balanced Scorecard • Add workplan detail • What specific data is needed for the metrics? • Who is responsible for collecting the metric? • When will the metric be collected and reported? • How will the metrics be reported (status reports, quarterly meetings, metrics reports)?

20. Manage Metrics Major Deliverable • Balanced Scorecard

21. Case Study #12 – Metrics Management Review the entire case study • Review the entire case study (especially the prior exercise on quality) • Use the process described earlier to create a balanced set of five metrics that will indicate overall project success • Validate whether the cost of collection seems reasonable

22. Manage the Project • Manage workplan • Manage issues • Manage scope • Manage communication • Manage risk • Manage documentation • Manage quality • Manage metrics

23. Learning Objectives By the end of class, each participant should be able to: • Understand the value of project management processes • Review the various aspects of defining a project • Build and maintain an appropriate project workplan • Identify and manage issues, scope and communication • Identify and manage project risks • Determine methods to manage project documentation • Identify and manage to the appropriate level of quality • Identify metrics to improve processes and declare success

24. Questions??

25. Post-Class Assessment

26. Final Questions??