Upcoming Changes in Earned Value Management

1. Upcoming Changes in Earned Value Management Materiel Policy & Services Branch October 2003

2. Comprehensive Change • Why Change? • Application • EVM Requirements & CSSR • Moving to the IBR • Validations & Surveillance • Reporting & Payment • Integrating measures • Products & Stakeholders

3. What’s in it for you? • Awareness of upcoming change • Opportunity to question • Opportunity to comment • Input will be considered in the development Chance to have your say !!!!!

4. Incorporate Lessons Learned Improve useability & Increase relevance Risk-based approach Need for Change • Complexity & “experts domain” • Benefits not being realised • Analysis and action missing • “Financial” stigma

5. Now Future Flexibility Lessons Learned Requirements 3 DEF(AUST) standards specify EVM requirements AS4817 + Defence Supplement CSCSC CSSR EVM Exclusion with advice Application $ Thresholds: > $200m All types > $60m Develop’t > $20m CSSR Assess risk Nominally > $20m Assess known & unknown Implementation * No Validations * Risk-based review requirements Lengthy Validation process “Meets contractual reqts” Assurance

6. Application • Policy thresholds for application • > $200m for Production = EVM • > $60m for Development = EVM • > $20m for CSSR • Project Risk Assessment • Where is the risk? • What is the risk type? • What management systems exist? • What are my information needs?

7. How? • Internal risk mgt workshops • Pre-contract, in line with SAMS processes • Measurement workshops • Particularly for software-intensive projects • Nominal minimum level for application • Contracts >$20m must apply EVM • Still assess risk for <$20m to ensure EVM not necessary • $ value is not the driver! Risk is the driver. • Exemptions for low risk projects regardless of $ value

8. EVM Requirements • Remove all DEF(AUST) standards • Adopt AS4817-2003 • Australian Standards • Various industries involved, wide consultation • Pragmatic, simple approach • Specific Defence reqts • Supplement to AS4817 • New Tendering & Contractual requirements • New Review Guide

9. CSSR • CSSR has been eliminated • Not successful • Industry feedback showed no differences in implementation • Original aim not achieved • Industry support for removing CSSR due to better consistency

10. Integrated Baseline Reviews • Offer definition role • Emphasis is on project responsibility • Technical review by technical staff • Target project risk areas • > emphasis on measurement • Opportunity to review payment schedules after IBR will be built into contracting templates

11. EVMS Validated And now Validations…. • Previously issued “Validation” of Contractor’s EVMS • Not meeting needs • Providing false confidence • Not assuring ongoing validity • Future • Will no longer issue validations • Assess each contract on a risk basis • previous experience, previous performance/ issues • > deemed risk, > level of review

12. More on validations • Onus upon Contractor to reduce risk of their EVMS • Proactive • Self-correcting systems through management use • Valid, meaningful data = Sound reporting on projects • May seek certification from independent 3rd party • Outcome = EVMS “meets reqts of contract” • i.e. meets requirements in contractually specified standards & Supplement • is therefore “acceptable” but only for this project

13. The impact • Project by project assessment • Contractors with sound, existing systems not adversely affected • Provide detail to support their system • Take greater ownership of system & actively reduce risk • Projects can take into account successes of other projects • Encouraging projects to seek info from past projects • Number of EVMS reviews will be proportionate to assessed risk • e.g. an existing system deemed to have low risk may only require an IBR and Ongoing Surveillance

14. System Surveillance • Used to have “annual” policy • Move to risk-based policy • Assess risk of the EVMS • Assess data integrity and success of implementation • Target risk areas rather than complete EVMS review • “Assurance” rather than surveillance • Seeking assurance that the system is functional and providing reliable data

15. Reporting & Payment • Target risk areas • Reporting Variances • seek high risk items not merely high dollar items • reduce tolerance levels for high risk items • WBS level for reporting at level needed to manage risks • Exception reports • Verify claims for payment • Target risk areas • Sample based on risk • Ensure technical staff review accuracy of claimed progress

16. Integrating Disciplines • Utilising project management disciplines together Risk PM inc. IPSSR EVM PSM

17. It could mean... So I should think about.. Integrating Disciplines • EVM & Practical Software Measures • Project Performance Management Guide If I see this...

18. Current Performance Future Performance Behind Schedule, Under Cost Ahead of Schedule, Under Cost What is the project’s status? Basis of Estimate Functional Volatility Feasibility Behind Schedule & Over Cost Behind Schedule & Under Cost Resource Usage Recovery Ahead of Schedule & Under Cost Basis of Estimate Process Adherence Duration Behind Schedule, Over Cost Ahead of Schedule, Over Cost Ahead of Schedule & Over Cost Basis of Estimate Resource Usage Project Status can be expressed in terms of cost and schedule variances. A variance is a difference between what was planned and what is happening. A useful indicator of project status is the Bull’s Eye Chart shown above. This indicator is derived from earned value measures. The Bull’s Eye Chart plots the cumulative cost variance percentage (%CVcum) against the cumulative schedule variance percentage (%SVcum). This provides a point within one of four quadrants for a point in time. Connecting these points provides a trend line over time. Management action is normally required when either the %CVcum or %SVcum are outside a defined threshold, typically 10%. All excursions beyond the set threshold should be investigated as even being ahead of schedule and under cost is not necessarily a good thing. Note: To ensure the accuracy and credibility of earned value measures, WBS elements should use defined, unambiguous, binary criteria, where possible, to determine each milestone or task completion point and to apportion earned value. This requires an thorough understanding of the work and how it is to be performed. The verification of these criteria should be of prime consideration during Integrated Baseline Reviews. Care should also be applied to ensure the majority of earned value is not given until the adequacy and success of testing has been evaluated.

19. Anticipated Completion April 04 Integrated Training • Analysis training to be developed • WBS, Schedule, Performance EVM PM PSM IPSSR

20. EVM Suite to Support Change Anticipated Completion April 04 • Requirements • AS4817 + DMO Supplement • Revised ASDEFCON clauses & handbook • Revised EVMS Review Handbook • Update review practices • Reflect new standards • Reflect risk-based approach to EVM • FAQ on EVM aspects of tender evaluation & contract negotiation

21. Existing Products • IBR Handbook • Currently available via QEMS • Guide to EV Payments • Currently available via QEMS • Recent study findings support use of EV Payments BUT • IBR is paramount • Trained staff are vital • Summary of findings is available - demonstrates key success criteria • Note: These products may undergo minor updates to reflect new terminology/ standards but the principles will remain unchanged.

22. Conclusion • Wholesale change for the better • Affecting every facet of EVM Goal = Improved use & flexibility

23. Consultation • Stakeholders are invited to register interest with PM-EVM • traci-ann.byrnes@defence.gov.au or (02) 62656928 • Stakeholders will have the opportunity to review and comment upon draft documents