C a p a bility S ys t em s C e n t r e

1. CapabilitySystemsCentre Crafting and Implementing Project Execution Strategies for Complex ProjectsPresented by: John Davies Capability Systems Centre Seminar – 3 April 2019 SchoolofEngineeringandInformationTechnology

2. Background Dealing with complexity is a key challenge with system acquisition and sustainment becoming more interdependent with other capabilities (existing and emerging), rapid technological advancement, and greater emergence associated with end-user needs. This seminar explores the dimensions of complexity and offers guidance on how appropriate project execution strategies can best deal with this complexity.

3. Seminar Outline 0900-1200 • Introduction (complexity and the failure of business as usual approaches) • Risk Management • Selecting and Implementing Project Execution Strategies 1300-1600 • Governance • Risk Management in practice • Project Execution Strategies in Depth

4. What is Complexity ‘Complex’ programs are not necessarily ‘complicated’. The United Kingdom National Audit Office (UK NAO) defines complexity as follows: • at the outset there is uncertainty over the route to delivering the project outcome, or the project has aspects that have not previously been encountered; or • there is a high level of change in the outcome required during the project’s lifetime Keep in mind that ‘change’ could be simply a symptom of sloppy project management and systems engineering practices.

5. Dimensions of Complexity Remington and Pollack

6. Assessment of Complexity Treating programs as a binary construct of either ‘complex’ or ‘simple’ is a dangerous proposition. Some elements of a program may be simple whilst others can be quite complex. Similarly, a project can start out complex and become simple (or vice versa).

7. Typical means of defining complexity • The Kraljic matrix • Acquisition Category (ACAT) • ICCPM (Complexity Assessment Tool) • Helmsman (Project Complexity Framework) • Canadian Government http://www.tbs-sct.gc.ca/hgw-cgf/oversight-surveillance/itpm-itgp/pm-gp/doc/pcra-ecrp-eng.asp

8. Helmsman complexity assessment -A ‘Comparison of Project Complexity between Defence and other Sectors’

9. CASG Risk and Complexity • Smart Buyer Framework • Core Acquisition and Core Sustainment Risks Drive the Project Execution Strategy • No hard rules (unlike the previous Acquisition Category (ACAT))

10. Smart Buyer – Detailed Design (2016)

11. Problems with Complexity Assessment • Uncertainty • Epistemic • Ontological “We don’t know what we don’t know”

12. Further Problems with Complexity Assessments • Temporal dimensions • Risks emerge and retire throughout the acquisition and sustainment life cycle • Structural issues • Applying a single complexity ‘grade’ to the whole capability lifecycle and all sub-systems is often inappropriate

13. Systems and Processes • Many organisations seek to classify procurement into discrete categories. • Is this consistent with a ‘smart buyer’? • Do the cost savings associated with templates, training and procedures outweigh the loss of flexibility and tailored approaches?

14. The Coastline Paradox Refer Andrew Pyke Presentation PCSG 2016

15. Arnold Rimmer’s Study Plan “The first week of study, he would always devote to the construction of a revision timetable. Weeks of patient effort would be spent planning, designing and creating a revision schedule which, when finished, were minor works of art. Every hour of every day was subdivided into different study periods, each labelled in his lovely, tiny copperplate hand; then painted over in watercolours, a different colour for each subject, the colours gradually becoming bolder and more urgent shades as the exam time approached. The effect was as if a myriad tiny rainbows had splintered and sprinkled across the poster-sized sheet of creamwove card. The only problem was this: because the timetables often took seven or eight weeks, and sometimes more, to complete, by the time Rimmer had finished them the exam was almost on him. He'd then have to cram three months of astronavigation revision into a single week. Gripped by an almost deranging panic, he'd then decide to sacrifice the first two days of that final week to the making of another timetable. This time for someone who had to pack three months of revision into five days.” Grant Naylor, Red Dwarf – Infinity Welcomes Careful Drivers (1989)

16. Goldilocks Planning? • Not too much, not too little! • Ask yourself: what is the plan for? • To Seek funding and create budgets • Identify resources • Achieve gate review milestones • Demonstrate confidence to stakeholders

17. Second Order Learning “If the only change that can be contemplated takes place in the context of an existing mental model, then you are limited to bringing about first-order learning. If, however, the mental model itself can be changed, and purposes radically altered, then second-order change is possible.” - M. Jackson, “Systems Thinking” (2003) p10.

18. Key Challenges • Striking the balance between consistency and responsiveness • Resources • Repeatability • Centralised, Decentralised and Centre-led approaches • Risk

19. Business as usual? • Linearity (water fall development life cycle) • Too much planning (reductionist approaches) • Adoption of worst practice?

20. Key Success factors for Complex Projects ICCPM ‘Contracting for Success in Complex Projects’ (2015) • Clearly defined project goals and vision, • Appropriate Relationship/behavioural management, • Prudent risk management and equitable risk allocation, • An acquisition and sustainment strategy suited to the project at hand, • Leadership and competencies of the team, and • A robust project management and systems engineering framework (hygiene factors)

21. Traditional Approach for Goal Alignment The customer defines outcomes in precise terms (statements of requirement and statements of work) with a robust contractual framework that precisely allocates liability. Industry Defence

22. Linearity Gate 0 –> Gate 1 –> Gate 2 The Capability Lifecycle Manual now contemplates skipping Gate 1 for non-complex projects and direct source options and having multiple Gate 1 reviews for the most complex projects.

23. Gate Reviews “Gate 2 - Assure decision makers the proposal to commit public money is sound and should proceed with the acquisition”. - The Capability Life Cycle Awareness Presentation (2016) What does this mean? (acceptable risk, requirements stability?)

24. Post Gate 2 “In the event contractual terms within that envelope cannot be agreed, the Project will return to the Investment Committee with a recommendation on whether to terminate the Project or expand the performance/time/cost envelope.” Capability Life Cycle Manual (2016)

25. Traditional Approach to Managing Relationships • A focus on liability assignment • liquidated damages (sticks rather than carrots) • termination for convenience • no waiver clauses • onerous contract change proposal/variation processes • Some attempts for Alternate Dispute Resolution (ADR)

26. Traditional Approach for Risk Allocation Abrahamson’s Principles of transferring the risk to the party best able to manage the risk if often ignored. Consider the following: • The principal assumes no responsibility for the reliability of the tender documentation, • The tenderer must acquaint itself with all information, and • Information is provided for the convenience of tenders only and this information is not guaranteed. J. Feehley ‘Tendering and Contract Documentation’ (1998)

27. The Illusion of Risk Transfer • Contracts cannot legislate for every permutation of risk • Rarely will parties have ‘clean hands’ • Risk take back is often the norm in complex projects

28. Contracts – not a panacea • The role of the lawyer is to legislate for risk in accordance with management’s instructions. • Problems • GFE/GFI/GFD/GFF – baseline risks • Interdependencies with other projects or contracts • Uncertainty • Variations (there is rarely price competition with variations) • Interpretation (fitness for purpose) • Insurance • Implied terms

29. Clean Hands and Risk Take Back • IPT participation • Risks outside of the contractor’s control • Actions and behaviours modify the contract • Poor Contract Configuration Management “A revised schedule is to business what a new season is to an athlete or a new canvas to an artist.” N. Augustine ‘Augustine's Laws’ (1986)

30. Inappropriate Project Execution Strategies One size will rarely fit all yet organisations love ‘boilerplate’ templates! Failure to vary the remuneration strategy as a function of lifecycle and risk.

31. Lack of Leadership Leadership is a key component for supporting collaborative behaviours as observed by the UK NAO: “Every case study ranked leadership as the most important factor in developing collaborative relationships.” UK NAO Good Governance ‘Measuring Success Through Collaborative Working Relationships’ (2006) Leadership

32. Worst Practice Leadership "Managing the NHS IT suppliers is like running a team of huskies. When one of the dogs goes lame, it is shot. It is then chopped up and fed to the other dogs. The survivors work harder, not only because they have had a meal, but also because they have seen what will happen should they themselves go lame." UK Parliament, Committee of Public Accounts (26 June 2006)

33. Don’t abandon the Hygiene factors A final success factor for complex projects is to ensure all the procurement ‘hygiene’ factors are supported. Adopting all the other attributes will be meaningless if we do not have competencies and capability in: • Project Management • Configuration Management • Quality Assurance • Systems Engineering • Commercial and Contracting Disciplines • Logistics

34. Risk Management and Complex Projects • The project execution strategy is crafted to deal with known risks. • Risks are typically high in complex projects • We also need to deal with uncertainty and flexibility

35. Complex Project Risk management Challenges • Very long implementation timeframes • Multiple Parties • Diverse stakeholders • Difficulty in assigning responsibility • First of type

36. Industry Risks • Cash Flow • Demand/usage • Financing • Indices/fluctuations • Risk adjusted rates of return • Solvency • Insurance

37. Contract Risks • Jurisdictional • Legislation change (Tax, R&D credits, Industrial Relations, WH&S, Contract Administrative Law) • Proportionate liability

38. What are the processes for identifying risk • Customer’s Drivers, Risks, Assumptions and Issues Log • Liability Risk Assessments • Craft Project Execution Strategy to manage these known risks • What are the problems with this approach?

39. How do we traditionally deal with risk • Accept the risk • Treat the risk • Avoid the risk • Transfer the risk (or partial transfer with risk sharing)

40. How does a contract deal with risk? • Remuneration Strategy • Warranties • Limitation of liability • Indemnities • Liquidated damages • Consequential damages • Breach/remedies

41. Problems with risk management approaches in complex projects • Clinical risk transfer is often not possible • Multi-party involvement exacerbates problems with allocation of risks • Failure to anticipate uncertainty • Failure to consider temporal dimensions of risk (see example)

42. Risk Time

43. Complex Project Risk Management Best Practice • Early Industry Involvement • Offer Definition and Improvement Activities • Risk Sharing where appropriate • Joint Decision Making • Joint Risk management • Agile/adaptive contracts

44. Early Industry Involvement • The quality of risk identification and classification will improve with early industry involvement • Probity Issues need to be carefully managed • The use of Early Contractor Involvement Strategies improve the quality of the risk management function.

45. Risk Sharing • Parties are more likely to be proactive in risk management if they have ‘skin in the game’. • Options for risk sharing include • Remuneration (target cost, gainshare/painshare) • Achievement of Key Performance Indices • Shared costs (e.g. insurance)

46. Joint Decision Making • Joint decision making does not mean that the customer’s decision making ability is fettered, rather joint decision making allows for: • Collaborative approaches to dealing with change • Support for agility and flexibility • Dealing with emergence

47. Joint Risk Management • Complex projects often fail because they do not take a holistic view of risk. • Joint risk management may involve joint risk logs to avoid duplication and errors of omission.

48. Agile/Flexible Contracts • No contract can anticipate every outcome. • Agile contracts allow for off-ramps, rescoping, and re-negotiation • We need to understand industry’s need for certainty and ensure arrangements are both fair and contestable.

49. No Such Thing as a Free Lunch • Risk Sharing will attract new risks in itself • Costs associated with bespoke contracts • Behavioural alignment • Trust!

50. Risk Management Pitfalls • Optimism Bias • Deviant Behaviours • Governance and Legislation