The Death of a Project: Can lessons from yesterday’s catastrophe prevent tomorrow’s disaster?

1. The Death of a Project: Can lessons from yesterday’s catastrophe prevent tomorrow’s disaster? Matthew Gillery Project Manager Hewlett-Packard

2. Agenda • Overview/Purpose • What is a project? • What is project management? • Introducing Risk Management • Defining Project Success • Common Causes of Project Failure • Role of Risk Management in Reducing Failure Rates • Risk Classification: ABCD Methodology

3. Overview/Purpose • Problem Statement • The purpose of this presentation is to identify the common causes of project failure, as suggested by members of the academy and industry leaders. • Clearly, there is no “one size fits all” approach/solution to preventing project failure. However, the presentation focuses on how risk management can be used to reduce the likelihood/impact of project failure. • Key question: Are we learning from yesterday’s mistakes?

4. Project Failure in the 70’s • In 1979, IEEE published an analysis on failed “Bankrupt” projects and reported: • “Bankrupt” projects are those that consistently missed target dates and resulted in cost over-runs, or cancellations • IEEE argued: • By the time the paper was written, the project “bankruptcy” phenomena hadn’t been analyzed successfully • Occurrence of “bankruptcies” are rarely detected early in the development stage. Most are identified during testing. • Effective methods for preventing “bankruptcies” are still being developed An Analysis of Software Project Failure. IEEE, 1979.

5. Standish CHAOS Report 2009 Is Project Success Really that Rare? • Specifically, 32 percent of IT projects were considered successful, having been completed on time, on budget and with the required features and functions. • Nearly one-in-four (24 percent) IT projects were considered failures, having been cancelled before they were completed, or having been delivered but never used. • The rest (44 percent) were considered challenged: They were finished late, over budget, or with fewer than the required features and functions. http://www.cio.com/article/495306/Recession_Causes_Rising_IT_Project_Failure_Rates_

6. The Project Management Problem The Standish Group has reported: • IT project success rates rose steadily from 1994 until 2000, when they dipped, and then began rising again from 2002 through 2006. Success rates decreased between 2006 and 2009. • Many projects experience governance issues: • Project take so long to complete that stakeholders lose interest and eventually decide to cancel it, or a project that eventually gets delivered but doesn't get used because it's no longer relevant to the business. In both situations, the project is considered a failure. http://www.cio.com/article/495306/Recession_Causes_Rising_IT_Project_Failure_Rates_

7. What is a project? • Temporary endeavor with a beginning and an end. • The end is reached when the objectives have (or cannot) be achieved. • Temporary DOES NOT mean short in duration; many projects last for several years but they do, eventually end. • Creates a unique product, service or result. • It’s unique in that it has not been attempted before by the organization. • Creates a product or artifact, is quantifiable and can be either and end item itself or a component of another item. • Performs a service such as business functions that support shipping and receiving. • Delivers a result such as outcomes or documents like results from a research project on drinking water conditions in mountainous regions of Ecuador and Chile. • Is progressively elaborated – distinguishing characteristics of each unique project will be progressively detailed as the project is better understood (matures). • It is comprised of interrelated activities. Crowe. The PMP Exam. p9. Mulcahy, PMP Exam Prep, p22 PMBOK p5-7.

8. What is Project Management? • Project management is the application of knowledge, skills, tools and techniques to project activities to meet project requirements. • Project management is accomplished through the application and integration of the project management processes of initiating, planning, executing, monitoring and controlling, and closing. • The project manager is the person responsible for accomplishing the project objectives. • Managing a project includes: • Identifying requirements • Establishing clear and achievable objectives • Balancing the competing demands for quality, scope, time and cost • Adapting the specifications, plans, and approach to the different concerns and expectations of the various stakeholders. PMBOK p8.

9. Progressive Elaboration & Risk Management • The term “progressive elaboration” simply means that you do not know all of the characteristics about a product when you begin the project. • Instead, they may be revisited often and refined. • The characteristics of the product emerge over time, or “progressively.” • Moving forward on a project without a proactive focus on risk management increases the impact that a realized risk can have on the project and can potentially lead to project failure. High Staffing, Resources & Project Costs Stakeholder Influence Risk to Project Success Low Early PHASES Late Conceptual Planning Construct Testing Implement Closure Crowe. The PMP Exam. p20. PMBOK p6 & 8. Page 9

10. Defining the term “project” “A project is a temporary endeavor undertaken to create a unique product, service or result.” In answer to the question, WHAT IS A PROJECT? Temporary, Unique. Crowe. The PMP Exam. p9. Mulcahy, PMP Exam Prep, p22 PMBOK p5-7.

11. Risk Management What is Risk Management? • Risk is an uncertain event or condition, that if it occurs, may have a positive or negative effect on project objectives. Project risk is always in the future. • Risk management is the systematic process of planning how to manage, identify, analyze, respond to, monitor and control project risks. • The objectives of Project Risk Management are to increase the probability and impact of positive events and decrease the probability and impact of negative events on project objectives. • A cause may be a requirement, an assumption, constraint or condition that creates the possibility of negative or positive outcomes. • Positive risks = Opportunities • Negative risks = Threats • Known Risks = those that have been identified, analyzed making it possible to plan responses for those risks • Unknown Risks = cannot be managed proactive, so need a contingency plan • Risk Tolerance = varying degrees of risk that organizations and stakeholders are willing to accept • NOTE:A project risk that has occurredcan also be considered an issue

12. Project Risk • Project risk is always in the future. Risk is an uncertain event or condition that, if it occurs, has an effect on at least one project objective. • Cost • Time • Scope • Quality • A cause may be a requirement, assumption, constraint, or condition that creates the possibility of negative or positive outcomes.

13. Risk Management Objectives • The objectives of Project Risk Management are to: • increase the probability and impact of positive events • decrease the probability and impact of negative events in the project.

14. Risk Example What risk(s) is/are evident in this example? University of Toronto http://www.cdf.toronto.edu/~csc340h/summer/lectures/w6/L6-part2-risk.pdf

15. Risk Semantics • Risks are typically expressed in the form of conditional statements • If A occurs, then B….. • Technology company example: • If project resources are not trained in the application of technology OO prior to project start, then the project may be delayed and experience cost over-runs as a result of on-the-job training

16. Risk Management Problem There are several reasons why risk management is sometimes not undertaken including: • There is an unwillingness to admit risks exists. • There is a lack of understanding of the benefits. • There is a natural tendency to postpone the hard parts of a project. (i.e., Do the easy things first.) • Some believe that it costs time up front without adding value overall. • It is difficult to prove that it’s necessary (e.g., like insurance).

17. Risk Management • Risk Management includes the processes of conducting: • Risk management planning • Identification • Analysis • Response Planning • Monitoring and Control

18. Key Project Success Considerations • How do we define project success? • How do we get the project team engaged in the risk management process?

19. What is success? • Answering “yes” to the following questions: • Did the project meet its objectives, requirements, budget and schedule? • Did the project meet the customer’s needs? • Was the decision to proceed with the project correct at the time? • Would we do the project again, knowing what we know now? • A project is a success if it meets its objectives, requirements, budget and schedule, and a failure otherwise Society for Automotive Engineers International Conference on Environment Systems: Explaining Space Project Failures

20. A perspective on “Good” vs. “Bad” Projects • A project that takes reasonable risks and fails by bad luck is not a bad project, since rational decision makes would repeat the attempt • A project that takes an unreasonable risk and yet meets goals by chance is, conversely, not a good project • Key question: What is a “failed” project? Society for Automotive Engineers International Conference on Environment Systems: Explaining Space Project Failures

21. Failed Project Defined • A failed project does not successfully meet requirements/objectives

22. Common Causes of Project Failure Study conducted by KPMG found several risk factors that lead to project failure. While focused on the IT industry, these risk factors are general. Project managers were surveyed. Responses were grouped, based on the experience level of the project manager. Controlling Software Project Risks- an Empirical Study of Methods used by Experienced Project Managers. Addison & Vallabh. SAICSIT 2002, pp. 128-140 Anatomy of a Failed Project. 2008. http://blogs.zdnet.com/projectfailures/?p=1100

23. FBI Trilogy Project • Goal • Upgrade FBI’s technology infrastructure to speed up data entry and analysis • Scope • Deployment of: • An enterprise-wide upgrade if desktop hardware and software • Modern network infrastructure • An integrated suite of software for entering, finding, sharing, and analyzing case information ($170M/$581M Trilogy project cost) • History • In 2000, started as the UAC project to modernize the FBI’s Automated Case System (ACS) and technological infrastructure, by “webifying” the system’s green screens • Problem: Updating the system’s screens didn’t fix the underlying business process issues and architecture • System was built on 1970’s standards and equipment from the late 1980’s • In 2001, the FBI contracted with IT service providers (DynCorp and SAIC) to upgrade the FBI’s after congress approved $379.8 million for the FBI Technology Upgrade Project (FITUP) http://www.infoworld.com/d/developer-world/anatomy-it-disaster-how-fbi-blew-it-243

24. FBI Trilogy Project • In 2001, scope changed form beautifying mainframe screens to replacing them with an web application for gathering and analyzing intelligence data. Created the Virtual Case File Project (VCF) • Requirements were difficult to “nail down,” since organizational processes were constantly changing as a result of 9/11. • Contractors were rated based on customer satisfaction, and continuously accepted new requirements • At least 30-31 changes were submitted each month • Endless change resulted in communications issues (misunderstandings) • In December 2003, SAIC deployed an incomplete system. The FBI was expecting the full version. • Leadership at all levels was surprised. The FBI had undergone two CIO changes. • In 2004, FBI and Homeland Security began planning an inter-agency Federal Investigative Case Management System (FICMS), which would make VCF obsolete • network infrastructure and desktop hardware/software upgrades were completed. • In 2005, the project was officially cancelled. The software development project was not making progress, which continuously resulted in cost and schedule over-runs http://www.infoworld.com/d/developer-world/anatomy-it-disaster-how-fbi-blew-it-243

25. FBI Trilogy Project Issues Issues • Lack of internal IT expertise • Management and IT support • Scope Creep/Poor Change Management • Key change in direction after one year: web application instead of upgraded screens • Leadership Changes • Inconsistent IT leadership: FBI experiencd 5 CIO changes between 2000 and 2005; each had their own vision for the project http://www.infoworld.com/d/developer-world/anatomy-it-disaster-how-fbi-blew-it-243

26. Trilogy Replaced by Sentinel • As of 2006, a new contract was awarded to Lockheed Martin. The Trilogy Project was renamed to become the “Sentinel Project”. • At this point, the project was projected to be complete by May 2010, with an estimated cost of $425 million. • This means that over $500 million would be spent on a project that was estimated at $170 million in 2001. http://www.govexec.com/dailyfed/0708/071608rb1.htm

27. Sentinel Status: November 2009 Following is in response to the Department of Justice’s Office of the Inspector General (OIG) report, “Sentinel Audit V: Status of the FBI’s Case Management System.” • “The FBI appreciates the Inspector General’s review of the FBI’s Sentinel Program progress and recognition of the FBI’s efforts to resolve concerns identified in previous Sentinel audits. As the OIG notes, the FBI estimates that Sentinel is scheduled to deliver full capability within the $451 million budget in the fall of 2010. The FBI already has implemented measures to resolve all six recommendations identified in this report and has successfully closed 30 of the 31 recommendations from the four prior OIG Sentinel audits. • “The Sentinel program has steadily improved and refined its business practices. During Phase 1 of this project, the FBI and its primary contractor, Lockheed Martin, chose to redesign and re-baseline Phase 2 to be more efficient and deliver additional capability to users earlier than originally planned. This approach reduces overall program risk. It was carefully planned and incorporated a flexible aspect to the design to further mitigate risk by shifting requirements forward in the program’s development to meet user needs. • “The FBI is pleased the report concluded the revised Sentinel schedule is more realistic. By extending the completion of Phase 2 by three months, requirements from the latter phases will be delivered earlier, providing capabilities to users sooner than originally planned. • “The FBI has begun user testing of Sentinel’s Administrative Case Management system, including three electronic forms and automated workflows. Following a brief pilot in the FBI’s Richmond, Tampa, and Chicago offices, the FBI will deliver these capabilities across the FBI, marking the completion of Phase 2.” http://www.fbi.gov/pressrel/pressrel09/oigsentinel111009.htm

28. Sentinel: Current Status • As of April 1, 2010: • The Associated Press reports that the FBI's $451 million Sentinel case management system is getting slower and costing more, according to a Justice Department audit. • The project was expected to be complete by September of this year, but FBI Director Robert Mueller said it would be delayed until 2011. The bureau is renegotiating the budget with Lockheed Martin, as well as the schedule and some of the work to be performed. • As of April 16, 2010: • FBI director reported, " When you have a project that was laid down in concrete four or five years ago, [with] technology changes, business practice changes, and complexity changes, one can expect some minor delays"– FBI Director http://www.itbusinessedge.com/cm/community/news/gt/blog/audit-fbi-sentinel-system-facing-more-costs-delays/?cs=40450 http://www.informationweek.com/news/government/enterprise-apps/showArticle.jhtml?articleID=224400547

29. NASA Failed Missions (Projects) • NASA has committed itself to organizational and project management improvement, by analyzing over-looked risk factors on the following space programs: Society for Automotive Engineers International Conference on Environment Systems: Explaining Space Project Failures

30. NASA Project/Organization Problems • Engineering quality/risk management error • “None of us gave any serious consideration to a fire in the spacecraft” • Astronaut Frank Borman, Apollo 1 Review Board • Lack of cross-functional coordination/informal work-arounds • Process changes were made at the work group level, and not considered during projects (Change Management/Communication) • Failed to ask basic questions • Example: Challenger. Per NASA study, “No launch incident escape system was provided because of the incorrect assumption that the shuttle had high reliability” (SAE) • Management failed to ask the question: What happens if the flight crew needs to escape during a launch? • Stated as a risk: If the flight is not able to escape during a launch, then….. Society for Automotive Engineers International Conference on Environment Systems: Explaining Space Project Failures

31. NASA: Recommended Remedies Risk Management as remedy for NASA’s problems Society for Automotive Engineers International Conference on Environment Systems: Explaining Space Project Failures

32. Risk Management What is Risk Management? • Risk is an uncertain event or condition, that if it occurs, may have a positive or negative effect on project objectives. Project risk is always in the future. • Risk management is the systematic process of planning how to manage, identify, analyze, respond to, monitor and control project risks. • The objectives of Project Risk Management are to increase the probability and impact of positive events and decrease the probability and impact of negative events on project objectives. • A cause may be a requirement, an assumption, constraint or condition that creates the possibility of negative or positive outcomes. • Positive risks = Opportunities • Negative risks = Threats • Known Risks = those that have been identified, analyzed making it possible to plan responses for those risks • Unknown Risks = cannot be managed proactive, so need a contingency plan • Risk Tolerance = varying degrees of risk that organizations and stakeholders are willing to accept • NOTE:A project risk that has occurredcan also be considered an issue

33. Role of Risk Management in Reducing Project Failure Rates • Moving forward on a project without a proactive focus on risk management increases the impact that a realized risk can have on the project and can potentially lead to project failure. • The intent of risk management practices is to promote a proactive stance towards reducing project failure

34. Risk Classification: ABCD • Risk management must be a team effort! • “Research suggests that a successful project environment may be characterized as on in which all systems professionals maintain a holistic view of organizational risk….” • Merrill Warkentin, Mississippi State University • The challenge for project managers is to get the entire project team actively engaged in risk management

35. Scope of Risk Management Complete Information No Information Partial Information (Unknown unknowns) (Known unknowns) (Knowns) TOTAL CERTAINTY GENERAL UNCERTAINTY SPECIFIC UNCERTAINTY TOTAL UNCERTAINTY SCOPE OF PROJECT RISK MANAGEMENT

36. Risk Management Processes 11.2 Identify Risks (P) 11.1 Plan Risk Management (P) 11.3 Perform Qualitative Risk Analysis (P) Major Inputs Tools & Techniques Major Outputs Major Inputs Tools & Techniques Major Outputs Major Inputs Tools & Techniques Major Outputs Project Scope Statement Cost Mgmt, Schedule Mgmt Communications Mgmt Plans Enterprise Environmental Factors Organizational Process Assets Planning Meetings & Analysis Risk Management Plan Risk Register Risk Management Plan Project Scope Statement Organizational Process Assets Risk Probability and Impact Assessment Probability and Impact Matrix Risk Data Quality Assessment Risk Categorization Risk Urgency Assessment Expert Judgment Risk Register Updates Risk Management Plan Cost Mgmt, Schedule Mgmt, Quality Mgmt Plans Scope Baseline Activity cost estimates Activity duration estimates Enterprise Environmental Factors Organizational Process Assets Documentation Reviews Info Gathering Techniques Checklist Analysis Assumption Analysis Expert Judgment SWOT Analysis Diagramming Techniques Risk Register

37. Risk Management Processes 11.5 Plan Risk Responses (P) 11.4 Perform Quantitative Risk Analysis (P) 11.6 Monitoring and Control Risks (M&C) Major Inputs Tools & Techniques Major Outputs Major Inputs Major Inputs Tools & Techniques Tools & Techniques Major Outputs Major Outputs Risk Register Project Management Plan Work Performance Information Performance Reports Risk Reassessment Risk Audits Variance & Trend Analysis Technical Performance Measurement Reserve Analysis Status Meetings Risk Register Updates Organizational Process Asset Updates Change Requests Project Management PlanUpdates Project Documentation Updates Risk Register Risk Management Plan Risk Register Risk Management Plan Schedule Mgmt Plan Cost Mgmt Plan Organizational Process Assets Data Gathering and representation techniques Quantitative Risk Analysis and Modeling Techniques Expert Judgment Risk Register Updates Risk Register Updates Risk-related Contract Decisions Project Management Plan Updates Project Document Updates Strategies for Negative Risks (Threats) Strategies for Pos Risks (Opportunities) Contingent Response Strategies Expert Judgment

38. Risk Breakdown Structure • The RBS is a decomposition of the risk categorization and the risks within those categories that could occur on a project.

39. Classifying Risks • Business risks vs. pure (insurable) risks • Classified by uncertainty (business risks) • Classified by impact on project elements • Classified by their nature • Classified by their source • Classified by their probability to occur and amount at stake

40. Risk Register & ABCD Rating [1] In larger projects, the consequences of the threat may not be evident, and noting them under each risk, or in a separate column can be useful in identifying appropriate mitigation actions. [2] WBS = Work Breakdown Structure, this is to indicate that the identified mitigation action has been included in the WBS (workplan). Source: PM 007 Project Risk Register Template and Guide: www.egovernment.tas.gov.au/__data/assets/word_doc/18512/pman-temp-open-risk-register.doc

41. ABCD Risk Management Overview http://www.de-risk.com/page.php?7

42. ABCD Framework Sensitivity Stability How sensitive to the project is the assumption? How stable is the assumption? Not sensitive / minimal impact Very stable / confident = = A Not very sensitive / manageable impact Fairly stable / confident = = B Fairly sensitive / significant impact Fairly unstable / uncomfortable = = C Very sensitive/ critical impact Very unstable / not confident = = D http://www.de-risk.com/page.php?7

43. Assumption Analysis http://www.de-risk.com/page.php?7

44. Benefits of ABCD • The key features and benefits of ABCD are: • Communication - Provides a simple, common, language for the communication of risk up, down and sideways within the organization, while avoiding the normal problems of political sensitivity and dislike of discussing risks. • Control - Enhances project control by an exception management approach and provides a simple overview of complex risks for senior management to prompt decision making • Flexibility - An adaptable process which, once tailored, is applied to ensure that all significant risks to the projects are identified and controlled at the appropriate time. • Acceptable - The non-intrusive/non-bureaucratic management process improves management discipline across the organization and is readily accepted by project teams http://www.de-risk.com/page.php?7

45. Discussion Question • In your project management career, what has been the most challenging element in managing project risk? • How do you intend to leverage risk management practices, as you manage projects now and in the future?

46. Recap • Recent reports have shown that project failure rates are rising • Risk management practices have the potential to reduce failure rates • Research has found that there are commonalities between projects that have failed • Risk management requires leadership and proactive thinking

47. Risk Management Requires Leadership • Final thought from John C. Maxwell’s “The 360 Degree Leader: Developing Your Influence from Anywhere in the Organization” • Do More than Manage—Lead!

48. Thank You!