Version 3.0 (Draft) Last Updated: 10 November 2011

1. AMERICA’S ARMY: THE STRENGTH OF THE NATION Cost-Benefit Analysis (CBA) Four-Day Training Briefing Step 7: Compare Alternatives Step 8: Report Results and Recommendations Visit our CBA Website for more information regarding locations, signing up, upcoming training sessions, and more https://cpp.army.mil Version 3.0 (Draft) Last Updated: 10 November 2011

2. Step 7: Compare Alternatives 7a. Compare alternatives using the selection criteria to identify the preferred alternative. 7b. Identify the billpayer if there is a bill associated with the recommended alternative. 7c. Identify the positive and negative impacts of the 2nd and 3rd order effects. What must be done to manage the negative impacts? 7d. Determine the robustness of the conclusions. • If anything changes (i.e., assumptions, costs, benefits) would the recommendation change? 7e. Identify the high-risk aspects of the recommended alternative and define appropriate risk mitigation measures.

3. Step 7: Compare Alternatives Compare alternatives by: Concepts Comparisons of costs with benefits Decision support tools and methods (bringing the CBA together) Sensitivity analysis and Risk assessment

4. Concepts for Comparing Costs with Benefits • The CBA process is essentially comparing the costs with the benefits between all alternatives • The preferred alternative provides the greatest amount of value • Risk and second- and third-order effects should be included in the comparison of alternatives Value = Benefit – Costs

5. Aid for Completing - Step 7a Most likely CBA Scenario CBA may not be appropriate

6. Overly Complex Final Recommendation

7. Comparative Methods

8. Best Practices Practices: • Compare the incremental costs with the incremental benefits • Compare the one or two most relevant benefits with the costs • Find the “knee in the curve” or optimal value solution

9. Common Mistakes • “Over-averaging” for sake of simplification: • Example: • “Fort Hood data is representative of all bases.” • Assuming away cost: • Examples: • “Year-end funds will be available,” when that’s not the case • “Higher headquarters will pay for it.” • “Other organizations will pay for it.” • “Military personnel are free.” • Assuming away the problem: • Example: • “Unused office space is available.” • “Chief of Staff said we need this.” • Adding a layer of oversight will increase process efficiency These are examples of flawed assumptions that should have been rejected in step 2.

10. Common Mistakes: Over Simplification Which alternative has the best value?

11. The Cost of a Green Chiclet • “Green (fill in blank, i.e., funding, status…) for $55M is the preferred alternative” is the value statement. This should help in making a decision.

12. Effects of Over simplification • What happens when using a per cost or per benefit metric? Best Case: COA 2 is best?

13. “Cost Free” Analysis • In some cases, new programs or requirements will be supported entirely by resources that are already funded. This could be described as “cost free” in the sense that there no additional costs in terms of the budget. • Nonetheless, a “cost free” analysis must still consider cost in terms of opportunity cost: what the already-funded resource could have been used for if it had not been applied to the new requirement.

14. “ Cost Free” Analysis • Example: the Organization of the Joint Chiefs of Staff is planning a holiday party for itself. The Chairman of the JCS must assign three current employees to serve on the Holiday Party Planning Committee, which requires a commitment of 16 hours of work per member. CJCS is presented with two courses of action: • COA1: Appoint the Chief of Naval Operations, the Chief of Staff, Army, and the Chief of Staff, Air Force, to the Holiday Party Planning Committee. • COA2: Appoint Intern #1, Intern #2, and Intern #3—three GS-05 recent college graduates—to the Holiday Party Planning Committee. • The additional budgetary cost of this new Holiday party requirement is equal for both COAs: it is “cost free” because the salaries of current employees will be paid regardless of whether the Holiday party gets planned. Determine which COA has the lower opportunity cost.

15. Common Mistakes: Assuming Away the Problem All COAs proposed within a CBA must be solutions to the problem/opportunity statement. Assumptions should not solve the problem.

16. Non-Quantitative Methods Subjective Reasoning Uses informal criteria to rank alternatives A Fortiori Analysis Used when intuitive judgment strongly favors an alternative Assumptions are the basis in choosing which alternatives are preferred

17. Decision Matrix Decision Matrix: • A tool that compares benefits with costs to produce a single value score One of the best ways to elucidate the resource-informed decision to senior leadership is to include a Decision Matrix in the Decision Brief.

18. Decision Matrix Merits Pros Cons Error prone Highly judgmental Loss of information via normalization Results not definitive Scoring is subjective • Easy to use • Normalizes costs and benefits • Is a familiar tool • Flexible

19. Example - Decision Matrix Cost = $ quantifiable cost – $ quantifiable benefit or saving Rating: 1 (worst) to 9 (best) Benefit = $ non-quantifiable benefit and$ non-quantifiable risk COA-1 highest benefit COA-2 best value COA-3 lowest cost

20. Decision Matrix Note: If calculating Cost-Benefit index, the benefits criteria for each COA should be rated, not ranked. Otherwise, the Cost-Benefit index calculated would be meaningless.

21. Questions for Steps 7c and 7d • How sensitive is the recommendation to possible changes in costs, benefits, assumptions, etc.? • If the recommendation is highly sensitive, can deeper analysis be done? • Which elements of the CBA require sensitivity analysis? • Only test elements with significant uncertainty or risk. • Elements may include assumptions, constraints, costs, benefits, weighting of selection criteria, etc. • Address sensitivity from one or both perspectives: • What is the impact of a change of such and such magnitude? • How large a change can occur before the recommendation changes? • Have all probable risks and their respective impacts been identified? • Can the risks decrease future benefits? • Can the risks increase future costs?

22. Risk Assessment and Mitigation The goal of risk assessment is to answer questions like: What risks may occur? What is the likelihood that risks will occur? Are the source of these risks internal or external? What causes these risks? What are the consequences if the risks go unresolved? What assets, operations, activities, functions, etc. will be affected as a result? How much risk is tolerable? What should be done to anticipate and limit risks? Always measure the risk by the potential adverse impacts on alternatives.

23. Types of Risks Business/Programmatic Risk: Affect the budget and viability of a program Operational Risk: Affect the ability to perform a mission Process Risk: Associated with failing to meet standards and performance benchmarks in a newly established process Technical Risk: Associated with failing to develop or implement technology Schedule Risk: Associated with allocating time to perform and manage tasks Organizational Risk: Associated with managementchanges All risks need to be reflected in Costs and/or Benefits of COAs Risks must factor into the decision-making process

24. Ways to Measure/Address Risk • Methods of measuring and/or addressing risk in a CBA include, but are not limited to: • Effective Mean • Cost of Risk Mitigation • Sensitivity Analysis

25. Effective Mean (Expected Value) • The effective mean, or expected value, of a measurable quantity is the sum of all possible outcomes multiplied by their corresponding probabilities. • Example: if the cost of a new ground combat vehicle is judged to be $1.4M with 50% probability, $2M with 25% probability, and $1.2M with 25% probability, then the effective mean (expected cost) is $1.4M x 0.5 + $2M x 0.25 + $1.2M x 0.25 = $1.5M. • Example: if inter-theater transit time for a sustainment brigade is projected to be 5 days with 90% probability and 4 days with 10% probability, then the expected transit time is 5 x 0.9 + 4 x 0.1 = 4.9 days.

26. Cost of Risk Mitigation If a cost can be associated with reducing risk, then risk can be measured by that monetary value. Example: If for $22K extra, the risk of a schedule over-run can be reduced from 15% to 3%, then $22K can be a measure for the difference in risk.

27. Sensitivity Analysis Sensitivity Analysis: • Done mainly on the most important and least certain selection criteria, assumptions, and cost estimates • Addresses the impacts of risks comprehensively • Identifies the impact on a recommendation when elements of the analysis change, such as: • Assumptions • Costs • Benefits • Constraints • Scope • Weighing of selection criteria • Risk probabilities

28. Suggested Steps to Conduct Sensitivity Analysis • Divide analysis into two groups of factors: • Those outside an agency’s control (i.e., assumptions) • Those agencies that have a degree of influence or control • Choose several elements (costs, assumptions, benefits, etc.) that have greatest impact and are most likely to change • Vary each element over a reasonable set of values while holding the other variables constant relative to each other • Determine the impact of these changes on: • Total cost • Total benefit • Ranking of alternatives • Some factors that may warrant sensitivity analyses are: •  The effects of a shorter or longer economic life. •  The effects of variation in the estimated volume, mix, or pattern of workload; for example, the production rate or learning curve. •  The effects of potential changes in requirements resulting from either congressional mandate or changes in functional responsibilities. •  The effects of potential changes in requirements resulting from changes in organizational responsibility at the site, installation, base, or Army command/direct reporting unit/Army service component command level. •  The effects of changes in configuration of hardware, software, data communications, prime support equipment, and other facilities. •  The effects of alternative assumptions on areas such as project operations, inflation rate, residual value of equipment, and length of development. •  The effects of changing the fielding strategy.

29. Example: Sensitivity Analysis • Crystal Ball software may be used to perform sensitivity analyses. • In the example to the right, “Cost of Revenues %” has the greatest effect on Net Present Value, while “Year 4 Units Sold” is the least sensitive.

30. Exercise: Sensitivity Analysis • Perform a sensitivity analysis on the weight of Lethality in this Decision matrix. Test for weights of • 16% • 44% • 58% • 86% Determine if the recommended COA changes. Cost = $ quantifiable cost – $ quantifiable benefit or saving Benefit = $ non-quantifiable benefit and$ non-quantifiable risk COA-1 highest benefit COA-2 best value COA-3 lowest cost

31. Paying for Alternatives Identify billpayers to fund alternatives! Defense funding is a zero-sum game: every dollar awarded to one program comes at the expense of another competing program that will not be funded. • The CBA team rarely will have the authority to identify a bill-payer. This is a task for the organization’s comptroller/resource manager (the G-3 for most organizations) • Don’t offer a “gold watch” as a bill-payer … a program that is so important to the Army that it’s certain higher headquarters will reject it. • Alternatives will compete with other developments for funding if no billpayers are identified Note: This does not guarantee funding.

32. Quick Review Comparing the costs with benefits of each alternative is a required part of the CBA methodology. Sensitivity analysis is a technique that analyzes whether changes in assumptions, quantitative values, or priorities will affect the recommendation. Billpayers are the sources that will fund the costs of alternatives.

33. Review Exercise: Henry Ford • In a rented garage at 58 Bagley Street in Detroit, Henry Ford completed his first gas-powered car on the morning of June 4, 1896. He had spent $250 FY1896 dollars on equipment to build the car, and $100 on labor. The garage cost $15 per month to rent. After the car was assembled, it was clear that it would not be able to fit through the door of the rented garage. The car was to be used as a model for the two-year long test-driving phase of his R&D process, with the final goal of arriving at a model for mass production and sale.

34. Review Exercise: Henry Ford • Identify selection criteria and compare each of the following Courses of Action: • COA1 (Status Quo): Leave the car in the garage, do nothing else. • COA2: Disassemble the car and reassemble it outside the garage. • COA3: Obtain permission of garage landlord to widen the garage door.

35. Step 8: Report Results and Recommendations CBA Decision Package: • Preferred format is a narrative or PowerPoint • Accompany with a decision briefing • Write it in such a way that a layman can understand the topic Results and Recommendations: • Summarizes the analysis • Makes conclusive statements about each of the alternatives • The recommended alternative should follow • Results address how the alternatives were ranked using the criteria developed in Step 6 • Document all data and information used in Steps 1-8 • Provide supporting information so reviewers can understand how Steps 1-8 were developed Questions for the Reviewer: - Does the package contain all key elements that are accompanied with supporting documentation? - Does the recommended alternative address the problem, and is it consistent with the assumptions and constraints? - Does the analysis explain how the recommended alternative is the best one to satisfy the selection criteria?

36. Key Elements for Documentation The CBA’s quality and comprehensiveness may be enhanced by the following supplementary content: • Glossary • Defines unfamiliar abbreviations, acronyms, and terms used in the CBA • Timeline • Displays key dates and actions associated with the CBA in terms of its development and/or implementation • Coordination Sheet • Identifies who will need/have to review the CBA

37. Briefing the Results of the CBA Briefing Format: • This is a suggested but not required format. • This is just a suggestion. The actual briefing for a CBA (or any briefing, for that matter) should be determined based on the briefer’s style, the known preferences of the individual being briefed, and the subject matter. • An example has been provided in the CBA guide • It is not a substitute for a well documented CBA in narrative form. • The CBA coordinator, the decision makers, and stakeholders should collaborate to determine the content and format of the CBA decision brief as they will have differing preferences.

38. CBA Workflow Automation Tool Requirements Approval Resource Approval CBA Registration, Development and Submission CBA Validation Start (Capability Gap) End • Requirement approval forums use CBA to: • Make resource-informed decisions • Validate and approve requirements • Resource approval forums use CBAs to make resource- informed decisions • Identify tradeoffs and billpayers • Decide funding approval decision • Proponent organizations: • Register CBA online • Develop and submit CBAs online with value proposition linked to command’s strategic priorities • Proponent leadership endorses submission online • Organization validates the CBA methodology • Accuracy and completeness https://cpp.army.mil/ Register and submit your CBA online Draft Submitted Lead Analyst Review CBARB Review Final Review Completed

39. https://cpp.army.mil/

40. CBA Workflow Tool

41. Quick Review • Document the CBA narrative and share the results with stakeholders for approval. • CBA templates are optional tools that can be used to brief the results of the CBA. • Include all documentation and calculations in back up. • Ensure that the CBA clearly describes the value proposition, and highlight how the benefits outweigh the cost, risks, and trade offs.

42. FOUR DAYS FOUR

43. Mini-case Exercise #7 or“can you hear me now?” or “what’s the expected value?” • U.S. Army Special Operations Command maintains a specialized communications link between the Commander, USASOC, the Chief of Staff and Vice Chief of Staff, Army, and Team Leaders in the field during highly sensitive counter-terrorism missions. The link provides video and voice transmissions that are typically delayed by 30 seconds. Currently, $2M per month is spent on operating and maintaining the communications link: mostly consisting of payments to the National Security Agency for “satellite bandwidth.” The NSA has recently launched into orbit a new satellite that promises shorter transmission delays: typically 10 seconds, but with a 20% risk of 20 second delays. The satellite takes advantage of recently developed technology: costs to the Army for the first month are set at $10M, and are projected to decrease based on an 85% learning curve ($10M for the first month, 0.85 x $10M per month for the next two months, 0.85 x 0.85 x $10M per month for the next four months, 0.85^3 x $10M per month for the next eight months, 0.85^4 x $10M per month for the next sixteen months, etc.), though there is a 40% risk that costs will only decrease according to a 90% learning curve. • Because time delay is the overwhelming factor in communications for counter-terrorism operations, Army planners have assigned a 100% benefit weight to signal delay time. The benefit score will be determined by the function , where t is the expected time delay in seconds.

44. Mini-case Exercise #8 or“to insource or not to insource, that is the question” • Currently, a contractor provides 12 translators to a SIGINT battalion, each contracted for 1885 hours annually, on a government contract costing $1.2M per year. The battalion commander would like to evaluate the option of insourcing the labor to Army civilians. The total burdened cost per civilian is $85,209 and the estimated annual productivity is estimated at 1740 hours each. Determine whether insourcing would be more cost effective than the status quo.

45. Mini-case Exercise #9 or“I’ll gladly pay you Tuesday for air conditioning today” • OA22 has invited a contracting team to the Pentagon to inspect its centralized HVAC system. Following the inspection, the contractor reported minor damage to the HVAC system, and offers to replace it. Despite the damage, the HVAC system continues to function at the same capacity, although with risk of damage to the building. You assess that in any one year, there is a 15% chance of damage being done. In the event that there is damage, you assess that the average amount of damage will be $200K if occurring within the first year, $300K if occurring during the second year, $400K if occurring during the third year, etc. (increasing by $100K each year). If you decide to replace the HVAC system, the contractor has offered a payment program of $256K for the first year of ownership, $128K for the second year, $64K for the third year, etc. (reduce by 50% each year). The expected life cycle of this HVAC system is 6 years. Alternatively, you may buy the same HVAC system from a separate contractor at a cost of $100K for the first year, and $90K for every subsequent year. Under this special program, the contractor will replace the HVAC system at the end of its life cycle without any additional material cost (operations & maintenance, replacement costs not included). All costs are given in constant FY12 dollars. Perform all steps of the CBA process to support a course of action.

46. Mini-case Exercise #10 • The Captain of the Pequod has ordered a cost-benefit analysis to determine the optimal number of rowboats to deploy to pursue a whale sighting. As defined by the Commander, the selection criteria are manpower, stealth, and ease of sustainment, with weights of 0.5, 0.3, and 0.2, respectively. You have been told as a rule of thumb that the total mission cost can be estimated at $2200 for each rowboat involved in the hunt. Each rowboat can transport 15 fishermen.