Introduction to Performance Measures

1. Introduction to Performance Measures

2. Measures of Performance • Define baseline conditions • Quantify achievement of planning objectives • Compare alternative plans • New performance measures are generated to improve decision making

3. Performance Measures Must Be • Clearly defined • Easily understood • Directly related to planning objectives • Relevant to decision makers and stakeholders • Capable of addressing risk and uncertainty

4. Two Types of Performance Measures • Accounts- describe the overall effect of an alternative in a specific area (e.g. cumulative economic impacts) • Metrics- statistical or numerical measure of system performance Metrics are often treated as sub-accounts or used as surrogates for estimating total effects (e.g. system reliability)

5. Requirements for Performance Measures • Stakeholders must help develop performance measures • The “role” of performance measures must be understood by participants • Performance measures must facilitate decision making

6. Four Principle and Guideline Accounts • National Economic Development (NED): effects to the national economy • Regional Economic Development (RED): changes in the distribution of regional economic activity • Environmental Quality (EQ): effects on significant natural & cultural resources • Other Social Effects (OSE): relevant effects that are not reflected in the other three accountsBoth beneficial and adverse effects must be cited within each account for use in plan evaluation

7. A Closer Look at NED, National Economic Development • Describes increases in the net value of national output of goods and services • Impacts are expressed monetary units • Estimates a user’s willingness to pay for each increment of output from a plan to value goods and services • The opportunity costs of resources used in plan implementation are used to estimate adverse NED effects

8. NED Includes Estimates of Market & Non-Market Goods & Services in the Following Areas 1) M&I Water Supply2) Agricultural Floodwater, erosion & sedimentation Irrigation5) Urban Flood Damage Reduction6) Power (hydropower)7) Transportation (inland navigation)8) Transportation (deep draft navigation)9) Recreation10) Commercial Fishing11) OtherThe actual procedures for estimating NED depend on type of goods & services being provide

9. A Closer Look at RED, Regional Economic Development • RED describes the regional incidence of NED effects • The regions used in RED estimation are those that will be significantly impacted by a plan • RED is expressed in monetary units, other numeric units, or non-numeric terms RED sub-accounts • Regional income • Regional employment

10. A Closer Look at EQ, Environmental Quality EQ describes effects on ecological, cultural and aesthetic attributes of significant natural and cultural resources • Impacts are described in terms of duration, location, magnitude and other effects relevant to decision making, in either numeric or non-numeric terms • The perceived significance of impacts is identified based on institutional, public and technical recognition

11. A Four Phase EQ Evaluation • Phase 1 - Definition of resources • Phase 2 - Inventory of resources • Phase 3 - Assessment of effects • Phase 4 - Appraisal of effectsThis process is coordinated with NEPA regulations.

12. EQ Attributes Include Ecological Attributes Functional components • Nutrient cycling • Succession • Assimilative capacity, erosion • Other dynamic, and interactive processes and systems

13. EQ Attributes Include Ecological Attributes Structural components • Plant & animal species • Populations & communities • Chemical & physical properties of air, water & soil

14. EQ Attributes Include Cultural EQ Attributes • Evidence of past and present habitation that can be used to reconstruct or preserve human lifeways (e.g. archeological remains) • Historic landmarks • Places of cultural/religious significance Aesthetic Attributes • Sights, sounds, smells, tastes & tactile impressions & their interactions with natural & cultural resources

15. A Closer Look at OSE, Other Social Effects • OSE incorporates perspectives that are relevant to the planning process, but are not reflected in the other three accounts • Rigorous descriptions are not required • OSE effects are expressed in monetary units, other numeric units, or non-numeric terms

16. OSE Sub-Accounts Urban & community impacts • Income distributions’ • The quality of community life

17. OSE Sub-Accounts Impacts to life, health & safety • Risk of flood, drought or other disasters affecting the security of life, health & safety • Potential loss of life, property, or essential public services due to structural failure • Changes in air and water quality not reported in NED & EQ

18. Indicators of Risk in Performance Measures • Reliability • Vulnerability • Resiliency • Robustness

19. Reliability • The probability that a system will perform without failure over a specified period.

20. Vulnerability • The severity of impacts which are experienced when a system failure occurs.

21. Resiliency • The likelihood of system recovery from a failure.

22. Robustness • The ability of a plan or strategy to perform effectively if future conditions are different from projected conditions.

23. A General Recipe for Formulating Metrics • First, identify concepts that best capture system performance and the interests of stakeholders • Then, translate the concepts into measurable terms

24. Common Types of Metrics • Frequency that a constraint is violated (reliability) • Duration of constraint violations (resiliency) • Maximum and average levels of performance during a specified period • Maximum and average levels of constraint violations (vulnerability)

25. Water Use Perspective System Performance Concerns • Safe yield • Failures to meet unconstrained demand • Imposition of curtailments of differing severity • Costs of supplying water (profitability) • Available reservoir supply • Remaining aquifer Municipal Water Supplier

26. Water Use Perspective System Performance Concerns • Price of water • Adequacy of receiving water • Ability to exercise water right Industrial

27. Water Use Perspective System Performance Concerns • Channel closings • Imposition of light loading requirements Navigation

28. Water Use Perspective System Performance Concerns • Lake levels • Boat ramp accessibility • Recreational opportunities • Quality of recreational experience • Profits/Visitation rates Lake Recreation

29. Water Use Perspective System Performance Concerns Hydropower • Power production capabilities • Ability to meet minimum production targets • Profits

30. Water Use Perspective System Performance Concerns Consumer • Price of water • Imposition of curtailments of differing severity • Ability to exercise water rights

31. Water Use Perspective System Performance Concerns Fish Habitat • Streamflow • Scour/Sedimentation • Water quality

32. Water Use Perspective System Performance Concerns Indian Tribes • Impacts to fisheries habitat • Ability to exercise water rights

33. Potential Surrogates for Impacts Which are Difficult to Simulate Impact of Concern Potential Surrogate Streamflow levels at different life cycle stages Fish Welfare Wildlife Welfare Streamflow Levels Quality of Recreational Experience Reservoir levels, streamflow levels Fire Hazard Reservoir Storage Type of water use restrictions imposed Aesthetics Number and duration of water-use restrictions Consumer Pain and Suffering

34. Summary • Measures of performance are required for planning. • Measures of performance include accounts and metrics. • Principles and guidelines utilize four accounts: NED, RED, EQ, OSE. • Metrics communicate risks and evaluate system performance in terms of multiple interests. • Formulation of performance measures requires input from managers, stakeholders and decision makers.

35. Exercise