NOAA CLIMAS

1. Considering Equity in Climate Prediction Applications: Implications for Development and Evaluation of Decision Support Tools NOAA CLIMAS NASA HyDIS Raytheon Synergy NASA EOS NOAA GAPP NSF SAHRA 1Department of Hydrology and Water Resources, University of Arizona 2Department of Civil and Environmental Engineering, University of California-Irvine Holly Hartmann1 and Soroosh Sorooshian2

2. Calls for Societally Relevant Research and Products • Information is appropriate to the knowledge and concerns of the recipient. • Ensure that.. modeling improvements and data products are useful to the water resources management community. • Develop a strategy for… how these could be made more useful for [user] purposes. • Need studies of the benefits and costs of [hydroclimatic] information services. • Increase the value of weather and related … information to society. • Bring scientific outputs and users’ needs together. • Make climate forecasts more socially useful. • Stronger sense of responsibility for delivering timely and relevant tools. • Accelerate activities to integrate science with the needs of decision makers. • Integrate user needs… and ensure that research results are provided in a form useful for users. • Sources: Various USGCRP and NRC reports, 1997-2001

3. Issue: So Many Stakeholders! Continental Scale: Focus of modelers Different Scales (time & space) Different Issues Different Stakeholders Watershed/Local Scale: Where impacts happen Where stakeholders exist

4. Lessons from Stakeholders Building Expectations and Trust “What are your motives?” (agenda) “How long is your project really going to last?” (failed promises of past projects) “What did you do with the last survey?” (checking your responsiveness) • Poor interactions with users affects: • opportunities for future work • credibility of agencies, institutions and products • Building trust requires repetition & responsiveness • Concerns: agendas, science will be used to hurt them • Effective stakeholder integration  generate support for science funding & programs

5. Evaluating Societally Relevant Research and Products • Changed decisions & decision processes • Enabling system-wide change (transferability, scalability) • Public support for climate research Concerns for Climate Science Enterprise Project Objectives Affect… • Metrics • Structure of stakeholder interactions • Research products • Perceptions of climate science enterprise • Research funding

6. Objective: Economic Efficiency Metrics: Cost/benefits. Return on Investment. Stakeholder Interaction Structure: Consultant-client relationship with high-value clients (e.g., hydropower). Research Products: Customized Decision Support Systems. System optimization rules. Perceptions: Science serving special interests. Increasing competitive imbalances. Research Funding: By clients through private sector.

7. Objective: Agency Impact Metrics: Policy and regulatory impact. Stakeholder Interaction Structure: Work with agencies. Important role for policy analysts/scientists. Research Products: Traditional products. Refereed methodology and results. Hold up in court. Perceptions: Science serving special interests, agendas. Increasing regulatory burden. Research Funding: By managed sector, perhaps public.

8. Objective: Societal Equity Metrics: Breadth/diversity of applicability, accessibility, usability. Sectoral ‘market’ penetration. Stakeholder Interaction Structure: Engagement with diversity of stakeholders. Important role for social scientists. Potentially huge demand on researchers’ time. Research Products: Diverse. Non-traditional, but not “dumbed down”. Note: data << information << knowledge << wisdom Perceptions: Science providing useable information and practical tools. Increasing capacity to adapt to climate variability. Research Funding: Public.

9. Evaluating Success of Products and Process PRODUCTS: Forecasts - traditional publications - MS/PhD degrees - newsletter outreach - database of forecasts - forecast evaluation tool - “Climate in a Nutshell” (450+) - presentations to stakeholder groups (25+) - workshops (research/forecast/stakeholder) (8+) ??? Frequent interaction, from the outset Interaction… not outreach! Getting and giving Starting where the stakeholders are Moving dialogue & action forward

10. Stakeholder Use of Hydroclimatic Forecasts Common across all groups Uninformed, mistaken about forecast interpretation Understand implications of “normal” vs. “unknown” forecasts Use of forecasts limited by lack of demonstrated forecast skill Common across many, but not all, stakeholders Have difficulty distinguishing between “good” & “bad” products Have difficulty placing forecasts in historical context

11. Stakeholder Use of Hydroclimatic Forecasts Common across all groups Uninformed, mistaken about forecast interpretation Understand implications of “normal” vs. “unknown” forecasts Use of forecasts limited by lack of demonstrated forecast skill Common across many, but not all, stakeholders Have difficulty distinguishing between “good” & “bad” products Have difficulty placing forecasts in historical context Unique among stakeholders Relevant forecast variables, regions (location & scale), seasons, lead times, performance characteristics Role of of forecasts in decision making Technical sophistication: base probabilities, distributions, math

12. Forecast Assessment: CLIMAS Alternatives Efficiency Work with hydropower agencies & other high-value clients Develop customized evaluation tools Transfer to agencies Impact Work with regulatory & policy agencies Inform water supply policy via peer-reviewed science & policy analysis Equity Also work with stakeholders affected by changing supplies & policies Develop tools for knowledge development and diverse decision processes Requires on-going support of research products and tools

13. http://hydis6.hwr.arizona.edu/ForecastEvaluationTool/ • Initially for NWS CPC climate forecasts • Six elements in our webtool: • Exploring Forecast Progression • Forecast Interpretation - Tutorials • Forecast Performance • Historical Context • Use in Decision Making • Details: Forecast Techniques, Research

14. 1. Forecast Progression

15. Climatology 33% EC 33% 33% 3% 33% 63% 2. Forecast Tutorial Unknown Sometimes forecasters don’t know what the chances are… Climatology is only a reference (1971-2000), not a substitute forecast “+30% Chance of Warm” Each colored contour indicates a shift in the normal chances. EC - EQUAL CHANCES THE PROBABILITY OF THE MOST LIKELY CATEGORY CANNOT BE DETERMINED = Unknown Chances!!

16. 3. Forecast Performance Evaluation Sub-setting: Seasons, Leadtimes, Regions Criteria: Simple/Intuitive to Complex/Informative Transparency: Data behind analysis

17. 3. Forecast Performance Evaluation Sub-setting: Seasons, Leadtimes, Regions Criteria: Simple/Intuitive to Complex/Informative Transparency: Data behind analysis

18. 2003 2002 2004 4. Historical Context for Forecasts Requested by Fire managers… Applicable to any climate variable La Nina Recent History | Possible Futures Neutral Non-ENSO sequences

19. Wet Norm Dry 3.7” 8+” 0” 1.9” 4. Historical Context for Forecasts Willcox Jan-March Total Precip. 1971-2000 Willcox Jan-March Total Precipitation 1930-2001 1971-2000 Wet Near-Dry Normal Precipitation (Inches) Year Exceedance Probability 10 years had more than 3.7 inches 10 years had less than 1.9 inches 10 years were in the middle

20. 4. Historical Context for Forecasts El NinoLa Nina 50% Wet 0% 30% Norm 25% 20% Dry 75% El Nino La Nina 1 2 3 4 5680 1 2 3 4 5680 Willcox, AZ: Precipitation, JFM

21. Lessons of FET for Climate Services Data << Information << Knowledge << Wisdom Custom real-time data access, analysis, and information Value-added interpretation Multiple entry points along continuum of sophistication Opportunities and tools for increasing sophistication Knowledge development emphasis vs. decision support Other issues: Accessibility, ease of use (information management, updating) • Are these concepts & tools transferable and scalable? • Test with new products, inter-RISA opportunities

22. Facilitating Information Intermediaries Ease of Use  Profile and Projects: save a history of your work on each "project", so you can return to your work any time, easily repeat past analyses using updated data. • Accessibility  Report Generation • create PDF reports of your analyses for non-Internet users • automatically includes legends, data sourcing, contact information, caveats, explanations • sections for user-customized comments Future: Automated Updating & Additional Products: water supply forecasts, experimental climate forecasts, drought monitoring

23. Lessons Learned: Knowledge Development Tools • Transferable, scalable tools are possible! • Focus on knowledge development, not just data & information. Stakeholders Information needs, understanding, access Social Science Effective communication Natural Science Forecast skill, interpretation

24. Lessons Learned: Knowledge Development Tools • Transferable, scalable tools are possible! • Focus on knowledge development, not just data & information. • Interactive webtools require major commitment and resources. • Prototypes insufficient! • Stakeholders need reliable tools, which require solid software foundation, organized development, sustainability for maintenance and expansion. Stakeholders Information needs, understanding, access Social Science Effective communication Natural Science Forecast skill, interpretation Computer Science Web programming

25. Climate Research Impacts Society… Unexpectedly? Have you heard comments about role of science? Which objectives are supported by your research and products? What are your success metrics? Are there synergies in webtool development?