Regional Climate Model Evaluation System (RCMES): Combining Observations & IT to Establish Core Climate Model Asse

1. Regional Climate Model Evaluation System (RCMES):Combining Observations & IT to Establish Core Climate Model Assessment Capabilities Jinwon Kim and Paul Ramirez and RCMES Science and IT Teams led by Duane Waliser (JPL), Science Leader Chris Mattmann (JPL), IT Leader NCPP Workshop, Boulder, Colorado, August 2013

2. RCMES: Combining Observations & IT to Establish Core Climate Model Assessment Capabilities Model evaluation is a key step in assessing climate model fidelity and in turn the (un)certainty of climate change impacts. Systematic experimentation and evaluation of GCMs have been undertaken for some time (e.g., AMIP, CMIP, CFMIP), while that for RCMs (e.g., NARCCAP, CORDEX) being more recent and less mature. NASA can provide critical and unique observational data to facilitate RCM evaluations and thus make key contributions to impact assessment process, e.g., the National Climate Assessment (NCA). RCMES was developed to facilitate model evaluations via easy access to observational data, especially from satellite remote sensing, and software tools for calculating and visualizing evaluation metrics.

3. RCMES: Combining Observations & IT to Establish Core Climate Model Assessment Capabilities(http://rcmes.jpl.nasa.gov; Powered by Apache Open Climate Workbench) Other Data Centers (ESGF, DAAC, ExArch Network) Model data User input URL Extract OBS data Extract model data TRMM Metadata Data Table MODIS Regridder (Put the OBS & model data on the same time/space grid) Users' own software for specific evaluation and analysis Data Table Extractor for various data formats AIRS Cloud Database Data Table Data extractor (Binary or netCDF) CERES Data Table Analyzer Calculate evaluation metrics & assessment model input data Data Table Soil moisture Assessment modeling Data Table Common Format, Native grid, Efficient architecture Visualizer (Plot the metrics) ETC Raw Data RCMED (Regional Climate Model Evaluation Database) RCMET (Regional Climate Model Evaluation Tool) • RCMES is flexible and open source software • Can utilize multiple (distributed) data sources. • Easily transferrable to local platforms (Linux, Mac-OS) • Open source (via Apache)

4. RCMED Datasets:Satellite retrievals, Surface analysis, Reanalysis, Assimilations • MODIS (satellite cloud fraction): [daily 2000 – 2010] • TRMM (satellite precipitation): 3B42 [daily 1998– 2010]‏ • AIRS (satellite surface + T & q profiles) [daily 2002 – 2010] • CERES and GEWEX-SRB radiation – Surface and Top of the atmosphere • NCEP CPC Rain Gauge analysis (gridded precipitation): [daily 1948 – 2010] • CRU TS 3.1: precipitation, Tavg, Tmax, Tmin [monthly means, 1901 – 2006] • University of Delaware precipitation and temperature analysis • Snow Water Equivalent over Sierra Nevada Mts [monthly 2000-2010] • NASA MERRA Land Surface Assimilation & pressure-level data [daily, 1979-2011] • ERA-Interim (reanalysis): [daily 1989 – 2010] • AVISO sea-level height [1992-2010] • (In progress)CloudSat atmospheric ice and liquid, Satellite-based snow (Himalayas), ISCCP cloud fraction, Fine-scale SST, etc. RCMET Metrics: • Bias (e.g. seasonal means or variance) • RMS error (e.g. interannual variability) • Anomaly Correlation (spatial patterns of variability) • PDFs (likelihoods, extremes and their changes) • Taylor Plots & Portrait Diagrams (overall model performance) • Statistical Tests • User-defined regions (e.g. water shed, desert, sea, political) • Datasets andmetrics are continuously updated.

5. RCMES: Combining Observations & IT to Establish Core Climate Model Assessment Capabilities • RCMES is being (will be) used for model evaluation in a number of regional climate experiment regions via worldwide collaboration • N. America – NARCCAP • WCRP CORDEX Regions • Ongoing/tested:Africa, South Asia, Middle East – N. Africa, Australia • Under arrangement: Caribbean, South America, East Asia, Arctic Not Illustrated Here Arctic & Antarctic Domains

6. Kim et al., 2013, J. Climate, 26, 5698-5715. Considerable biases with large-scale spatial structures exist in surface insolation fields. Figure. Surface insolation biases against the GEWEX SRB data Interannual variability (% SRB) Bias (% SRB) NARCCAP Multi-decadal Hindcast Evaluation: Surface Insolation Model performance varies according to regions and seasons. Figure. Evaluation of the mean & interannual variability in various regions within the conterminous US.

7. Model Evaluation and Observational Uncertainties • RCMES' capability to handle multiple model- and/or/both observation datasets allows to visualize model performance as well as uncertainties in the model evaluation due to observational data. Observational Uncertainty TRMM, CPC, CRU, UDEL, GPCP Figure. Evaluation of the annual-mean climatology from multiple RCMs and observations against the equal-weight multi-observation ensemble. The figure shows that there exist noticeable uncertainties among widely used observations. Kim et al., 2013, J. Climate, 26, 5698-5715.

8. The Sacramento River basin Area mapping Map the watershed area onto the RCM domain (Kim et al. 2000, J. Hydromet.) Construct watershed-mean met data for hydrology model The shaded area is the 0.5o-resoln RCM grid boxes that are entirely or partially included in the basin. Figure. Calculation of area-mean data for an irregularly-shaped watershed from gridded climate model data. • Transferring the gridded climate model data onto a watershed is the first step in assessing water resources using a bulk hydrology model thatruns on watershed-mean met data. • Overlay model grid over the watershed area. • Calculate the percentage of each grid box contained within the watershed area. This is the weighting factor for calculating the area-mean meteorological data. • Calculate the watershed-mean value of a variable P using the weights

9. Evaluation of extreme events Analysis of long-tailed temperature PDFs in NARR and NARCCAP RCMs: • Variance/skewness of daily temperature are related with weather extremes. • These PDF properties can also be related with regional processes • e.g., Positive tail in the LA basin is related with the Santa Ana winds • Utilize these PDF characteristics to evaluate RCMs in simulating extreme events Loikith et al., 2013: Geophys. Res. Lett., in press.

10. Regional Evaluation of GCMs CMIP5 GCMs - Observation NARCCAP RCMs - Observation Fig. July precipitation bias: CMIP5 present-day Ens & NARCCAP RCM Ens RCMES can be also used to evaluate multiple GCMs for various regions using multiple observation data. Can be useful, for example, in identifying GCMs most suitable for downscaling.

11. Summary The Regional Climate Model Evaluation System (RCMES) was developed to facilitate model evaluation via easy access to key observational datasets especially from satellite remote sensing data sets. RCMES is being used in evaluation studies for several regional climate experiments including NARCCAP and multiple CORDEX domains. RCMES includes calculation of a number of metrics for model evaluation. RCMES is continuously improved with additional datasets, more efficient database scheme, and evaluation metrics. RCMES is useful for a wide range of users: Can be obtained and installed as a Virtual Machine image http://rcmes.jpl.nasa.gov/training/downloads Deployable to user laptops and server machines with ease Source Code at Apache Open Climate Workbench http://climate.incubator.apache.org/ Command line interface Browser based interface API to construct your own evaluations

12. Ongoing and Future Efforts Links to ESGF Enable users to import data directly from ESGF for processing. Made-to-order system Distribution of RCMES in a virtual machine package as specified by users. Link climate model data with assessment models Preparation of met forcing data from climate model data, bias correction of met forcing data for specific assessment models. Performance-based variable weighting model ensemble construction Data processing and metrics Handling of very large datasets. Develop metrics for evaluating PDF characteristics (e.g., variance, skewness) Cluster analysis Collaborative Development through Apache Open Climate Workbench Open source effort based on a well known open source software community and team expertise at Apache

13. RCMES: Evaluation of Multi-model Hindcast in the CORDEX Africa jkim@atmos.ucla.edu; duane.e.waliser@jpl.nasa.gov; chris.a.mattmann@jpl.nasa.gov Terrain of the evaluation domain and the annual precipitation climatology Evaluation of the spatial variability of the annual mean precipitation Evaluation of the simulated precipitation annual cycle RMSE (% Annual mean) Correlation coefficients Kim, J., D. Waliser, C. Mattmann, C. Goodale, A. Hart, P. Zimdars, D. Crichton, C. Jones, G. Nikulin, B. Hewitson, C. Jack, C. Lennard, and A. Favre, 2013: Evaluation of the CORDEX-Africa multi-RCM hindcast: systematic model errors. Clim. Dyns, DOI 10.1007/s00382-013-1751-7. Project supported by: NASA: NCA (11-NCA11-0028), AIST (AIST-QRS-12-0002), American Recovery and Reinvestment Act. NSF: ExArch (1125798), EaSM (2011-67004-30224)

14. RCMES: Precipitation Evaluation of Multi-model Hindcast in the CORDEX South Asia – Indian subcontinent The hindcast and evaluation domain Uncertainties in the Observations R01 R04 R05 R06 Hindcast domain RCMs % of Observation Ensemble Model biases in simulating spatial variations in the annual-mean climatology Annual cycle evaluation for the northern mountainous region Indian subdomain R02 R01 mm/day R04 R03 Observations Supported by: NASA: NCA (11-NCA11-0028), AIST (AIST-QRS-12-0002); NSF: ExArch (1125798)

15. RCMES – Regional Climate Model Evaluation System Regional Downscaling Decision Support Global Climate Projections http://rcmes.jpl.nasa.gov/ • Sponsor Acknowledgements • NSF G8 Initiative: ExArch Project • NASA: National Climate Assessment (NCA) • NASA: Adv. Information. Systems Technology (AIST) • NASA: Comp. Modeling Alg. & Cyberinfras. (CMAC) Selected Publications Crichton et al., IEEE Software, 29, 63-71. Hart et al., ICSE 2011 Workshop Software Engineering for Cloud Computing - SECLOUD, Honolulu, HI, May 2011, 43-49, ISBN: 978-1-4503-0582-2, doi: 10.1145/1985500.1985508. Kim et al., 2013a, J. Climate, 26, 5698-5715. Kim et al., 2013b, Climate Dynamics, DOI 10.1007/s00382-013-1751-7. Loikith et al.,2013, Geophys. Res. Lett., in press. Mattmann et al., 2013, Earth Sci. Informatics, doi 10.1007/s12145-013-0126-2. Waliser et al., 2012, Tech. input for consideration to the 2013 Us National Climate Assessment, 19pp. Whitehall et al., 2012, WMO Bulletin, 61, 29-34. plus a number of IT papers and conference papers