GOES-R AWG Product Validation Tool Development

1. GOES-R AWG Product Validation Tool Development Downward SW Radiation at Surface and Reflected SW Radiation at TOAHongqing Liu (Dell) Istvan Laszlo (STAR) Hye-Yun Kim (IMSG) Rachel Pinker (UMD) Ells Dutton & John Augustine (ESRL)

2. OUTLINE • Products • Validation Strategies • Examples • Ideas for Further Enhancement and Utility of Validation Tools • Summary

3. Products • Shortwave Radiation Products: • Downward Shortwave Radiation at Surface (DSR) • CONUS: 25km/60min • Full Disk: 50km/60min • Mesoscale: 5km/60min • Reflected Shortwave Radiation at TOA (RSR) • CONUS: 25km/60min • Full Disk: 25km/60min Only daytime

4. Monitoring & ValidationBackground • Functions of tools: • routine monitoring (may not need reference data) • routine validation (reference data, matchup procedure) • deep-dive validation (reference data, other correlative data, matchup) • Basic elements: • data acquisition (ABI, ground, other sat products) (Fortran 90) • spatial and temporal matching (lots of possibilities) (Fortran 90) • analysis (computing statistics) (IDL) • Metadata • Accuracy/Precision • RMSE • Minimum/Maximum Error • present results (display maps, scatter plots, tables) (IDL)

5. Validation Strategies Reference Dataset Ground Measurements • High-quality routine ground radiation measurements over Western Hemisphere from 20 stations from SURFRAD (ftp://ftp.srrb.noaa.gov/pub/data/surfrad/) and BSRN (ftp://ftp.bsrn.awi.de/) networks. • Satellite Measurements • Clouds and the Earth’s Radiant Energy System (CERES) Cloud and Radiative Swath (CRS) dataset: (1) measured TOA upward SW flux, (2) calculated Surface and Atmospheric Radiation Budget (SARB). http://eosweb.larc.nasa.gov/PRODOCS/ceres/level2_crs_table.html Collocation/Match-up • Reference (“truth”) data • Collocation of ABI retrievals and reference data is performed at the instantaneous time scale. • Matching: ABI retrievals averaged spatially; ground measurements averaged temporally. Averaging window size is flexible. • Independent satellite retrieval (CERES) • Collocation: CERES data are averaged to the ABI retrieval grid on a daily basis. • Matching: current retrievals use MODIS data as input; CERES is on same platform; no need for temporal matching.

6. Routine Validation ToolsInstantaneous Monitoring • Present retrieval results • Specify date & load data • Selection from ‘Variable’ menu • Primary Outputs (image) • DSR • RSR • Diagnostic Outputs (image) • Surface diffuse flux • Surface albedo • Clear-sky composite albedo • Clear-sky aerosol optical depth • Water cloud optical depth • Ice cloud optical depth • Quality Flags (image) • 66 flags (inputs, retrieval, diagnostics) • Metadata (ascii file output) • Independent of validation truth; can be executed automatically by scripts once retrievals are available.

7. Routine Validation ToolsValidation with Ground “Truth” • Validates DSR&RSR for a period of time • Specify time period & load data • ‘Validation’ menu Generate scatter plot of retrievals against measurements Generate validation statistics and output to ascii file • ‘TimeSeries’ menu Generate time series plots of retrieval and measurements over ground stations

8. ”Deep-Dive” Validation ToolsValidation with CERES • An expansion of routine validation with CERES including cross validation against NASA SARB satellite products • Options: • Scene types • all; snow ; clear ; water cloud; ice cloud • Retrieval path • Hybrid path • Direct path only • Indirect path only • TOA matching (all; succeed; failed) • Surface albedo (all; succeed; failed) • Specify date & load data • Selection from ‘Validation’ menu • Reflected SW Radiation at TOA (RSR) • Retrieval; Retrieval-CERES; Retrieval-SARB Tuned; Retrieval-SARB Untuned; • Statistics (Scatter plot; Statistics in ascii file) • Downward SW Radiation at Surface (DSR) • Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics • Absorbed SW Radiation at Surface (ASR) • Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics • Absorbed SW Radiation in Atmosphere (ABS) • Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics • Surface SW Albedo (ALB) • Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics

9. Ideas for Further Enhancementand Utility of Validation Tools • Calculate and display • additional statistics (histograms) • temporal averages on different scales (daily, weekly, monthly) • Identify signatures by which even non-experts can identify potential problems – needed for routine operational monitoring • Implement automatic detection of possible systematic drift or continuous abnormal retrieval in routine validation. • establish “reference” (expected) statistics from good data • compare time series of actual statistics with reference stats • trigger action (e.g., sending warning email) when actual stats exceed reference stats + x std. • Combine SW validation with LW radiation retrievals • check consistency • e.g., high RSR low OLR is expected for cloudy scenes • additional diagnostic information for deep-dive validation (LW radiation) • Current tool uses retrievals from MODIS proxy data. Adjustment to tools for retrievals from geostationary orbit will be needed (data preparation).

10. Summary • Current tools perform three functions: • routine monitoring of product • routine validation with reference data • deep-dive validation with reference and intermediate data • Validation truth data have been identified and processed • Planned enhancements include: • more stats • automatic detection of problems • checking consistency with LW

11. GOES-R AWG Product Validation Tool Development Upward LW Radiation at TOAUpward and Downward LW Radiation at SurfaceHai-Tien Lee (CICS/UMD) Istvan Laszlo (STAR/NESDIS) Ells Dutton & John Augustine (ESRL) Acknowledgments: NOAA SURFRAD, NASA CERES, BSRN, DOE ARM, Eumetsat GERB & LSA-SAF GOESR AWG Annual Meeting, June 14-16, 2011, Fort Collins, CO

12. Products • Longwave Radiation Products: • Upward LW Radiation at TOA (OLR) • CONUS: 25km/60min • Full Disk: 25km/60min • Downward LW Radiation at Surface (DLR): Clear sky only • CONUS: 25km/60min • Full Disk: 25km/60min • Upward LW Radiation at Surface (ULR): Clear sky only • CONUS: 25km/60min • Full Disk: 25km/60min GOES12 Imager OLR

13. Validation StrategiesReference Dataset (Ground) • Ground Measurements • High-quality routine ground radiation measurements over Western Hemisphere used for validating ABI Longwave Radiation retrievals are collected from 7 stations from SURFRAD network. • Selected stations of BSRN and Eumetsat LSA SAF that provide surface upward and downward longwave radiation measurements can be used for offline/framework algorithm evaluation.

14. Validation StrategiesReference Dataset (Satellite) • Satellite Measurements • OLR product from Clouds and the Earth’s Radiant Energy System (CERES) Single Scanner Footprint (SSF) datasets are used as algorithm validation reference. • Future NPP and JPSS OLR (from CERES FM5/6) can be used for routine monitoring and evaluation (possibly with lag). • Operational HIRS OLR from NOAA and MetOp polar orbiters will be used as a backup for routine monitoring purpose.

15. Validation StrategiesTools , Statistics & Visualization • Tools: • IDL (primarily) • Data Collocation • Instantaneous Monitoring • Validation over Ground Stations • Validation with CERES • Deep-dive Validation over Ground Stations • Deep-dive Validation with CERES • Statistics: • Metadata (ATBD), plus Mean/StDev for Global, zonal and selected domains of interests • Mean, StdDev, RMS, Min and Max of Errors • Visualization: • IDL, GrADs • Figures rendered in PNG format

16. Validation StrategiesExample of Deep-Dive Extended OLR Validation (March 2004) FM1 FM3 FM2 FM4 OLR Error vs LZA OLR Error vs SEVIRI Ch 5 radiance (UTH) OLR Error vs SEVIRI Ch 9 radiance (window) OLR Error as a function of Ch 7 and Ch 9 radiances

17. Summary • Validation truth data have been identified and being acquired • Validation tools are designed to perform: • Routine monitoring of product • Routine validation with reference data • Deep-dive validation with reference and auxiliary data • Planned enhancements include: • Temporal tracking of stats • Define level of alarms (for routine monitoring) • Explore possible sources of more ground truth • Clear-sky identification with auxiliary data.

18. END

19. Monitoring & ValidationBackground • Functions of tools: • Routine monitoring (may not need reference data) • Routine validation (reference data, matchup procedure) • Deep-dive validation (reference data, other correlative data, matchup) • Basic elements: • Data acquisition (ABI, ground & satellite products) (Unix Script, IDL) • Spatial and temporal matching (closed pixel vs area average) (IDL) • Analysis (computing statistics) (IDL, Datadesk) • Present results (display maps, scatter plots, tables) (IDL, GrADs, Kaleidagraph) • Special considerations: • Degradation flag