Validation of AMESD DMS drought products over South Africa

1. Validation of AMESD DMS drought products over South Africa Presented by Johan Malherbe , Researcher Agricultural Research Council (ARC), South Africa AMESD Final Workshop June 2013

2. Background • As part of the AMESD product deliverables by the ARC-ISCW, the AMESD Drought Monitoring System (DMS) was developed AMESD Final Workshop June 2013

3. Background • Two of the products produced by the AMESD DMS are evaluated for South Africa • Vegetation Condition Index (VCI) • Boolean (1/0) Drought Risk Map • The Drought Risk Map is a combination of current conditions and the SAWS forecast. In the specific case evaluated here, current conditions are indicated by the Percentage of Average Seasonal Greenness (PASG) product. • A = 3-month chance for rainfall below 33rd percentile larger than 40% • B = PASG < 75% • Drought risk = A*B • All products are calculated from the Normalized Difference Vegetation Index (NDVI), Rainfall (NOAA CPC) or Seasonal Forecast data by the AMESD DMS. AMESD Final Workshop June 2013

4. Background The 2012/2013 season • Near-normal to above-normalto above-normal rainfall occurred during the 2012/13 season over most of South Africa, except for the central parts of the country, including most of the North West Province. • Dry conditions over the central parts developed mostly due to relatively low rainfall during late summer • Seasonal forecasts however didn’t give any indication of relatively dry conditions to develop over central parts • Negative effect on drought outlook products where forecast is used AMESD Final Workshop June 2013

5. Data and Methodology • 3-month PASG for October-to-December (OND) • Evaluated against rainfall (as percentage of average) for September-December for randomly selected automatic weather stations • 3-month PASG for January-to-March (JFM) • Evaluated against rainfall (as percentage of average) for December-March for randomly selected automatic weather stations • Evaluated against photo evidence collected during March 2013 • VCI for 1-10 October 2012 • Evaluated against rainfall (as percentage of average) for September • VCI for 1-10 January 2013 • Evaluated against rainfall (as percentage of average) for December • VCI for 1-10 March 2013 • Evaluated against rainfall (as percentage of average) for February and January to February AMESD Final Workshop June 2013

6. Data and Methodology • VCI and Drought Risk products all created by AMESD DMS • Validation Data • Rainfall data • ARC Automatic weather station network • +- 50 randomly selected stations per aridity region • Arid • Semi-arid • Sub-humid/humid • Digital Photos with GPS coordinates • Focus on summer rainfall region AMESD Final Workshop June 2013

7. Data and Methodology • Spearman Rank Correlation • The Spearman Rank Correlation coefficient is a statistical measure of the strength of a monotonic relationship between paired data. It assesses how well the relationship between two variables can be described using a monotonic function. • Error matrix • specific table layout that allows visualization of the performance of an algorithm, in this case the PASG, in delineating areas (in this case) where rainfall is indicative of vegetation stress or not. • The error matrix is then used as a starting point for a series of descriptive and analytical statistical techniques reported. • Overall accuracy • Producer accuracy • User accuracy • Kappa (which provides a somewhat more reliable indication of accuracy)x AMESD Final Workshop June 2013

8. Results - VCI • Simple “eye-ball” verification shows promising results for early and late summer • Spearman rank correlation coefficients remained significantly (>95%) positive for most regions during the whole season AMESD Final Workshop June 2013

9. Results - VCI Spearman rank correlation between percentage of average rainfall for September 2012 and VCI for 1-10 October (early season), rainfall for December 2012 and VCI for 1-10 January 2013 (middle (of the rainfall) season), percentage of average rainfall for February and VCI for 1-10 March 2013 (late season (1 month)) and percentage of average rainfall for January to February 2013 and VCI for 1-10 March 2013 (late season (2 months)). Positive correlations that are significant above the 95% and 99% level of confidence are indicated in light blue and dark blue respectively. • The VCI successfully indicated the increase in vegetation activity shortly after significant rain as seen in the spearman rank correlation coefficient for early summer • middle part of the summer season, the correlation between rainfall situation and the VCI was significantly positive over most areas • lower correlation coefficient over the sub-humid areas • A very early start to the summer rainfall season, causing vegetation to be somewhat riper than average during this part of the summer with less new activity during December than would have followed on a dry early season • Cloud contamination can also hamper the product’s effectiveness over this humid region (10-daily composite) • A very significant rainfall event over eastern South Africa during January, still had a positive effect by March, hence better correlation with 2-month rainfall and late summer VCI AMESD Final Workshop June 2013

10. Results - PASG • Simple “eye-ball” verification shows promising results for early and late summer AMESD Final Workshop June 2013

11. Results – PASG (Rainfall stations) • Overall accuracy relatively high – all regions, • Early/late summer • Late summer – sub-humid – low • Same seasonal characteristics as described earlier • Producer accuracies remained high especially over arid and semi-arid areas where rainfall was above normal as well as below normal AMESD Final Workshop June 2013

12. Results – PASG (Photos) – late summer • Overall accuracy relatively high – both regions, • Especially over the semi-arid region, producer accuracies remained very high for wet and dry conditions AMESD Final Workshop June 2013

13. Recommendations • The study underlined the importance of emphasis on land-cover and lagged responses • The AMESD DMS PASG needs to be calculated not only from 10-daily maximum NDVI composites, but from monthly maximums derived from the 10-daily data • Monitoring products should always be used to track conditions, wether or not drought is indicated by the outlookproducts • The use of seasonal forecasts must be made cautiously as such products are still in their infancy. Once seasonal forecasts become more useful than an ENSO-based statistical forecast, promising results will filter through into higher accuracy of drought risk outlooks, especially during seasons such as 2012/2013, where the ENSO signal was relatively small/absent. • The decile-based index is safer to use than a yes/now index based on uncertain forecast • The VCI product of the AMESD DMS needs to be investigated and fixed for isolated no-data values occurring where the product needs to indicate near-zero values. • Climatic conditions over the far eastern parts of South Africa, over high rainfall areas, renders the use of vegetation monitoring products from space less reliable during late summer. The negative effect that cloud contamination and seasonal Rainfall-Vegetation lagged responses can have over these moist regions, necessitates the inclusion of rainfall-based products also (as available in the AMESD DMS), and not only NDVI-based products, over these types of regions AMESD Final Workshop June 2013

14. Season as per AMESD DMS • AMESD DMS monitoring products greatly exceeded usefulness of monitoring/FCST combined products • While boolean drought indicator failed due to lower accuracy of FCST, the decile-based index already indicated a tendency towards possible drought conditions within 3 months from the end of January • Observed rainfall anomalies were reflected in cumulative vegetation indices FCST OBS AMESD Final Workshop June 2013

15. END Thank you AMESD Final Workshop June 2013