The Challenges of Economic Forecasting: Insights and Recommendations

1. Some comments about forecastingBased on a paper provisionally entitled “Forecasting GDP and its expenditure components by the Economist Intelligence Unit: Are Country Reports worth paying for?” Corné van Walbeek

2. Forecasting techniques • Non-quantitative techniques • “I think that…” • Consensus seeking (e.g. Delphi method) • Scenario planning • Quantitative techniques • Time series methods (e.g. ARIMA) • Predicting with simple (single equation) behavioural models • Multiple equation models • Others • Technical analysis, especially for shares and currencies

3. Some background about macroeconomic forecasting • Economists are not particularly good at forecasting • Especially not in turbulent times (Granger, 1996) • Very poor at predicting recessions (Loungani, 2001) • Forecasts tend to cluster together, often quite far from the actual value (Granger, 1996) • Most studies consider the accuracy of GDP growth and inflation forecasts (Ash et al, 1998, Oller & Barot, 2000, Vogel, 2007) • Strong focus on industrialised countries (US agencies, IMF, OECD) • Strong focus on institutional forecasts; not much on private sector forecasts

4. Criteria for forecast accuracy • Bias • Mean error • Size of forecast error • Root mean square error • Ability to beat naïve alternative • RMSEEIU/RMSEnaive < 1 • Directional accuracy • Forecasting accelerations and decelerations correctly

5. A typical forecasting process • Use econometric models • Details are often published if organisation is “public” • If it is a private company, details typically not provided • Model consists of • Behavioural equations • Standard macroeconomic identities (e.g. GDP = C + I + G + X - M • Global identities (e.g. ΣX = ΣM) if relevant • Distinguish between exogenous and endogenous variables • Manual adjustments are made to forecasts if deemed necessary • Rigorous and iterative process of quality control and checking of forecasts

6. Next-year (t+1) forecast An example of the data: Austria, January 2007 Current year (t) forecast “Actual” value of last year (t-1) Against this value the forecasts for 2006 are measured

7. Magnitudes of the forecast errors

8. Some comments about the RMSEs • They are large • For current year forecasts: between 1.4 and 9.8 percentage points; median = 3.5 percentage points • For next-year forecasts: between 15 and 30 per cent larger than current-year forecasts • Large differences in RMSEs between magnitudes • RMSEs around 2 percentage points: C, G, TDD and GDP • RMSEs around 5 percentage points: I, X and M • Lower RMSEs for developed countries; higher RMSEs for developing countries

9. Comparing the EIU’s forecasts against naïve predictions • Assumption used for this paper: • The naively predicted growth rate for this year and for next year is the “estimated” growth rate for the previous year • Calculate RMSE ratio = RMSEEIU/RMSEnaive • If RMSE ratio < 1, then EIU forecasts are better (have smaller errors) than naïve alternative

10. Average of 0.77 Average of 0.82

11. Two recommendations • More modesty please! • Words like “prescient”, “decisive verdicts”, “precision”, etc. do not belong in a forecaster’s vocabulary • Publish confidence intervals • E.g. 67% confidence intervals (= point estimate ± RMSE) • 67% (or 50%) confidence intervals • Are not affected by outlying forecast errors • Are not as large as 95% confidence intervals (see Granger, 1996) • The existing RMSEs would be a good first approximation for such intervals • What if the intervals are embarrassingly large? • Be honest (“This magnitude is very difficult to forecast”) Advantages of publishing confidence intervals: • Emphasises the stochastic nature of forecasting to clients • Increases the credibility of the EIU (“Now they are always wrong. At least they will be right two thirds of the time”) • Allows users to do scenario planning with realistic “optimistic” and “pessimistic” scenarios