EXPLORING DATA PATTERNS INTRO TO FORECASTING METHODS

1. EXPLORING DATA PATTERNSINTRO TO FORECASTING METHODS Topics: • Measuring forecast error. • Determining the adequacy of a forecasting method.

2. USING RESIDUALS • A residual is the difference between an actual observed value and its forecasted (or estimated) value. • Residuals (or errors) are the basis for typical measures of forecast accuracy. • Residuals are also used to assess the adequacy of a forecasting model.

3. RESIDUALS A residual is defined as follows: where et = residual (or forecast error) in time period t; Yt = actual value in time period t; Ft = forecast value for time period t.

4. FORECAST ACCURACY FORECAST ACCURACY MEASURES ME = mean error MAD = mean absolute deviation MSE = mean squared error RMSE = root mean squared error MPE = mean percentage error MAPE = mean absolute percentage error U = Theil’s U-statistic

5. FORECAST ACCURACY MEASURESBASED ON et

6. FORECAST ACCURACY MEASURESBASED ON et ME: Since there are both negative and positive errors, this measure does not accurately represent the average magnitude of the errors. MAD: This shows the average of the magnitudes (the absolute values) of the errors. MSE (or RMSE ): This penalizes large errors since each error is squared before averaging. This is used when a method with moderate errors is preferred over one with small errors and occasional extreme errors.

7. FORECAST ACCURACY MEASURES BASED ON PEt Relative or percentage error =

8. FORECAST ACCURACY MEASURES BASED ON PEt MAPE: Useful when the error relative to the magnitude of the values in the series is important. Also since these are percentages (no units) they can be used to compare methods used on different series. MPE: Used when it is important to determine if a forecasting method is biased (consistently forecasting low or high).

9. THEIL’s U - STATISTIC • Not covered in your textbook but a useful measure of forecast accuracy. • It allows comparison of any forecasting technique with the naïve method. • A simple naïve method uses the last observation as the forecast for the next value. • It is ratio that involves the squared relative error of the forecasting technique as the numerator and the squared relative error of the naïve method as the denominator.

10. THEIL’s U - STATISTIC • Since it deals with squared errors, large forecast errors are given more weight than small errors. • This is how it is interpreted: U = 1 : naïve is as good as forecasting technique being evaluated U < 1 : forecasting technique is better than naïve U > 1 : naïve is better than forecasting technique Why use a formal forecasting technique if the naïve is better (or just as good?)

11. ADEQUACY OF A FORECASTING METHOD Residuals should be random (method used has captured and accounted for all of the explainable pattern(s) in the data). Check using the ACF of residuals.

12. ADEQUACY OF A FORECASTING METHOD Residuals should be approximately normally distributed (this is to allow inference in statistical models). Check by constructing a histogram of residuals.

13. ADEQUACY OF A FORECASTING METHOD The parameters should be significant. Check by performing statistical inference using the parameter estimates (i.e., hypothesis test). Most models we will fit involve using a t-test for this. But since every hypothesis test has a calculated p-value as part of its standard output, we need only compare it to α = 0.05 in order to draw a conclusion. A p-value less than 0.05 means that the parameter is significant.

14. ADEQUACY OF A FORECASTING METHOD The technique should be simple to use and easily understood by planners and policy makers. Just as important as the other considerations when assessing adequacy!