Theory and Application of Benchmarking in Business Surveys

1. Theory and Application of Benchmarking in Business Surveys Susie Fortier and Benoit Quenneville Statistics Canada -TSRAC ICES – June 2007

2. Content • Introduction • Notation • Benchmarking methods • Timeliness issue • Implied forecasts and annual growth rates • Other uses: • Seasonally adjusted data • Linking problem • Conclusions

3. Introduction Main references • Dagum, E.B. and Cholette, P. (2006) Benchmarking, Temporal Distribution, and Reconciliation Methods for Time Series, New York: Springer-Verlag, Lecture Notes in Statistics 186. • Bloem, A. M., R. J. Dippelsman, and N. Ø. Mæhel (2001): Quarterly National Accounts Manual, Concepts, Data Sources and Compilation. International Monetary Fund, Washington DC.

4. Introduction Benchmarking : • Combining a series of high-frequency data with a series of less frequent data into a consistent time series.

5. Introduction Issues in Benchmarking : • Preserve period to period movement of the indicator (monthly/quarterly) series while simultaneously attaining the level of the benchmarks (annual). • Consider the timeliness of the benchmarks.

6. Introduction Example of a quarterly series

7. Introduction A quarterly series with its auxiliary source

8. Timeliness issue Introduction

9. Introduction Benchmarked series

10. Notation Methodological details :

11. Notation Methodological details : • With binding benchmarking, the benchmarked series is such that

12. Notation • A bias parameter can be estimated and used to pre-adjust the indicator series: • A bias corrected series is obtained as:

13. Notation • Alternatively, the bias can be expressed in terms of a ratio: • The bias corrected series is then:

14. Notation • Bias correction is a preliminary adjustment to reduce, on average, the discrepancies between the two sources of data. • Useful for periods not covered by benchmarks.

15. Notation Effect of the Bias Correction (ratio)

16. Methods : Pro-rating • A simple way to respect the constraints is to use This is the well-known formula for pro-rating.

17. Methods : Pro-rating Benchmarked series with pro-rating

18. Methods : Pro-rating BI ratio with pro-rating

19. Methods : Pro-rating Growth rates with pro-rating

20. Methods : Pro-rating Growth rates with pro-rating

21. Methods : Proportional Denton Benchmarked series with Prop. Denton

22. Methods : Proportional Denton BI ratio with Prop. Denton

23. Methods : Proportional Denton Growth rates with Prop. Denton

24. Methods : Proportional Denton Growth rates with Prop. Denton

25. Main method • Based on Dagum and Cholette (2006). • Generalization of many well-known methods: • Pro-rating • Denton (and proportional Denton) • Implemented at Statistics Canada with a user-defined SAS procedure: PROC BENCHMARKING • Project Forillon • Software Demo

26. Main method : Formula • The benchmarked series can be obtained as the solution of a minimization problem. • For given parameters and find the values that minimize the following function of : subject to

27. Main method : Formula • Solution when : Solution: “Regression-based” model from Dagum & Cholette

28. Main method : Formula • Solution when : Solution: where W is the T x M upper-right corner matrix from :

29. Main method : Formula • We can obtain pro-rating with the general formula with and : minimise under gives

30. Main method : Effect of • Consider the case where and . The function to be minimized under the constraints which aims at preserving the period-to-period change in the original series. Modified Dentonmethod

31. Main method : Effect of • Consider the case where and . The function to be minimized under the constraints which seeks to minimize the change in the ratios (not to preserve the growth rates but a fairly close approx). Variant of Proportional Dentonmethod with positive data!

32. Main method : Effect of • 3 parameters at play: • : model adjustment parameter • : “smoothing” parameter • bias (implied with ) subject to

33. Main method : Effect of

34. Main method : Effect of bias Benchmarking without bias ( )

35. Main method : Effect of bias Benchmarking with bias ( )

36. Main method : Effect of bias Benchmarking without bias ( )

37. Main method : Effect of bias Benchmarking with bias ( )

38. Timeliness issues • Adjustments for periods without benchmarks: • Benchmarked series give an implicit forecast for the unknown annual values. • The better the forecast, the lesser the revision! Proportional Denton (ρ=1, λ=1) Benchmarking with bias (ρ=0.93, λ=1)

39. Timeliness issues • 2 implicit forecasts for 2006: • Enhanced benchmarking method with explicitforecasts

40. Timeliness issues • One possibility for explicit forecast: • Use the annual growth rate from the indicator series on the last known benchmark.

41. Timeliness issues With explicit forecast ( )

42. Timeliness issues With explicit forecast ( )

43. Timeliness issues With ″recent″ bias( , bias=0.94)

44. Timeliness issues With ″recent″ bias( , bias=0.94)

45. Timeliness issues Minimize revision?

46. Methods : Summary so far! • Summary of methods presented: • Pro-rating • Denton (and proportional Denton) • Regression-based (Dagum and Cholette) • with or without bias correction • Denton with explicit forecast • Results from all of the above can be obtained by PROC BENCHMARKING.

47. Methods • Other methods • Other numerical methods revolve around different minimisation functions. • Statistical model-based approaches • See annex 6.1 in Bloem, Dippelsman, and Mæhel (2001) for variants and references • See also Chen and Wu (2006) for link between numerical, regression based and signal extraction methods. • Future version of PROC benchmarking?

48. Syntax : PROC Benchmarking PLEASE SEE SOFTWARE DEMO !! PROCBENCHMARKING BENCHMARKS=myBenchmarks SERIES=mySeries OUTBENCHMARKS=outBenchmarks OUTSERIES=outSeries OUTGRAPHTABLE=outGraph RHO=0.729 LAMBDA=1 BIASOPTION=3; RUN;

49. In SAS Enterprise Guide®(Demo)

50. Other uses : Seasonal adjustment • Seasonally adjusted series can be required to ″match″ given annual totals : • System of National Accounts (typical cases) • X-12-ARIMA version 0.3+ • FORCE spec (table D11 A) • With argument Type=regress : same methodology as PROC BENCHMARKING