Population Forecasting: What Method is Best?

1. Population Forecasting: What Method is Best? • Analysts that undertake population forecasting have a wide variety of method available to them, all with a mix of strengths and weaknesses. -Simple extrapolation -Complex Ratio -Complex extrapolation -Cohort Survival -Simple Ratio -Cohort Component • The question then arises:How should analysts go about choosing a method (or set of methods) to use to make population projections and a forecast?The answer? • IT DEPENDS • Pittenger and Smith et al make it clear that choice of projection method depends upon a number of factors. What factors do these authors identify as important when making the choice of forecasting method?

2. Factors Influencing the Choice of Forecasting Method Needs of the Users --Geographic Detail --Demographic Detail --Temporal Detail “Are User Needs Satisfied?” Plausibility “Do the Outputs Make Sense?” Face Validity --Availability of Data --Quality of Data “Are the Inputs Good?” Model Complexity --Ease of Application --Ease of Explanation “Can we do this?” “Can we explain what we did?” Political Acceptability “Are the Outputs Acceptable?” Resources--Money--Personnel--Time“Can we afford it?” Forecast Accuracy “Is the Forecast Accurate?”

3. Choosing a Forecasting Method:A Multi-Criteria Decision Making Process • What this decision boils down to is what is commonly called “multi-criteria decision making” (MCDM). • MCDM attempts to answer the question “What is the best method/answer for a given problem?” given that there are numerous, conflicting criteria for making a choice. • For example, “What home should I buy?” is a common “problem” faced by many individuals and families. • This is a question that requires MCDM, although most don’t recognize that they are using this method in making this decision. • What factors go into the choice of a new home? --Location --Cost --Neighborhood --Size (SqFt) --Size (BRs) --Schools --Amenities --Commute Time --Etc.

4. The Simplified MCDM Method • There are a number of MCDM methods for making decisions (some of which you will learn about in “Policy Analysis” (Methods IV)), but the overall approach to making a decision is based in a relatively simple procedure: 1) Identify the criteria of interest 2) Rank the criteria in terms of their importance 3) Weight the criteria 4) Use these rankings/weightings to “score” the alternatives 5) Make a choice based upon this analysis • What all of this points to is our original answer “it depends”: 1) It depends upon what criteria are important 2) It depends upon the rankings and weightings of these criteria 1+2 = 3) It depends upon what mix of factors is deemed the most valuable to the analysts and users

5. Forecasting Accuracy • In theory, the most important criteria for a forecast is its level of accuracy. We assume that a forecast that is off by only 2% is much better than one that is off by 20%. • However, the likelihood of forecast accuracy serving as the most important criteria in choosing a method depends on local conditions. • For example, sometimes politics plays a very important role in the choice of a method and, by extension, the result generated by a forecast (Riverkeepers vs St. Joe in Franklin County). • However, let’s assume for a moment that forecast accuracy is indeed the most important criteria for choosing a method. What does the empirical evidence have to say about the different methods?

6. Forecasting Methods: Evidence to Date • Chapter 13 of the Smith et al book provides an excellent, detailed summary of the current state of knowledge concerning the various forecasting methods: --Simple extrapolation (simple ratio) --Complex extrapolation (complex ratio) --Cohort component methods --Structural models (to be discussed) • What we think we know: Structural Models > CC > Complex Extrap > Simple Extrap (OR More Complexity leads to Greater Accuracy) • What are the major conclusions suggested by Smith et al’s review of the evidence? Does research suggest that this belief is correct?

7. Forecasting Methods: Evidence to Date • The evidence to date suggests that: • 1) More complex methods do not produce more accurate forecasts of total population.“to date, neither the sophistication of structural models nor the complexity of cohort component models has led to greater accuracy for projections of total population than can be achieved by simple extrapolation techniques.” (p. 312) • 2) No single method is consistently more accurate than the other methods • Why is this? Why doesn’t complexity result in accuracy? • --Uncertainty, uncertainty, uncertainty --The CC method still requires extrapolation; extrapolation of birth, death, and migration rates --Structural models still require extrapolation from recent or historical data

8. Forecasting Methods: Evidence to Date • Research into forecast accuracy has yielded other conclusions: • Forecast accuracy generally increases with population size. • Forecast accuracy generally increases for areas with slow, but steady positive growth rates. It decreases for areas with rapid population increases or population losses. • Forecast accuracy generally declines as the projection horizon (distance from the launch year) increases. • The rule of thumb on base period is generally found to true; The length of the base period should generally correspond to the length of the projection horizon. --Short projection horizon (1-5 years), short base period--Long projection horizon (20+ years), longer base period** **But evidence suggests that too much input data (greater than 10 years) may increase error

9. Responding to the Evidence • Given these general conclusions, what can analysts do to improve their projections and ultimately their forecasts? 1) Combine forecasts: Complete a number of projections and try to incorporate many of these in the final forecast. --It is assumed that every projection has error, but by completing and comparing different projections you can cancel out the errors across these projections and arrive at a more accurate forecast.--Methods for combining: 1) Average different projection results 2) Weight and average different projection results 3) “Composite” method; Find methods that work under certain circumstances and rely on these 2) Account for Uncertainty: Use methods to incorporate the concept of uncertainty in our forecasts. 1) Complete a Range of Projections using different assumptions (High, Med, Low Series) (Easy) 2) Use Prediction Intervals (aka Confidence Intervals) to generate different forecasts (Difficult)