ELECTION PROJECTIONS: How It Started and Where We Are Now

1. ELECTION PROJECTIONS: How It Started and Where We Are Now Clyde Tucker

2. Overview • A History 1936-2016 • What’s Happening Now • What Happens on Election Day • Pre-election Polls

3. Overview • The focus here is on Presidential elections • In a general election we also project winners of other statewide races, including senators, governors and some referenda • There are usually 50 or more of these other races • Of course, calling the Presidency involves calling 51 separate state races • On any given election night, some races are never called because the margin is too small, usually less than 1 % • In 2004, no network called the Presidency

4. A History 1936-2016 • The 1936 Literary Digest Poll predicted that Alf Landon would defeat Franklin Roosevelt based on a biased sample and a biased response. Roosevelt won all but 8 electoral votes. • Pre-election polls mistakenly predicted that Dewey would defeat Truman in 1948, but Truman’s late surge was not captured by the polls that quit polling too early • These events occurred largely before major television networks projected elections

5. First computer (UNIVAC I) used by CBS on election day in 1952 using historical voting patterns and early returns, correctly predicting the Eisenhower landslide • By 1956, all networks used computers to predict the election early in the evening • In 1960 and 1964, an NBC team led by statistician John Tukey developed a procedure for predicting the Presidential election using early results from bellwether (or “swing-o-metric”) precincts • ABC began using regression-based models to predict the outcome of the Presidential election in 1964

6. Beginning in 1968, the networks went from projecting the nationwide vote to calling individual states At the same time, the CBS News Election and Survey Unit, led by Warren Mitofsky, began to develop models of election outcomes using the latest methods for statistical estimation These models used either actual election returns from a random sample of precincts collected shortly after polls closed or, later on election night, county reports Beginning in the 1980s, models based on results from exit polls in sample precincts were used in some cases to estimate the outcome at poll closing (before actual returns were available)

7. What combination of data were used to make these calls (exit polls, sample precinct returns, county returns) depended on how close a race was—the closer it was, the more information needed • In the early 1990s the networks consolidated their data collection operations under Mitofsky and his colleague, Murray Edelman, who also was responsible for developing methods of quality control • This new organization, Voter Research and Surveys, later became the Voter News Service when Mitofsky left after the 1994 election

8. By 2000, calling elections had gotten more complicated • No longer could the networks rely just on votes from election day as the number of early voters and absentee voters rapidly increased in certain states. In fact, Oregon and Washington relied heavily on voting by mail • Added to that was the decline in response rates to exit polls which could result in incorrect results referred to as within precinct error (WPE) • Errors in reports of county returns were also a factor in the incorrect calls in Florida in 2000

9. After VNS closed in 2003, Edison Media Research (EMR) and Mitofsky International were hired to produce an election projection system for the newly formed consortium of networks and AP (the National Election Pool or NEP) By 2004, it was no longer possible to make accurate projections without taking into account large numbers of early voters in some states Telephone polls using random-digit dialing (RDD) were conducted the weekend before the election in certain states with large numbers of early voters State estimates involved combining the exit poll of election-day voters and the RDD survey of early voters using weights reflecting the expected size of the two groups of voters

10. RDD surveys have a long history of use by political polling firms for pre-election surveys Prior to the late 1960s, most polls had been conducted with face-to-face interviews, but, by then, sending interviewers into homes was becoming prohibitively expensive Work in the 1970s by Mitofsky, with assistance from Joseph Waksberg, led to an efficient RDD procedure for locating residential numbers among all the possible telephone numbers in the U.S. In the early 1990s, Lepkowski, Casady and Tucker developed an even more efficient design using only groups of 1000 numbers that included listed residential numbers

11. Yet several problems still plagued RDD surveys—declining response rates, an expanded universe of telephone numbers leading to less systematic assignment of residential numbers with lower densities of residential numbers in 1000 banks, and cell phone-only households Today cell-only households have increased to at least 50% of all households and well over 60% when households with both land and cell phones that receive most calls by cell are included To improve RDD surveys, banks of 100 numbers as opposed to 1000 numbers are now used and a new “dual-frame design” combining landline and cell numbers has been developed

12. To prevent leaks of early exit poll results, since 2006 the heads of the decision desks for the NEP members are locked in a “Quarantine Room” from 11 AM to 5 PM on election day • Only they and employees of Edison have access to results • After 5 PM, the decision desk heads brief producers and reporters on the results • In 2008, neither party fielded an incumbent or a sitting Vice President for the first time since 1952 • Super Tuesday included primaries and caucuses involving both parties in more than twenty states • The Democratic race lasted into June

13. Projecting primary elections are more difficult than general elections—fewer useful past races for use in models exist, order of finish has to be determined when there are 3 or more candidates, caucuses are more complicated than primaries, and primary rules vary more state-to-state than in general elections • Early voting increased in 2012, requiring even more RDD surveys of early voters • Again, this year neither an incumbent nor a sitting Vice President was a candidate • The Republican primary race lasted into May and the Democratic one went into June

14. What’s Happening Now Samples of precincts are being selected from all precincts in a state Some of these will be designated as exit poll precincts and final results will be collected from all of the sample precincts Telephone samples are being selected for dual-frame surveys of early voters Simulated exit poll results and returns from both sample precincts and all counties are being used to practice calling elections Rehearsals began in August and are now run twice a week using the simulated data

15. The three election decision teams are assigned a set of states for which they will call all designated statewide elections • Besides rehearsing calling the elections, the decision teams are becoming familiar with all of their assigned races • One of the teams also practices calling which party will control the new House • All election projection software is being tested by programmers and decision team members • Two weekends in October will be devoted to studio rehearsals in which the decision teams and the on-air talent practice calling the elections and reporting the results

16. Election Eve ...we hope all the last-minute details have been dealt with. Everyone has his or her assignment. Everyone has been trained and rehearsed: the vote collectors at the precincts, the exit poll interviewers, the analysts, those entering and reviewing vote returns and [other] data, those who use the results in the television and radio studios. There are literally tens of thousands of people in a myriad of jobs. All our reference materials are where they should be. All the phone numbers we need are handy. Every computer system works. We are ready for Election Day, the culmination of the last two years’ work.

17. A good night’s sleep and we can be ready for a day that will last from the time we get up on Election Day until after the next night’s network news broadcasts—some 36 hours later. Unfortunately, the night’s sleep is not very restful. Usually, the phone rings too early in the morning about yet another problem and a few more missed details cross our thinking --Warren Mitofsky (2002) “Father of Exit Polling”

18. What Happens on Election Day • Exit polls are conducted at a subset of the sample precincts during the day by Edison interviewers • Exit poll interviewers sample every nth voter leaving the polls based on the past vote totals and factoring in likely nonresponse to produce about 100 survey ballots at the end of the day • The interviewers call the results in 3 times a day and, on the third call, they get an estimate of the total number voting at the precinct from poll officials for use in turnout estimates • Other Edison employees will collect the actual results from all the sample precincts after the polls close

19. At 11am, I, along with one other decision team member and the CNN pollster, go to the Quarantine Room to view the results from the first wave of exit polls as well as the polls of the early voters • At that point, we begin to look at the first wave of exit polls from the East Coast • The first waves of the exit polls from the Midwest and West come in early in the afternoon • The second wave of results from the East and the Midwest come in after 3pm

20. During the afternoon, we examine the results to determine if they seem to be what we would expect and that there are no obvious errors We also are sensitive to the possibility of within precinct error in different races As 5pm draws near, the CNN pollster prepares a report about the results, including who is voting for whom and why At the same time, the two of us on the decision team create a paper and electronic record of what we believe to be the best estimates for each of the races based on the models computed for each race And there are a number of models

21. Both simple geographic and party strata estimates just using the exit poll are computed These strata are based on past voting history The strata estimates are weighted by expected or actual size of the strata and summed to state estimates The same strata estimates are formed using ratios of current candidate vote to past candidate vote for all exit poll precincts in the strata In addition to estimates based solely on the exit polls, two composite estimates are formed using the best estimate (the one with the lowest variance) In states with large numbers of early voters, a composite of the best estimate with an RDD survey of early voters is formed using weights directly proportional to the expected size of the two groups

22. The best estimate (with the early voters included where available) is composited with a prior estimate based on pre-election polls and weighted inversely proportional to their variances This is usually the best estimate we have coming out of the quarantine room at 5pm However, we still only have at best two waves of exit poll data at that time After returning to the television studio, we receive information from the third wave of calls shortly before the polls close in a state We do not call a race until the polls close in the entire state, even if we have county returns in a split time zone state So calls are staggered throughout the night beginning at 7pm

23. Each decision team reviews the races in the states where polls are ready to close and recommends which races can be called • A colleague and I consider the recommendations and either decide to make the call or return a race to the decision team for further review • A call is made based on an estimate of the difference between candidate percentages of the vote in the best estimator • An estimate of sampling error (the standard error) is used for testing if there is a significant difference between the candidates (t-test of difference in proportions) • The difference must be large enough to fall outside of a .005 confidence interval

24. Only when a candidate’s lead meets this standard will a call at poll closing be considered • Even then there may be good reason not to call the race • One reason may be that we are not sure that the exit poll results are correct • Once 10 exit poll precincts are overlaid with the actual votes, an estimate of WPE is formed and updated as more sample precinct votes come in • This information about the accuracy of the exit poll estimate may provide the confidence needed to make a call

25. Of course, even then many of the races are just too close to call without more data • If we cannot call a race with the exit poll and/or the actual returns from the sample precincts, we need to wait for county returns from the Associated Press • The county returns are the officially reported votes by precinct • These returns are used to create both geographic and party county models using ratios of the current results to the results from a past race

26. Each county estimate for a candidate in the current race is the vote count for the candidate weighted up by the total number of precincts in the county divided by the number currently reporting Standard errors of the difference between candidates statewide in the county models are based on a regression estimator using historical data where the independent variables are the proportion of precincts reporting statewide and the square of that proportion The same .005 significance level is used to determine if the difference in candidate percentages are sufficient to call the race

27. A state often releases early voting results by county very quickly • When those results are a large portion of the total votes, the county models may allow us to call the race • Otherwise, we have to wait for more county results, which can come in slowly • To improve our chances of calling the race, we composite the county model with the best precinct estimate, using all the information we have to produce an integrated model • The same standard (.005) must be met to call a race with the integrated model

28. As the returns come in, decision teams will recommend a call in more and more races • The House model usually allows us to announce which party will have control between 9 and 10pm • As each state is called for one candidate or the other, a running tally is kept of electoral votes for each candidate • Once one candidate reaches 270, the Presidency can be called • This could happen before 11pm or after 11pm when the polls close on the West coast • It can also be much later

29. Meanwhile, we have all the other statewide races to call (including the presidential races in the remaining states) • Calling races will continue into Wednesday morning • Usually we finish between 4am and 6am, when only a few races remain that are going to be too close to call and may require recounts • However, at least some of us will remain in the studio if the Presidency has not been called • We will stay until we make the call or decide that we cannot based on the information we have

30. Using Pre-election Polls to Estimate the Horserace • Cell-phone only households and dual-frame design • Increasing nonresponse affecting representativeness (especially the young) • Identifying likely voters • Allocating undecideds • House effects (political bias, mode, funding, methodological sophistication) • Time constraints • Weighting the results

31. National polls are of great interest in presidential elections • But the role of state polls will continue to grow because of the focus on battleground states and calling the electoral college • Given the low response rates in all the polls, representativeness will remain an issue • We still have a need for large national in-person surveys like the CPS and ACS for weighting • An average of several polls is more reliable than relying on a single poll

32. Problems with Using the Horserace Question to Evaluate Survey Research • The importance of the horserace • Very important behavior--the best known product of survey research (unemployment rate or presidential approval probably next) • A gold standard is available • The problem with electoral choice • Only one of many products of survey research • Binary choice • Constrained distribution • More than 40% can be initially assigned to each party • In poll with 1000 respondents, margin of error (+/- 3%) covers a good part of the remaining difference between the candidates • Worry if off by 3 or 4 points, especially if the wrong winner