British Election Study - 2005

1. British Election Study - 2005 Harold Clarke, David Sanders, Marianne Stewart, Paul Whiteley

2. Face to Face Probability Surveys - The Gold Standard? • Probability Samples Yield Lists Not Achieved Samples • Non-Response is a growing problem in all probability surveys (61 percent response rate for 2005 BES) • respondents are unlikely to be a random subset of the list – ‘top-ups’ do not help • They are very expensive (The American National Election Study in 2004 cost $5500 per voter interviewed) • They are slow moving – two to three months to get the data

3. Face to Face Probability Surveys - Additional Issues • There is a limited ability to do panel surveys – when these are required for causal inference • They have limited ability to do experiments when field experiments are increasingly being utilised in social science research • The costs mean that we get one study only – replication is problematic • The have limited or no ability to study voters in the inter-election period where a lot of the interesting action takes place • They can’t be used to study election campaigns when these are becoming more important for understanding election outcomes.

4. Is the Internet an Alternative? • Fast responses to internet surveys – typically 95 per cent within a couple of days. This is ideal for campaign studies focusing on short-term dynamics • They are very cost effective – in the BES significant savings are obtained in terms of the costs per interview • Large N’s are possible – the marginal cost of adding a respondent is small • Experiments are possible with large treatment groups • No interviewer effects – which can be a problem when interviewers are socially homogenous • BUT – they are not random samples. In the BES they are a type of quota sample

5. BES 2005 CORE FACE-TO-FACE PANEL SURVEY: Wave 1Pre-election Probability Sample, Face-to-Face N=3589 128 Primary Sampling Units Wave 2Post-election Probability Sample, Face-to-Face N=4161 Including top-up, mail-back; 128 Primary Sampling Units BES 2005 INTERNET CAMPAIGN PANEL SURVEY: Wave 2 Campaign survey 275 interviews per day for 30 days N=6068 Wave 4 One Year Out Interview N=6186 Wave 1 Pre-campaign Baseline Survey N=7793 Wave 3 Post-election Interview N=5910 Waves 5, 6, 7 Annual interviews through to 2010

6. Comparisons of Survey Modes • Election Studies are unique in allowing us to compare survey data with actual votes. • Two objective measures are highly visible – turnout and vote choice • If we compare in-person probability surveys with internet surveys what do we see?

7. Turnouts in the 2001 and 2005 BES surveys(RDD in 2001; Internet in 2005)

8. All Types of Survey Over-Report Turnout

9. How Turnout influences response rates

10. Vote Choice in the 2005 BES Surveys

11. What About Modelling the Vote? • This involves a variety of different models reflecting theoretical debates in the literature • One set of variables relates to social backgrounds of respondents – principally their social class • Spatial Issues – issues over which the voters and parties disagree (eg Taxation and spending) • Valence Issues – issues over which there is widespread agreement (eg reducing crime) but differences arise over which party can do the best job. • Partisanship – the ‘Brand Loyalty’ of the parties • Leadership Evaluations – Who do you like and dislike? • Campaign effects

12. Logistic Regressions Panel APanel B Liberal Other Predictor VariablesLabourConservativeDemocratParty Age -.01** .02** .01* .02* Ethnicity -.97*** .43 1.02** 2.89** Gender -.26 -.18 .27 .77** Social Class -.38* .85*** .25 .06 Party Identification: Conservative -1.09*** 1.56*** .14 .52 Labour .91*** -1.72*** -.88*** -.33 Liberal Democrat -1.51*** .25 1.69*** .89* Other Party -.98** .51 .40 2.77*** Party Leader Affect: Blair .41*** -.50*** -.41*** -.38*** Howard -.13*** .58*** -.00 .14* Kennedy -.31*** .04 .49*** -.01 Party Best on Most Important Issue: Conservative -.91*** 1.42*** .08 .79* Labour .80*** -.84** -.72*** -.80* Liberal Democrat -.66* -1.21* .84** -.89 Other Party -.10 -.84 -.32 1.00* Party-Issue Proximities: Conservative -.09*** .20*** .07** .15*** Labour .15*** -.14*** -.12*** -.19*** Liberal Democrat -.12** -.03 .17*** .10 Economic Evaluations .09 -.55*** -.05 .17 Party Best on Economy .94*** -1.28*** -.67*** -.42 Iraq Evaluations .02 .05 -.06 -.09 Emotional Reactions: Economy -.04 .15 .08 -.16 Iraq -.06 .22* .05 .09 NHS .13* -.24** -.07 -.30** Tactical Voting -.34x -.13 .43* .50 Constant 2.53* -2.95 -4.80*** -5.73** McFadden R2 = .59 .60 McKelvey R2 = .78 --- % Correctly Classified = 87.5 81.6 Lambda = .68 .70

13. Factors Affecting the Probability of Voting Labour in the 2005 Election

14. The Performance of Rival Labour Voting Models (Logistic Regressions) with In-Person and Internet Surveys McFadden R2 McKelvey R2 AIC BIC A. Models Estimated Using In-Person Survey Data Social Class .01 .02 2794.20 2805.51 All Demographics .03. .06 2753.54 2810.08 Economic Evaluations .07 .13 2633.38 2644.69 Issue Proximities .12 .22 2507.63 2530.25 Most Important Issue .27 .40 2079.75 2108.02 Party Identification .37 .48 1794.87 1823.14 Leader Images .40 .65 1692.95 1715.56 Composite Model .58 .76 1256.45 1414.76 B. Models Estimated Using Internet Survey Data Social Class .01 .01 6409.16 6422.16 All Demographics .02 .04 6328.65 6400.17 Economic Evaluations .14 .24 5564.96 5577.97 Issue Proximities .19 .34 5229.46 5255.52 Most Important Issue .33 .48 4299.71 4332.29 Party Identification .36 .50 4163.88 4196.45 Leader Images .44 .64 3617.93 3643.94 Composite Model .59 .76 2715.98 2898.40

15. Does Mode Make a Difference?Using the Parameters of the In-Person Model to Predict Labour Voting in the Internet Model in 2005(and Vice Versa)

16. Conclusions • There are differences in frequency distributions of variables measured with in-person and internet surveys • Sometimes the in-person version is more accurate (eg turnout) sometimes not (eg Labour vote choice) • The coefficients of models of turnout and party choice are indistinguishable from each other when models are estimated with in-person and internet surveys

17. Internet Experiment – ‘Feedback to Respondents’ • Key Issue – Are Voter Preferences Exogenous? • If they are then survey respondents should not change their preferences simply because others differ eg. Parties and leaders • If they are not then voters are likely to change their preferences if they discover that others differ

18. Feedback to Respondents • The BES campaign panel survey, N=7793 • Respondents place themselves on proximity scales ‘taxation versus spending’ and ‘crime reduction versus rights of the accused’ • For example: • __________________________________ • Raise taxes and spend more Cut taxes and spend less

19. Examples of FeedbackTreatments • Treatment 1: Control - Respondent’s position only • Treatment 2: Respondent and ‘average voter’ • Treatment 3: Respondent and ‘party supporters’ • Treatment 4: Respondent and ‘party leaders’ • Treatment 5: Respondent and ‘party leaders with party labels’

20. Percentages Wanting to Move

21. Absolute Distances Moved on the Tax and Spend Scale

22. Conclusions • Respondents change their preferences on issues if given feedback • Cues relating to political parties are particularly likely to move them • If preferences are not exogenous then spatial models of party competition are wrong • Much of neo-classical economic theory is wrong too!

23. Summary – Advantages of Internet Surveys Huge N’s Quick & Flexible– not tied to slow-moving in-person data collection Clean administration of surveys – avoiding priming and interviewer effects Highly cost-effective Hi-tech experimentation possible (again cost effective) Panels – study dynamics in multi-wave panels easily