Fun?

1. Fun? • How could it be fun?

2. Selling methods and statistics Methods are good for you! • Research methods and statistics develop transferable skills. Will you ever need to write an essay ….. But report?: • The ‘ah but’ mentality • Always query: • advertisement claims • politicians’ claims • anyone who can’t produce evidence • the nature of any evidence

3. The nature of evidence • ABBA • Did A cause B or could B have caused A?

4. The loneliness of virtual living • 169 participants followed up over three month period after purchasing internet capable computer. Researchers measured internet use and depression. Participants used predominantly social features (e-mail. chat) BUT • (according to Guardian article): • “Researchers found that one hour a week on the Internet led to an average increase of 1% on the depression scale, a loss of 2.7 members of the subject’s social circle, and an increase of 0.4% on the so-called loneliness scale.”

5. The nature of evidence • 2. Big numbers are serious

6. Great news! Twenty-five per cent of maths teachers to go! A recent article writing in rather panicky terms about the shortage of maths teachers in the country made the claim that 25% of maths teachers were due to retire in the next ten years. Is this an exceptional worry?

7. Health service heading for £1billion deficit! • In 2006 the NHS was heading for a £1 billion deficit • This eventually came down to around £800 million which was about 1% of the total budget. • Average predicted deficit for whole of government spending was 2%. • NHS was doing twice as well as other departments. • Most businesses would think spending hitting target within 1% would be sheer magic. • Source: Blastland and Dilnot (2007) The tiger that isn’t (London: Profile Books)

8. The nature of evidence • 3. Mis-use of statistics

9. Shock finding! Scientists find millions of English people have above average number of legs! • Sadly a few have only one or none and even fewer have three (?) so mean is 1.999….. • What about the median? • Should be the mode!

11. The nature of evidence • 4. Problems with significance

13. Sex determination clinic – ‘So far we’ve had 6 couples through and four of these have left satisfied’

14. Svelte Gel by Christian Dior • 550 women given free sample • Asked to use it for a month. • “52% of women reported losing up to one inch from their hips” • “56% reported losing up to one inch from their thighs” • “56% reported losing up to one inch from waist” • How measured? Perhaps take a measure now and another after one month. • What would happen if Svelte was useless (the null hypothesis) • We would expect about half to ‘lose’ and half to ‘gain’ • Also, how many other parts of the body were measured? • And how many samples were used?

15. The nature of evidence 5. Questioning the method – professional ethics Richard Lynn’s estimate of ‘Black African IQ’ at 75 points

16. Some tricks Challenges simple ‘common sense’ Folding paper • Fold paper three times. Note it gets slightly thicker. Suggest 50 more folds if physically possible (Blue Peter did it simply by stacking paper sheets). • How thick would the paper be? • Tall as a house? Telepathy Can use to discuss testing hypotheses Also demonstrates empirical and logical probability Think of a number under 50 Must have two digits Both must be odd and different 11 would not count but 15 would

17. The ropetrick A piece of rope circles the earth. We want to raise it off the surface by I foot. How much more rope would we need? Challenges simple ‘common sense’ ANS: about 6.3 feet Circumference = 2r New circumference = 2 (r+1) = 2r + 2 new circ – old circ = 2r + 2 - 2r = 2 How many people needed in a room to have 50% chance of two having same birthday? Birthdays Anti –common sense and probability values ANS: 23

18. More fun….. • If you do it yourself – use what you already know from experience • …. The heat/aggression hypothesis • If it relates to everyday events and normal thinking • ….. Most statistics and especially significance (later) • (e.g. dispersion and darts) • If it is integrated into theory delivery

19. Practicals, written up, need to be extremely simple (even ‘artificial’) in order to demonstrate very clearly the points being made – e.g. design, analysis • e.g. Pichert and Anderson (1977). What’s the difference? Practicals I’ve found fun: Clock this! Barnum effect Caffeine and dropped ruler as RT measure IQ estimates Stroop Wiggly wire Attribution: self-serving bias and football manager explanations Round things and long things Driver behaviour and gender/type of car/van or car Rating cartoons whilst holding pen in mouth sideways or forwards Nicolson 1981 – faster people talk the better STM they have (quicker articulatory loop) Buss: mate choice and ads Self-concept gap and anxiety (Rogers) Santa’s sack before and after Christmas (schema) Audience effects (e.g. Bopper game)

20. You have a need for other people to like and admire you, and yet you tend to be critical of yourself. While you have some personality weaknesses you are generally able to compensate for them. You have considerable unused capacity that you have not turned to your advantage. Disciplined and self-controlled on the outside, you tend to be worrisome and insecure on the inside. At times you have serious doubts as to whether you have made the right decision or done the right thing. You prefer a certain amount of change and variety and become dissatisfied when hemmed in by restrictions and limitations. You also pride yourself as an independent thinker; and do not accept others' statements without satisfactory proof. But you have found it unwise to be too frank in revealing yourself to others. At times you are extroverted, affable, and sociable, while at other times you are introverted, wary, and reserved. Some of your aspirations tend to be rather unrealistic.

21. Significance and the null hypothesis Because we assume Ho that: N reds = N blacksAND that cards were selected at random Nine reds from a pack of cards – AMAZING! -- WHY? That’s amazing if the pack was shuffled That’s a low p if the null hypothesis is true

22. The null hypothesis is always about populations from which samples were drawn No researcher ever states their null hypothesis A null hypothesis can only be stated once we know exactly which statistical test is to be applied to which specific data A null hypothesis is therefore out of place at the end of an introduction What is stated at the end of an introduction is usually the research prediction (s) A typical null hypothesis: Population mean for imagery condition scores is equal to the population mean for rehearsal condition scores

23. We test the alternative hypothesis • H1: Female reading scores are higher than male reading scores • By making a specific research prediction: • e.g. The female sample will have a higher mean reading score than the male sample • This is not a hypothesis it is a test of one.

24. A concrete example Does the drawer contain 50% right and left hand gloves? You pull out five right hand gloves in succession The null hypothesis H0:fright = fleft Significance? Probability of five right hand gloves is about 0.03 (ignore non-replacement)

25. In every significance test we: • Assume a null hypothesis of no effect • Calculate the probability that our difference would occur if the null hypothesis is true ( ‘under H0’ ) • Declare a significant difference if p is less than 0.05 back

26. Remember this mantra: • What’s the odds of that happening if there’s nothing going on? Evidence that we use significance reasoning in everyday life reasoning “If they’re not seeing each other then how come he knew her number?” “If she’s not mean than how come she’s never first to get a round in?” “Five times out of seven the tripe was underweight and twice it was over. How can that happen if he ain’t a crook?

27. Now let’s try screws Random sample N Random sample S Suspect screws Normal screws Null hypothesis: normal= suspect

28. Back to psychology experiments Random sample C Random sample D Caffeine condition Decaff condition H0: caffeine= decaff

29. Trouble at t’chip shop Psychology chips drawn from Sociology chips drawn from Null hypothesis Sociology chips drawn from Psychology chips drawn from Alternative hypothesis

30. H0 null hypothesis H1 alternative hypothesis p is the probability that the sample difference would occur under the null hypothesis (NOT ‘by chance’) Substitute ‘by chance’ with under H0 We do NOT have ‘The probability that the null hypothesis is true’ ‘The probability that the results occurred by chance’

31. From now on never say: “The null hypothesis is that there will be no (significant) difference between the rehearsal and imagery scores”  This is a prediction from the null hypothesis so it cannot be the null hypothesis The null hypothesis is a statistical claim about the underlying population or populations from which samples have been drawn: “The means of the rehearsal and imagery population scores are equal.”

32. The link with validity Hypothesis is about these: Population 1 (untrained/control) Population 2 (trained/experimental) CONCLUSIONS Sample 1 Sample2  research study test is about these FINDINGS Validity is about how you might have drawn the wrong conclusion from the samples you tested

33. Internal and external validity Did manipulation of IV  really affect  DV? Internal validity External validity Will the effect generalise to: Populations? Locations (‘ecological validity’) Times and contexts?

34. The original technical meaning • Brunswik (e.g. 1947) introduced the term ‘ecological validity’ to psychology as an aspect of his work in perception ‘to indicate the degree of correlation between a proximal (e.g., retinal) cue and the distal (e.g., object) variable to which it is related’ (Hammond, 1998, para 18). This is a very technical use. The proximal stimulus is the information received directly by the senses – for instance two lines of differing lengths on our retinas. The distal stimulus is the nature of that actual object in the environment that we are receiving information from. If we know that the two lines are from two telegraph poles at different distances from us we might interpret the two poles as the same size but one further away than the other. The two lines have ecological validity in so far as we know how to usefully interpret them in an environment that we have learned to interpret in terms of perspective cues. The two lines do not appear to us as having different lengths because we interpret them in the context of other cues that tell us how far away the two poles are. In that context their ecological validity is high in predicting that we are seeing telegraph poles.

35. 2. The external validity meaning Bracht and Glass (1968) defined ecological validity as an aspect of external validity and referred to the degree of generalisation that is possible from results in one specific study setting to other different settings. This has usually had an undertone of comparing the paucity of the experimental environment with the greater complexity of a ‘real’ setting outside the laboratory. In other words people asked how far will the results of this (valid) laboratory experiment generalise to life outside it? Cook and Campbell (1979) also supported this interpretation though they have more recently, and because of the controversy, replaced it with the term ‘external validity with regard to settings’. On this view effects can be said to have demonstrated ecological validity the more they generalise to different settings and this can be established quantitatively by replicating studies in different research contexts. Cook and Campbell, 1979, Quasi-experimentation

36. 3. The ‘pop’ version The pop version is the definition very often taught on basic psychology courses. It takes the view that a study has (high) ecological validity so long as the setting in which it is conducted is ‘realistic’, or the materials used are ‘realistic’, or indeed if the study itself is naturalistic or in a ‘natural’ setting. The idea is that we are likely to find out more about ‘real life’ if the study is in some way close to ‘real life’, begging the question of whether the laboratory is not ‘real life’. Knee-jerk mantra – the more realistic the more ecological validity. There is however no way to gauge the extent of this validity. Teaching students that ecological validity refers to the realism of studies or their materials simply adds a new ‘floating term’ to the psychological glossary that is completely unnecessary since we already have the terminology. The word to use is ‘realism’. As it is, students taught the pop version simply have to learn to substitute ‘realism’ when they see ‘ecological validity’ in an examination question.

37. Milgram or Hofling – which is more ecologically valid? Milgram’s results replicated in many different ways and in many differing environments, not least, in basement of private company rather than in prestigious university. Hofling’s result never completely replicated. Rank and Jacobson (1977) showed how ‘artificial’ Hofling’s study was – when nurses allowed to talk with peers and familiar with drug no obedience effect.

38. Kvavilasvilli and Ellis, 2004, Ecological validity and real-life/laboratory controversy in memory research: A critical and historical review, History of Philosophy and Psychology, 6, 59-80. They argue that a highly artificial and unrealistic experiment can still demonstrate an ecologically valid effect. They cite as an example Ebbinghaus’s memory tasks with nonsense syllables. Hamond (1998) – pop version is unnecessary since Brunswick also talked about representative design.

39. Artificiality of the laboratory experiment?Relationship between theory, controlled research environment and real life settings Form theory about relationship between phenomena Isolate phenomena in controlled environment; obtain reliable effect between phenomena Observe several phenomena in everyday life Support theory and extend/apply theory in real life settings

40. Significance boxes • 1024 numbers from 1 to 11 according to frequencies in the 10 line of Pascals’ triangle – a binomial distribution. • i.e. 1 x 1 10 x 2 45 x 3 120 x 4 and so on. (a job for the kids) • Take samples 5 at a time and 10 at a time to show how distribution of means (‘sampling distribution’) becomes narrower with higher N • Take samples two at a time and record only the difference between means. These should centre round zero. If you discuss the variation among these means, suggest it would be useful to know the standard deviation of the population (‘standard error’) then you are a fraction away from the t test. The distribution you are producing IS the null hypothesis for a t test. • You can also create a box where numbers run from 4 to 14, distributed in the same way. Try taking one sample from each box and comparing the difference between means with distribution of mean differences you just created. This is what a t test does.

41. Easy Mann-Whitney – the original • Select group with higher scores • For each score in that group: • Give 1 point each time beaten by score in the other group • Give ½ point each time there is a tie • U is the addition of all points for that group • Ua + Ub = Na x Nb Na x Nb because each score in a group is compared with each score in the other group. Just like playing darts matches league style with members of two teams.

42. Resources • British Psychological Society • APA • http://www.psychology.heacademy.ac.uk/ • http://davidmlane.com/hyperstat/ - HYPERSTAT • Research Methods Knowledge Base • http://davidmlane.com/hyperstat/ Rice virtual lab in stats • http://www.stattucino.com/berrie/dsl/regression/regression.html play with correlation scatterplot • http://psych.unn.ac.uk/prac/y1pr.html Psychology practicals on the web* • http://psychology.wikicities.com/wiki/Main_Page • Sandy McRae’s disc from British Psychological Society • Murphy’s Law video – BBC Education, Wood Lane, London • Psychology Review • The Psychologist • McIlveen BPS Manual of practicals

43. Qualitative sources • Braun & Clarke (2006): A definitive article describing thematic analysis and giving a step by step methodological guide. Qualitative Research in Psychology, 3, 77-101 • Flick (2008): This is the new practical hands-on volume from Flick, a leading authority on qualitative research, and takes the student through all the design, data collection and analysis stages of a project. • Hahn (2008)A very useful text that shows how qualitative coding can be carried out using Microsoft Word, Excel and Access • Willig (2008):A comprehensive text taking you through the way to carry out qualitative research under several approaches (IPA, GT, DA — discursive and Foucaldian), memory work, plus a discussion of quality and many research examples. • Smith (2008):This text is also hands-on, covering much the same ground as Willig (2008), but includes narrative analysis, CA, focus groups and cooperative enquiry. Perhaps more practical and less theoretical than Willig.

44. Qualitative sources • Online QDA – Online information about many different qualitative analysis approaches. Don’t worry about the title in the address this is all methods (27 of them in all) with links to further reading. • http://onlineqda.hud.ac.uk/methodologies.php#Interpretive_Phenomenological_Analysis • The content analysis Guidebook – is what the title suggests. A comprehensive guide to content Analysis with many links. • http://academic.csuohio.edu/kneuendorf/content The Psychology Network at the Higher Education Academy whose web address is http://www.heacademy.ac.uk/ - select the Psychology network under ‘Subject Centres’ or use this link if it still works: http://www.psychology.heacademy.ac.uk/ .