This is a teacher caught dozing whilst planning a lesson on reliability and validity

1. It’s my favourite part of the syllabus…honest. Would a teacher lie to you? I can reliably predict that you’re all yawning already This is a teacher caught dozing whilst planning a lesson on reliability and validity

2. Y Right, own up. Who wrote that? Detention for a week! Yes, we know it seems boring... Anything...I’ll do anything but reliability and validity. Detention for a year would be lovely or clearing out your cat’s litter tray Reliability and validity suck!

3. …but reliability and validity underpin everything that we do as scientists – without them our research would be worthless! Sorry, not convinced yet, and I had a very late night, so I have a valid reason for falling asleep in this lesson...

4. Did someone say they wanted to talk about salad?

5. Validity Valid, not salad!!!!!! We need to find out if our research is sound. Do our tests measure what they claim to measure?

6. Validity Can we trust any effect that has been found to be the result of manipulating our independent variable and not from another unwanted variable? Are techniques used to collect data in tests, questionnaires, interviews and observations measuring what is claimed? If a measurement is not reliable, then our research cannot be valid or ‘true’

7. Reliability We could just measure something using a piece of string, but it wouldn’t be very accurate

8. Reliability We need to be able to measure or observe something time after time and produce the same or similar results

9. I want to measure intelligence. If the same person sits the test on several occasions and the results change each time, then that test lacks reliability

10. The test also lacks validity because the scores are meaningless

11. If I test my participants again several months later and their scores remains consistent, I can say the test is reliable, but it might still lack validity. The second score might just be measuring what a person has learned since taking the first test.

12. Does a driving test measure your competence to drive on the road or is it a measure of your ability to pass the driving test? Would you be able to pass it again in six months time? Would you do better? Is it a reliable and valid test? L

13. External Reliability This measures consistency from one occasion to another – the same result should be found on different days, in different labs , observations or interviews, by different researchers I exposed these teenage brain cells to 1000 PowerPoint slides last Monday and they’re all dead I thought that was a fluke but they seem to be shrivelling after only five minutes!

14. Improving External Reliability Test-retest Participants take the same test on different occasions – a high correlation between test scores indicates the test has good external reliability . Timing is crucial. Why? I hope that’s the right answer this time January June

15. Internal Reliability This measures the extent to which a test or procedure is consistent within itself, i.e., questionnaire items or questions in an interview should all be measuring the same thing Do you like to keep to deadlines? Do you get impatient driving? Do you like cheese? Do you like doing several tasks at once? Do you like chocolate? Do you get easily irritated? Are you competitive? This interviewer seems a little confused about Type A personality traits

16. Improving Internal Reliability Odds/Evens Top/Bottom Split-half Compares a participant’s performance on two halves of a test or questionnaire – there should be a close correlation between scores on both halves of the test. Questions in both halves should be of equal quality for good internal reliability.

17. Improving Researcher Reliability Intra-rater reliability This refers to the consistency of a researcher’s behaviour. A researcher should produce similar test results, or make similar observations or carry out interviews in the same way on more than one occasion. How much longer before I can get in the pub and relax my facial muscles? Thanks for taking part today. Any problems and I’ll be right over. Take your time. Right. Let’s get on. Fast as you can.

18. Improving Researcher Reliability Inter-rater reliability Consistency between different researchers working on the some study is very important for reliability In observational studies this is known as inter-observer reliability – observers have to agree on what they see and carry out the same procedure There should be a high positive correlation between the scores of different observers

19. Today’s training session for researchers: Increase reliability by standardising instructions Carry out a pilot study to improve procedures and materials You will be thoroughly trained in the use of materials and procedures prior to our study taking place

20. Would you see this as bullying or horseplay in the playground? You would see this from your own subjective viewpoint – we’re biased by experience and expectation Observers must agree about what they are observing – they need to use standardised behavioural categories

21. Internal Validity = The tool is measuring what it is intending to measure External Validity = The findings can be generalized beyond the context of the research situation

22. Assessing and Improving Internal Validity Face Validity One or more judges assess whether the test seems appropriate and suggest changes if necessary Does our measuring tool appear to be doing what it should?

23. Content Validity Does the content of a test cover everything in the area of interest? More rigorous – experts in the field systematically examine the tool’s components and compare them with set standards They have to agree the content is appropriate You expect me to read through this lot by Friday?

24. Concurrent Validity New measure test scores are correlated with those from an established valid test As you can see, we have a high positive correlation between scores on the new and old tests. I declare this test valid! What would we do if the correlation were low?

25. Predictive Validity Can an intelligence test at age 3 predict academic performance at 21? Can a diagnosis of a certain mental illness predict recovery?

26. Temporal Validity Do our findings endure over time or are they era-dependent? Oh, Mr Asch, we’d be delighted to say those lines are the same size! ‘Ere mate, are you having a laugh? Do I look that stupid? Assessing and Improving External Validity

27. Population Validity Can we generalise findings from our research participants to other population groups?

28. Context Validity Better known as: Can we apply our findings to other contexts and situations outside of the research setting? Ecological Validity You want me to pretend to do sums as well as talk? I’ll do it for two bananas You want me to falsify your accounts? I’ll do it for £200,000

29. Now that really wasn’t so bad. Some of you are still awake! OK, we get it now. If our studies are unreliable and invalid we may as well not bother doing research at all I can reliably predict that if you learn this you will score higher in your valid A2 exam I can reliably predict I’m going to ask her out and that I have a perfectly valid reason for it