Questionnaires

1. Questionnaires

2. Bike with no hands - Helena Nelson

3. Today • A check on the way we think about the world • Quantitative vs Qualitative Research • The Carbon Footprint questionnaire • Questionnaires • OAP questionnaire and preparation • Tutorial on lit review assignment

7. Five assumptions1. The Ontological Assumption Reality is like the hardware of a computer programme What is real and knowable depends on the software What is the nature of reality? What is real and knowable?

8. Qualitative vs Quantitative AssumptionQuestion Ontological What is the nature Assumption of reality? Quantitative Qualitative Reality is objective Reality is subjective and singular, and multiple, apart from the as seen by participants researcher in a study

9. 2. Epistemological assumption The outstanding researcher is all powerful In order to know I must train myself to win “I want to be a Millionaire” What is the relationship of the researcher to that researched? What we must do in order to know?

10. AssumptionQuestion Epistemological What is the relationship assumption of the researcher to that researched? Quantitative Qualitative Researcher is Researcher interacts independent from that with that being being researched researched

11. 3. Axiological Assumption What is the role of values? What is worth knowing and why? Survival values – nature red in tooth and claw Social Values – fitting in with society Transcendent values – beyond self, for the greater good Our values determine what we want to research

12. AssumptionQuestion Axiological Assumption What is the role of values? Quantitative Qualitative Value-free and Value-laden and unbiased biased

13. Rhetorical Assumption What is the language of research? How best to share our research with others. Dramatic Appealing to the personal Stretching the truth Large group of believers present

14. AssumptionQuestion Rhetorical What is the language assumption of research? Quantitative Qualitative Formal Informal Based on set definitions Evolving decisions Impersonal voice Personal voice Use of accepted Use of accepted quantitative words qualitative words

15. Method-ological assumption Research is complex There is no single approach Before you use any particular method you have to understand the assumptions behind it. Methodology deals with the assumptions behind specific methods of research What is the process of research? What techniques will lead to fruitful inquiries?

16. AssumptionQuestion Methodological What is the process of assumption research? Quantitative Qualitative Deductive process Inductive process Cause and effect Mutual simultaneous shaping of factors Static design, categories Emerging design, categories identified before study isolated during research Context-free Context-bound Generalisations leading to Patterns, theories prediction and explanation developed for understanding Accurate and reliable Accurate and reliable through validity and through verification reliability

17. Qualitative Methods descriptive, theory building, meaning making • Focus Groups • Interviews • Case Studies • Ethnographic Inquiry • Collaborative Inquiry • Grounded Theory • Qualitative analysis of documents, video, observation, etc What are they?

18. Quantitative Methods measuring, theory confirmation, defining • Experiments • Questionnaires and surveys • Psychometric Scaling and Tests • Quantitative analysis of documents, video, observation, etc • Audits What are they?

19. Mixed Methods • Research that uses both qualitative and quantitative methods – e.g. Interviews and questionnaires. Gets information from two very different perspectives. If the data from both sources is in agreement, the research is more convincing. • Multi-method: research that uses two quantitative or qualitative methods – e.g. interviews and focus groups OR structured observations and questionnaires. Multi Methods has many of the same benefits as Mixed Methods

20. Positivism Metaphysical (nature of reality) assumptions • Nature is orderly and regular (measurable); • We can know nature. (Some theorists suggest that there exists a limit to such knowledge. Up to now, such a limit has not been defined.) • All natural phenomena have natural causes (Determinism). • Nothing is self-evident

21. Positivism Epistemic (nature of knowledge) assumptions • Knowledge should only be derived from experience. (Empiricism) • The meaning of a proposition consists in how it is verified by experience. (verifiability). • The application of logical analysis will reach the goal of unified science. (Logicism). • Sciences should all be unified syntactically and semantically.

22. Interpretivism Interpretivism rests upon idealism. Idealism holds the view that the world is the creation of mind; the world is interpreted through the mind; e.g., classificatory schemes. Given this, we cannot know the ‘true’ nature of the object world, separate from our perception of it. Research methods focus on phenomenological enquiry – interviews and focus groups that discover the lived experience of the participant.

23. Constructivism Constructivists maintain that scientific knowledge is constructed by scientists and not discovered from the world. Constructivism believes that there is no single valid methodology and that qualitative research is a valid methodology for social science. Constructivism criticizes objectivism (objects and facts exist independently of the mind), and holds that the only reality we can know is that which is represented by human thought

24. Structuralism & Poststructuralism In Structuralism the meaning of any one word is determined by its relationship with other words. Any approach that sees the meaning of something as subordinate to its place within a system is structuralism. It promotes the objectification of structures and systems. Systems analysis is a structuralist approach. Poststructuralists reject structuralism and maintain that meanings and intellectual categories are shifting and unstable. Deconstruction and Narrative enquiry are poststructural research methods

25. Postpositivism Acknowledges the constructivist, Interpretivist and post structural critiques of positivism and seeks to incorporate qualitative approaches into research methodology. It retains a place for positivist methodology and analysis. It mediates critical space between positivism and constructivism based upon the quest for a stable truth as a regulative ideal within a fallible scientific epistemology.

26. Transformative/Emancipatory Has a focus on social justice, the experience of oppression, the differentials of power, and the cultural, political, economic and historical perceptions of “reality”. It builds on Foucauldian ideas of ethics and asks for a constant effort to move taken-for-granted knowledge to conscious examination. It does accept that using quantitative approaches can be consistent with the central premises of the T-E paradigm

27. Pragmatism • The current meaning of an expression is to be determined by practical consequences of belief in or use of the expression in the world • We should consider what practical effects we conceive the object of our conception to have. Then our conception of these effects is the whole of our conception of the object • Places high regard for the reality of and influence of the inner world of human experience in action. • Knowledge is viewed as being both constructed and based on the reality of the world we experience and live in.

29. Questionnaires and Surveys 1)      Surveys are Structured interviews. Face to face, or telephone/video/internet. A prescribed set of questions, mostly tick-box/numerical, some open ended questions, some opportunity for asking additional questions – Deaf Epidemiological Survey 2)      Questionnaires. Mail out, internet or drop offs, or groups of people. Often anonymous. Tick/ numerical, some open questions – Carbon Footprint Questionnaire 3)      Audits: These are the analysis of data already in the system. No new questions are created – Analysis of Mental Health Data

30. General Health Questionnaire-12 INSTRUCTIONS We should like to know if you have any medical complaints, and how your health has been in general, over the past few weeks….. We want to know about present and recent complaints, not those you had in the past. • Able to concentrate • Lost much sleep • Playing a useful part • Capable of making decisions • Under stress • Could not overcome difficulties • Enjoy normal activities • Can face up to problems • Feeling unhappy and depressed • Losing confidence • Thinking of self as worthless • Feeling reasonably happy

31. Principles of Psychometric Tests • Three important concepts: • reliability, validity and standardisation are essential criteria for a good psychometric test • Test standardisation –ensures that the conditions are as similar as possible for all individuals who are given the test. • Standardisation also ensures that no matter who gives the test and scores it, the results should be the same – i.e. the test is reliable

32. Test Reliability • Test Reliability–a test must measure the same thing in the same way every time someone takes it • There are two types of test reliability • Internal consistency reliability – all the parts of your test questionnaire are reliable throughout (Split half tests using Cronbach’s alpha (a)) • Test–retest reliability–the test remains reliable over time (e.g. gives the same result under the same conditions). This is tested by correlation (r) • A test can be reliable, but not valid

33. Test Validity • There are four types of test validity: • Face validity: does your test appear to measure what it purports to measure. Would experts agree that people who score above 3 on the GHQ would will have MI signs • Concurrent validity: does the GHQ correlate with existing standardised tests of MI? • Predictive validity: do the results of your test predict future behaviour – good and poor MH outcomes • Construct validity: if all our hypotheses about the test variable (construct) are supported then we have a high degree of construct validity

34. Carbon footprint questionnaire • Do the questionnaire • Tick the answers closest to your understanding of the way you live • Add up your score by adding up the numbers in each box you ticked • Discuss the outcome • Critique the questionnaire

35. Question Issues • What types of questions can be asked and how? • Demographic vs topic • How complex will the questions be? • How long will the questionnaire/ survey be?

36. Demographic questions The big three: age, gender, ethnicity/culture. Socio-economic status: • Income (tax brackets/income support levels), • Occupation 1=factory, unskilled; 2=clerical, fishing, farming, trades; 3=administrator, professional, technician

37. Demographic questions - 2 Socio-economic status (continued): • Employment (full-time, part-time, student, not working, on the benefit), • Education (SC, UE/bursary, tertiary cert, diploma, degree) • Family: partners, dependants, responsibilities • Health: disabling conditions Beliefs: religion, political affiliation, sexual orientation

38. Developing your questions - 1 Managing bias. • The researchers – how easy is it to disprove your hypothesis, theory, bias? • The participants – will the subject and the questions put off the participant? Opening questions. • Simple • Key open ended questions to get uninfluenced “off the top of my head” answers.

39. Developing your questions - 2 Response option lists.(forced choice)

40. Developing your questions - 3 Rating questions (likert scales) Did your mental health change because of the help you got from the hospital or the service(s)? 1=much better; 2=better; 3=didn’t change/not sure; 4=worse; 5=much worse • What to do about don’t know/not sure/did not answer responses • Rating option lists – putting in an “other” option space • Question Matrices

41. Developing your questions - 3 The “other” option

43. Developing your questions - 4 • “Why” questions sparingly used after key options questions • Mopping up questions. Solutions, answers, ideas, “anything else you want to say” • Reliability checks. • Check response patterns for rigidity • Vary direction of rating responses • Set up correlating questions • Set up some very unlikely options

44. Strengths of Questionnaires • Good for measuring attitudes and eliciting other content from research participants • Inexpensive (especially mail questionnaires and group-administered questionnaires) • Can be administered to large probability samples • Quick turnaround from data collection to report • Can be administered to groups • Perceived anonymity by respondents is high • Moderately high measurement validity for well-constructed and well-tested questionnaires • Low dross rate for closed-ended questionnaires • Ease of data analysis for closed-ended items

46. Weaknessesof Questionnaires • May need validation and may have poor reliability • Must be kept short • Often has missing data, particularly to open ended questions • Possible reactive effects (e.g. response sets, social desirability, dislike of questionnaires) • Failure to reach certain groups – low SES, low literacy • Response rates generally low for mail questionnaires • Open-ended answers are vague or reflect differences in verbal ability, obscuring the issues of interest • Data analysis sometimes time-consuming for open-ended items

48. Sampling

49. Population issues • Can the population be counted? What data is available • Are response rates likely to be a problem? • Is the population literate? • Are there language issues? • Will the population cooperate? • What are the geographical restrictions? • Generalisation. • How small is the effect size we are seeking? • What sub-groups do we want to compare? • How do we represent the people who won’t answer?

50. Probability sampling • Based on statistics which can describe the similarity of a sample to the population that it is supposed represent. • The simplest form of random sampling is called simple random sampling. In this weselect participants from a given population such that each person in the population has an equal chance of being selected.