Understanding Research Methodologies for Effective Data Collection

1. Types of Methodologies

2. Qualitative • Interview • Unstructured • Semi-strutured • Structured

3. Qualitative • Direct observation • Observing & recording actual behavior, rather than reported or recalled behavior. • Observations may focus on an individual, a location, event • Observer records as much behavior as possible, including actions, conversations, and descriptions of the locale and persons observed.

4. Qualitative • Pile sorting • Allowing informants to group together items according to their own system of categorization. • Informantsare asked to sort cards on which relevant items are written, drawn or attached. • They are then asked to explain the basis on which they sorted the cards.

5. Qualitative • Case Study • In-depth analysis of a single entity • Can be quantitative as well • Can do a collective/multiple/multi-site case study

6. Qualitative • Photo elicitation • Interviews using photos & video • Homeless leisure • Sense of community • Focus Group

7. Qual/Quan Mix • Nominal Group Technique • Delphi Technique • Q-Sort • Used for subjective areas of study • See example

8. Quantitative • Survey instruments • New instrument • Pilot test to establish validity & reliability • Existing instrument • Permission • V & R established • Modified instrument • Permission & possibly pilot test/panel of experts • Need to establish V & R

9. Types of Survey • Questionnaire • Paper & pencil • Electronic • Interview • Answer questions orally

10. Process • Determine the objectives • Delimit the sample • Construct the questionnaire • Conduct the pilot study (if needed) • Establish the distribution process • Collect & analyze the data

11. 1. Determine the Objectives • No plan, weak results • Plan the entire survey • Every objective needs 1+ question(s)….

12. 1. Determine the Objectives • To determine the quality of the graduate admissions process

13. 1. Determine the Objectives • To determine the quality of academic advising.

14. 1. Determine the Objectives • To evaluate student financial motivations

15. 2. Delimit the Sample • Who is the sample? • How do you get to them? • What is the sample size?

16. 3. Construct Questionnaire • Writing questions • Types of questions • Structure • Layout & design • Scales of measurement

17. 3. Construct Questionnaire Types of Questions (5) • Experience/behavior questions • Opinions/values/attitudes questions • Feeling questions • Knowledge questions • Demographics questions

18. 3. Construct Questionnaire Types of Questions (1 of 5) • Experience/behavior questions • What a person does/has done • How many times have you ridden your bicycle to school in the last month? • What intramural programs are you currently registered for thru campus rec? • What ISU athletic events will you attend this week? Past Present Future

19. 3. Construct Questionnaire Types of Questions (2 of 5) • Opinions/values/attitudes questions • Cognitive & interpretive processes held by an individual • What you think about something • Needs & interests

20. 3. Construct Questionnaire Types of Questions (2 of 5) • Opinions/values/attitudes questions • What did you like most about the trip taken through Adventure Connection? • When you call the ROE, are the receptionists usually very helpful, helpful, somewhat helpful, or not helpful? • Would you support the addition of a practice facility to Redbird Arena? Past Present Future

21. 3. Construct Questionnaire Types of Questions (3 of 5) • Feeling questions • Emotional responses to experiences • How do you feel about the quality of programming at Braden Auditorium? • How responsive do you feel the staff is to your needs at Student Health Services? Past, present…no future

22. 3. Construct Questionnaire Types of Questions (4 of 5) • Knowledge questions • Factual information participant has • What is your resting heartbeat during Zumbaclass? • What are the 5 types of canoe strokes? Past, present…no future

23. 3. Construct Questionnaire Types of Questions (5 of 5) • Demographics questions • Personal characteristics • Age, gender, race/ethnicity, geographic location, income, # of children • What is your age? q under 25 years q 25-40 q 41-55 q over 55 Past, present…no future

24. 3. Construct Questionnaire Question Structure

25. 3. Construct Questionnaire Question Structures • Open ended • Closed ended • Unordered closed ended • Ordered forced choice • Partially closed ended

26. 3. Construct Questionnaire Open Ended • Respondent supplies the answer • Used when variety of info is needed • Used to collect qualitative data

27. 3. Construct Questionnaire Open Ended • How could the ISU Homecoming parade be made more accessible for people with disabilities? • What 3 services do you feel IAPHERD should offer to its members? • What services provided by the Athletic Study Center are most valuable?

28. 3. Construct Questionnaire Open Ended • Did you enjoy playing in the soccer league? • Poorly written O.E. question • Revision: • What did you enjoy most about the soccer league?

29. 3. Construct Questionnaire Open Ended - Advantages • Advantages • Richer information • Answers less influenced by the evaluator • Questions easy to write

30. 3. Construct Questionnaire Open Ended - Disadvantages • Disadvantages • Lack of uniform responses • Difficult to analyze • Takes longer for respondent to complete • People don’t like them

31. 3. Construct Questionnaire Closed ended • Specific responses are provided • Overall, how satisfied were you with the MCH renovations? • Very satisfied • Satisfied • Dissatisfied • Very dissatisfied

32. 3. Construct Questionnaire Closed ended • Issues with these questions • Responses should never overlap • Less than 20,000 • 20,000-40,000 • 40,001 + • List all possible responses

33. 3. Construct Questionnaire Closed ended • Advantages • Easily coded and tallied • Easy to administer • Easy to analyze • Disadvantages • Sometimes difficult to provide all answers • Makes them invalid • No opportunity to explain answers

34. 3. Construct Questionnaire Closed ended • 2 types of closed ended • Unordered closed ended responses • Ordered forced choice • Likert Scales • Semantic differentials • Ranking • Self-Assessments

35. 3. Construct Questionnaire Closed ended • Unordered closed ended responses • Items have no particular value associated with them • What activities have you participated in within the last 6 months thru Campus Rec? __ Aerobics __ Body Pump __ Kickboxing __ Personal Trainer

36. 3. Construct Questionnaire Closed ended (forced choice) • Ordered forced choice • Ranks responses • Scales of measure • Likert scales….

37. 3. Construct Questionnaire Closed ended (Likert) Strongly Agree Agree No Opinion Disagree Strongly Disagree The staff was courteous The staff was knowledgeable I am satisfied with the service I received The facility was clean • 4 3 2 1 • 5 4 3 2 1 • 5 4 3 2 1 • 5 4 3 2 1

38. 3. Construct Questionnaire Closed ended (forced choice) • Ordered forced choice • Likertscales • Uses balanced “ + “ & “ – “ responses • 4-6 responses • See website • Even vs. odd # of responses • Grid/matrix like format • Easy to score

39. 3. Construct Questionnaire

40. 3. Construct Questionnaire Closed ended (forced choice) • Ordered forced choice • Semantic differentials • Polar responses • Used for attitudes & meanings of things • Vary placement of positive & negative statements along 1 side of page • Often 7 point response scale

41. 3. Construct Questionnaire Semantic Differential Please indicate your perception of the leadership ability of your coach. Circle where you believe she/he falls on the scale. Dedicate 7 6 5 4 3 2 1 Undedicated Strict 7 6 5 4 3 2 1 Lenient Organized 7 6 5 4 3 2 1 Disorganized + - +

42. 3. Construct Questionnaire Would you say our web site is: Very Attractive Very Unattractive 7 6 5 4 3 2 1 0

43. 3. Construct Questionnaire Closed ended (forced choice) • Ordered forced choice • Semantic differentials • Weakness: • Adults need strong understanding of the language • Not good for children • Strengths • Show strength of feelings • Easy to write & analyze

44. 3. Construct Questionnaire Closed ended (forced choice) • Ordered forced choice • Rankings • Assess relative meaning of things • Use less common statistics (can’t average) • Spearman Rank Order Correlation • Relationship to other rankings • Too many items can be confusing • 6-8 work • Need to give very clear directions • Ties; High vs. Low

45. 3. Construct Questionnaire Rank Order Rank the following 5 activities from what you like to do the most to what you like to do the least. Your favorite activity would receive a “1” and your least favorite activity would receive a “5”. Please do not give more than 1 item the same number. _____ Swimming _____ Hiking _____ Biking _____ Running _____ Walking

46. Rank Order • Strengths • Each item has a unique value • Weaknesses • Force respondents to differentiate between items that they may regard as equivalent • Earlier in the list are more likely to be ranked highest • Takes 3 times longer to answer

47. Rank Order vs Rating • Rating question asks to compare different items using a common scale • Ranking question asks to compare different items directly to one another • Rating questions higher R & V • Use ranking to set priorities

48. 3. Construct Questionnaire Closed ended (forced choice) • Ordered forced choice • Self Assessments • Describe where you fall on a scale • Based on a 1-5 scale with 1 being low and 5 being high, after taking this course, how do you rate your knowledge of survey design?

49. 3. Construct Questionnaire Partially Closed Ended • Some responses with room for comments • “Other” • Develop as many responses as possible to minimize “other” responses…

50. 3. Construct Questionnaire