1 / 94

An Introduction to Research Methods: Needs Assessment, Surveys, Focus Groups and Personas

20 November 2013. An Introduction to Research Methods: Needs Assessment, Surveys, Focus Groups and Personas. Prepared for the ULS Leadership Program by Luke Ferdinand, John Fudrow , Karen Calhoun and Jeff Wisniewski . This work is licensed under a

kemal
Télécharger la présentation

An Introduction to Research Methods: Needs Assessment, Surveys, Focus Groups and Personas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 20 November 2013 An Introduction to Research Methods: Needs Assessment, Surveys, Focus Groups and Personas Prepared for the ULS Leadership Program by Luke Ferdinand, John Fudrow, Karen Calhoun and Jeff Wisniewski This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

  2. Needs Assessment Defining and Addressing The Needs of an Organization and Its Audience Luke Ferdinand ULS Leadership Program 20 November 2013

  3. Objectives • Define Needs in Organization • Define Needs Assessment • Understand the methods and tools used in Needs Assessment • Know potential risks in Needs Assessment methods ULS Research Methods Workshop

  4. What is a Needs Assessment? "A systematic process for collecting information and making justifiable decisions" - Ryan Watkins Ryan Watkins, 2013, “Needs = Gaps in Results”, http://ryanrwatkins.com/na/naintroduction.html, ULS Research Methods Workshop

  5. What is a need? • “Needs are the differences between your current achievements and your desired accomplishments.” Ryan Watkins, 2013, “Needs = Gaps in Results”, http://ryanrwatkins.com/na/naintroduction.html, ULS Research Methods Workshop

  6. Needs Assessments Help Identify • Data that define your needs • Prioritization of your needs • Criteria for implementing solutions • Information necessary to justify selection of one or more activities to improve performance Ryan Watkins, 2013, “Needs = Gaps in Results”,http://ryanrwatkins.com/na/naessentials.html ULS Research Methods Workshop

  7. Why? • “Provide a systematic process to guide decision-making in organizations • Provide justification for decisions before they are made. • Scalable for any size project, time-frame, or budget. • Offer a replicable model that can be applied by novices or experts. • Provide a systemic perspective for decision-makers. • Allow for interdisciplinary solutions for complex problems. “ Ryan Watkins, 2013, Needs = Gaps in Results”, http://ryanrwatkins.com/na/naessentials.html, ULS Research Methods Workshop

  8. Where can we use a needs assessment? Strategic Action 9a : Ground Floor Needs Assessment • (obviously) • Opportunity to rethink the ground floor • Many assumptions and ideas for what's best • Gather information from stakeholders, relevant resources • Make an informed decision to best meet the needs of organization and community • Other opportunities? ULS Research Methods Workshop

  9. 12 Steps Brought to you by the National Oceanic & Atmospheric Administration http://csc.noaa.gov/needs/ ULS Research Methods Workshop

  10. Needs Assessment Steps • Confirm the Issues & Audience • Establish the Planning Team • Establish the Goals and Objectives • Characterize Your Audience • Conduct Information & Literature Search • Select Your Data Collection methods • Determine Your Sampling Scheme • Design and Pilot the Collection Instrument • Gather & Report Data • Analyze Data • Manage Data • Synthesize Data & Create Report ULS Research Methods Workshop

  11. 1: Confirm Issues & Audience • Summary: • Establish purpose of assessment and summarize • Questions: • Is this a new issue or audience for the organization? • Is there agreement up the organizational chain this issue or audience needs to be addressed? • Risks: • Unknown stakeholders • Communication issues • LACK OF SUPPORT ULS Research Methods Workshop

  12. 2. Establish the Planning Team • All steps should be conducted with planning team • Ideally members of stakeholder groups and experts • Communication is essential • Consider resource needs: time, expertise etc. • Questions: • Are the stakeholders new or well known? • How will geography impact planning? • Expertise within the team? • Risks • Sense of ownership • Communication • Planning can overtake action ULS Research Methods Workshop

  13. 3. Establish Goals and Objectives • Summary • Identify the desired and actual levels of knowledge or skill • Identify cause(s) for the lack of knowledge or skill • Devise solution(s) • Questions: • Are the goals widely shared by the audience? • Are your objectives measurable? • Will the project be considered a success if the objectives are met? • Risks: • Potential to disengage by different team members who have different priorities • Not knowing if goals have been achieved and to what degree • Long term-support for doing needs assessments ULS Research Methods Workshop

  14. Play Time

  15. 4. Characterize Your Audience • Summary • Sample size • Skill & knowledge level • Educational Level • Attitudes and biases • Questions: • How long have you worked with the audience? • How much variation is there within the audience? • Risks: • Assumptions can backfire • Broadly generalizing and audience can lead to failure ULS Research Methods Workshop

  16. Topics for Audience Characterization • Knowledge • Training • Tools & Techniques • Benefits • Attitudes & Biases • Ability to Attend or Access • Cultural Characteristics ULS Research Methods Workshop

  17. 5. Information & Literature Search • Summary • Environmental Scan • Readings • Surveys to address questions raised in review • Questions: • Has this audience or issue been surveyed in the past? • What other ways might we find useful information about this audience or issue? • Risks: • Results may be outdated • Redundant effort, wasted time • Missing important information that may make results easier or better ULS Research Methods Workshop

  18. 6. Select Your Data Collection methods • Summary • Observation • Personal Interviews • Surveys • Focus Groups • Questions: • Have all methods been considered? • Consider audience characteristics when drafting collection methods • How much expertise is there in-house? Seek help! • Risks: • Can be intrusive or upsetting • Time spent designing vs time spent analyzing ULS Research Methods Workshop

  19. 7. Determine Your Sampling Scheme • Summary • Sample more than you think you need • Questions: • What is statistically recommended? • What is the population size of audience? • Risks: • Likely not scientifically sound • Too many or too few in sample ULS Research Methods Workshop

  20. Exercise: Develop Your Goals Discuss how you would approach Audience Characterization Data Collection Methods Sample Size Measure of Success ULS Research Methods Workshop

  21. 8. Design and Pilot Your Collection Instrument • Summary • Pilot first to identify weaknesses • Questions: • How will you pilot your instrument? • What kind of expertise is on your planning team? • How important is statistical precision? • How will data collection be standardized? • Risks: • Instrument will not be clear or gather necessary data • The audience will not be receptive to the survey instrument • Asking too many questions may irritate the respondents ULS Research Methods Workshop

  22. 9. Gather and Record Data • Summary • Find means to incentivize respondents (survey prizes, etc) • Ensure anonymity if this is a priority to audience • Coordinate with assessment team • If audience is outside of library, make use of External Communications group • Questions: • Will you recruit people outside planning team to assist in data gathering? • Are you getting the desired response rate? • Risks: • Data will be biased • Language or vocabulary issues • Invalid Study ULS Research Methods Workshop

  23. 10. Analyze Data • Summary • Keep findings and interpretation of findings separate in reporting • Questions: • Seek assistance in interpreting data if necessary • Risks: • Under or over-whelming audience with report • Missing trends and patterns • Not accounting for possible critical barriers • Letting bias slip into the process ULS Research Methods Workshop

  24. 11. Manage Data • Summary • This step involves determining how data will be organized and archived. • The importance of this step is often not recognized until it is too late. • Questions: • Will this data ever need to be referred to again? • Will the raw data be stored? How will it be stored? • Can/should the data be shared? • Risks: • There may be unforeseen reasons that would necessitate the data being used again • Inadequate metadata ULS Research Methods Workshop

  25. 12. Synthesize Data and Create Report • Always address your goals and objectives in synthesis. Report must include problems or errors with the design and the implementation of the survey. An executive summary is often helpful. • Questions: • Who is the audience for the report? • Did you address your objectives? • Risks: • Potential to disengage by different team members who have different priorities • Not knowing if goals have been achieved and to what degree • Long term-support for doing needs assessments ULS Research Methods Workshop

  26. Gather your Data/Report ULS Research Methods Workshop

  27. Thank you! Works Cited: http://www.cscnoaa.gov/needs http://ryanrwatkins.com/na (includes link to free NA eBook) Additional Reading: http://en.wikipedia.org/wiki/Needs_assessment ULS Research Methods Workshop

  28. Survey Basics and Survey Monkey John Fudrow ULS Leadership Program 20 November 2013

  29. When to Use a Survey • Explore a Topic • Discussion with Target Population • Gather Objective Data on Subject • Benchmarking of Service Levels ULS Research Methods Workshop

  30. Survey Monkey Templates • Over 150 pre-made templates • Variety of topics • Community (2) • Customer Feedback (7) • Demographics (9) • Education (20) • Events (7) • Healthcare (14) • Human Resources (18) • Industry Specific (58) • Just for Fun (9) • Market Research (12) • Non-profit (5) • Political (11) ULS Research Methods Workshop

  31. Surveys We Have Applied • ULS General Survey • Library Instruction Survey • E-book Survey • Event Feedback Surveys • Internal Planning ULS Research Methods Workshop

  32. Types of Survey Questions • Open-Ended • Closed-Ended ULS Research Methods Workshop

  33. Types of Survey Questions • Open-Ended • i. Single Textbox • ii. Multiple Textboxes • iii. Comment/Essay Box • iv. Numerical Textboxes • v. Demographic (U.S. or International) • vi. Date and/or Time ULS Research Methods Workshop

  34. Types of Survey Questions • Close-Ended • i. Multiple Choice (Only One Answer Allowed) • ii. Multiple Choice (Multiple Answers Allowed) • iii. Rating • iv. Ranking • v. Matrix of Choices (Only One Answer Per Row) • vi. Matrix of Choices (Multiple Answers Per Row) • vii. Matrix of Drop-down menus ULS Research Methods Workshop

  35. Likert Scales and Ratings • What is a Likert Scale? • A scaled response of a respondents feelings toward a topic based on a presented scale. ULS Research Methods Workshop

  36. Likert Scales and Ratings • Clear wording of response choices. • Number of choices to relevancy • 5 to 7 choices allow for a consistent distribution. • The third choice should be the undecided or neutral decision. • Be careful not to force ranking by the wording of the base question. ULS Research Methods Workshop

  37. Anonymity • IRB Authorization • Service Improvement • Anonymous Ratings • Do not store data publically ULS Research Methods Workshop

  38. Survey Delivery • Target your audience • Multiple avenues for large samples • Create Multiple links to analyze effectiveness of delivery methods • DON’T SPAM • You cannot mass email ULS Research Methods Workshop

  39. Statistical Relevance and You • We are not statisticians • We don’t have the resources nor time to run full analysis • Our reports are not statistically relevant • Our analysis is focused on report fulfillment • We don't properly sample • Our audience is often “expert” library users • This alters the influence of their input • We don't run via SPSS • Statistical Package for the Social Sciences ULS Research Methods Workshop

  40. Using Survey Monkey (Free Version) • https://www.surveymonkey.com/ • Create a free account ULS Research Methods Workshop

  41. Survey Monkey (Free Version) • Limitations • 10 Questions • 100 Responses • No Page Logic • 20 Less Templates • No Customization ULS Research Methods Workshop

  42. Survey Question Creation • Create 2 questions you would like to ask of your department or patrons • Think about how you would use the results and write that idea in the page description. • We will look at several and discuss their efficacy. ULS Research Methods Workshop

  43. Focus Groups: the Method and How to Use It Karen Calhoun ULS Leadership Program 20 November 2013

  44. Learning objectives • Know the advantages and disadvantages of focus groups and when to use them • Be able to select and recruit focus group interviewees • Know how to prepare a focus group interview guide • Have information about: • The roles of the facilitator and recorder • How to analyze focus group interview data ULS Research Methods Workshop

  45. Focus Groups, Defined • Qualitative social science research method • Feature open, interactive, in-depth small group discussion (typically 6 to 10 people), led by a trained moderator/facilitator • Participants carefully selected • Evaluate participant thoughts, opinions, practices, values, beliefs, feelings in a nonthreatening, semi-structured setting ULS Research Methods Workshop

  46. Purpose of a focus group • IT IS TO: • Collect information and ideas on a pre-selected topic • IT IS NOT: • To achieve consensus • To answer participants’ questions (facilitator ≠ sage on stage) • To solve one or more problems • To make decisions • To provide a forum for participants to gripe or vent frustration (group therapy) ULS Research Methods Workshop

  47. Generate insight into not just what people think or feel, but why Can be comparatively low cost method research method Interactive, not one way Can be used alone or in combination with other research methods Advantages of Focus Groups CC BY NC Francois Proulx http://www.flickr.com/photos/91569742@N00/404909051/ ULS Research Methods Workshop

  48. Disadvantages/Risks of Focus Groups • Findings not representative of entire population of interest • Quality of results highly dependent on: • Appropriate preparation • Skill of moderator • Skill of recorder • Skill of analyst (preparation of report) • Can be challenging to evaluate responses to open-ended questions ULS Research Methods Workshop

  49. Steps to Set Up Focus Group Interviews • Decide how many focus group interviews to hold (usually from two to a dozen) • Select appropriate facility (see next slide) and time(s) for the event(s) • Decide on participant incentives • Recruit participants (usually 6 to 10 people in each focus group) • Prepare interview guide/script • Assign moderator and recorder • Conduct the focus group(s) • Analyze and report results ULS Research Methods Workshop

More Related