What are Online Research Methods? Tristram Hooley & Jane Wellens

1. What are Online Research Methods? Tristram Hooley & Jane Wellens

2. What are online research methods? • Research methods that utilise the internet as medium for research • Sometimes referred to as internet mediated research • Includes research methods designed to investigate both online & offline phenomenon

3. Types of ORM • online questionnaires • synchronous and asynchronous interviews • virtual ethnographies • online experiments • web analytics & content analysis • + others

4. Online Research Methods - Challenges • Draw on a wide range of methodological, theoretical and disciplinary traditions • Understanding the range of tools, environments and online cultures is a precursor to successful research • Technological and social environments change year on year • Increased blurring of the online and offline environments creates a need for new composite methodologies • Potentially poses new and different ethical considerations

5. Advantages of online research? Huge uptake of ORM, especially online questionnaires Useful in particular situations e.g. researching online populations and previously difficult to contact groups Can mitigate distance and space and so internationalise research Can save time and money Decrease personal risk Different dynamics of communication: - more neutral venue? - more thoughtful responses? - different participants open up/contribute?

6. But….disadvantages But issues of digital access and digital literacy can limit who we talk to Drop off rates for questionnaires high Issues of identity verification Challenges to building rapport in interviews Circumstances of interview beyond interviewers control Technical obstacles Still need degree of technical competence and institutional support (although this is getting easier all the time)

7. Online Questionnaires - key questions What are your research questions/aims? Who are you talking to? Sampling Recruitment Non-response How are you talking to them? Question type Survey length Language

8. Advantages of online questionnaires Increasingly common approach (familiar to respondents) Speed and volume of data collection Savings in costs (to researcher) Flexible design Data accuracy Access to research populations Anonymity

9. Online questionnaires - challenges Sample bias Measurement error (Sax et al, 2003) Non-response bias Length, response and dropout rates Technical problems Ethical issues Recruitment Identity verification Response rates Identifying appropriate tools & implementation

10. Design issues Will your questionnaire be easy to use? Or will the interface, colours and question types frustrate people and put them off? Usability Will your questionnaire be accessible? Or will it crash on old computers, render weirdly on different operating systems and be impossible to use with a screen reader? Accessibility Will your questionnaire be doable? Or are your multi-media dreams running ahead of your technical ability and are your data arriving in an unusable format? Doability

11. Improving response rates • Make contact before mailing the survey e.g. an introductory letter • Provide information that builds trust e.g. names and photos of the researchers • Engage gatekeepers and encourage them to endorse the survey • Think about how your brand (e.g. the university) will be perceived by the subjects • Provide clear instructions on how to complete the questionnaire; • Limit the amount of personal information you request • Use simple questionnaire format and avoid complex or open-ended questions • Design survey so it takes approximately 10 minutes to complete; • Do not include more than 15 questions • Send one or two follow up reminders • Emphasise confidentiality (if appropriate) Key resource: Fan, W., & Yan, Z. (2010). Factors affecting response rates of the web survey. Computers in Human Behavior , 26 (2), 132-139.

12. Some Software Options Google Forms http://docs.google.comFREE Lime Survey http://www.limesurvey.orgFREE BOS http://www.survey.bris.ac.ukCHEAP SurveyMonkey http://www.surveymonkey.com Snap http://www.snapsurveys.com Using the functionality of your website or VLE Bespoke systems

13. So…. The quantity of information that may be generated, and the speed at which responses can be collected, can result in pleasing piles of data- but we should be wary of being seduced by sheer quantity; data is only useful if it is representative of the larger population. Wakeford (2000, 33)

14. Online Interviews – key questions • Why use online interviews? • What type of interview (synchronous or asynchronous)? • One-to-one or group? • Text based or using audio or video? • What software/service will you use? • How are you going to record, retrieve and analyse data?

15. Online Interviews: advantages • Carry out interviews with a very geographically dispersed population • Interview individuals or groups who are often difficult to reach, such as the less physically mobile or the socially isolated or those living in dangerous places. • Provide savings in costs to the researcher • Supply ready transcribed interview data, quickly, providing fast and cheap alternatives to face-to-face interviews • Reduce issues of interviewer effect as participants cannot 'see' each other

16. Online interviews - challenges • Trust and rapport building • Design of interview schedule • Keeping people on topic • Dealing with technical hitches • Maintaining momentum • Dealing with ‘silence’ • Guaranteeing the ethical rights of respondents including informed consent, confidentiality and privacy

17. Some software options • Adobe ConnectNow http://www.adobe.com/acom/connectnow/ • TinyChat http://tinychat.com/ (probably need to upgrade for anything serious) • GoogleTalk http://www.google.co.uk/talk/ • Many VLEs also have chat and virtual classroom tools that can be used for this kind of purpose.

18. So... • The data collected by virtual interviews can be rich and valuable to the researcher, but the potential of on-line research should not be exaggerated: many of the issues and problems of conventional research methods still apply in the virtual venue • Moreover, it is unlikely that online interviewing is going to replace face-to-face interviewing but rather it is another option in the methodological ‘toolkit’

19. Online research ethics • There is currently a big push to embed research ethics more formally in the culture of the social sciences. • Yet there is far less of a consensus about online research ethics. • This asks the question “is there anything special about the online environment that requires new set of ethical guidelines?” Key resource: Economic and Social Research Council (2005). Research Ethics Framework.

20. Ethical decision making and Internet research • Ess & AoIR (2002) stresses “Ethical pluralism” and argues that “there is more than one ethical decision-making framework used to analyze and resolve those [internet research] conflicts.” • Note: the world of chatrooms, MUDs and MOOs and USENET newsgroups, described in this report in 2002 is now radically transformed. Key resource: Ess, C. and AoIR Ethics Working Committee (2002) Ethical decision-making and internet research http://www.aoir.org/reports/ethics.pdf

21. Public Private Published Informal Writing Speech Personal Broadcast Anonymous Identified Where does it fit in? One of the problems of mapping existing research ethics onto the online world is the difficulty of deciding how to categorise the spaces that you are observing/interacting with. Key resource: Internet Research Ethics: Digital Library, Resources Centre, and Commons - http://internetresearchethics.org/

22. Ethical Challenges “The great variety of human inter/actions observable online and the clear need to study these inter/actions in interdisciplinary ways …This interdisciplinary approach to research leads, however, to a central ethical difficulty: the primary assumptions and guiding metaphors and analogies - and thus the resulting ethical codes - can vary sharply from discipline to discipline, especially as we shift from the social sciences. Ess, C. and AoIR Ethics Working Committee (2002)

23. Careful and critical use ORMs • So ORMs are not a shortcut `easy option’ • Many issues and problems of onsite research remain • Divide between onsite/online methods inappropriate • Must be used, carefully and critically and appropriately in light of each specific research topic Attempting to undertake online data collection is far easier than successfully accomplishing it. For those who chose to perform it, they must do so deliberately and cautiously.(Best and Krueger, 2004) While online methodological frameworks are in constant flux, change is not necessarily always progressive: there is a need for online researchers to practice their 'craft' with reflexivity. (Madge and O’Connor, 2005)

24. Further reading Exploring online research methodshttp://www.geog.le.ac.uk/orm/ Tristram’s ORM bibliography on CiteULikehttp://www.citeulike.org/user/pigironjoe/tag/online_research_methods Tristram’s blog http://adventuresincareerdevelopment.posterous.com/ I talk about ORM sometimes and technology often on this blog.