BTG 315 Research and Methodology

1. BTG 315Research and Methodology Dr. F. N. Nwalo

2. Basic steps of a research project • Find a topicWhat, When • Formulate questionsWhat, Why • Define populationWho, When • Select design & measurementHow • Gather evidenceHow • Interpret evidenceWhy • Tell about what you did and found out

3. Topic ideas • Online chat reference • Types of questions • Subject? Type? • # of turnaways* • Difference in discourse • In-person vs. chat • Partnership studies • Similar libraries with same software

4. Topic Ideas • E-book usage • Usability studies of • Online tutorial(s) • ‘My Library” portals • Analysis of library web sites or library instruction sites or pathfinders by best practices • Student learning outcomes in LI programs

5. Types of methodologies • QuaLitative Measures • Descriptive • Numbers not the primary focus • Interpretive, ethnographic, naturalistic • QuaNtitative Measures • N for numbers • Statistical • Quantifiable

6. QuaLitative measures • Content Analysis • Analyzed course syllabi of library use through discipline and level (Rambler) • Studied online tutorials, applying best practices recommendations (Tancheva)

7. QuaLitative Measures • Discourse Analysis • Analyzed student responses in writing and discussions to a short film & compared findings to parallel study with LIS grad Ss (Vandergrift) • Focus Groups • Discussed how participants experience & use the library (Von Seggern & Young) • Studied why students use the Internet and how much time they use it (Wilson)

8. QuaLitative Measures • Interviews • Studied 25 HS students’ web use for research assignments (Lorenzen) • Looked at what type of information first year students need and how they go about acquiring it (Seamans) • Observation (obtrusive) • Observed students as they conducted online research & noted their activities (Dunn) • Observation (Unobtrusive) • Retrieval of discarded cheat sheets to analyze academic misconduct (Pullen et. al.)

9. QuaLitative Measures • Think Aloud Protocols • Studied how users navigate a library web site (Cockrell & Jayne) • Usability testing • Examined students’ mental models of online tutorials (Veldof & Beavers)

10. QuaNtitative measures p < .05 • CompareThings • Count Things • Survey People About Things

11. QuaNtitative measures p < .05 • Comparison studies • Experimental and control groups • Instructional methodologies (Colaric; Cudiner & Harmon) • Program assessment using before/after analysis of research papers(Emmons & Martin)

12. QuaNtitative measures p < .05 • Pre & Post Tests (Van Scoyoc) • Measures & Scales • Bostick’s Library Anxiety Scale (Onwuegbuzie & Jiao; Van Scoyoc) • Procrastination Assessment Scale (Onwuegbuzie & Jiao)

13. QuaNtitative measures p < .05 • Numeric Studies • Citation AnalysisBibliometrics (Dellavalle) • Webometrics (Bar-Ilian)

14. Ready Made Data Sets • National Survey of Student Engagement (Whitmire) • College Student Experiences Questionnaire (Kuh and Gonyea) • The Web • Internet Archive (Ryan, Field & Olfman) • Electronic journals (Dellavalle) • Library server logs

15. Research Defined and Described “Research is the systematic approach to obtaining and confirming new and reliable knowledge” • Systematic and orderly (following a series of steps) • Purpose is new knowledge, which must be reliable This is a general definition which applies to all disciplines

16. Notice that: “… truth was not used in the definition of research” (p 16) “This concept of truth is outside of the productive realm of thinking by researchers” (p 16)

17. Research is not … cont. Data Collection • an intermediate step to gain reliable knowledge • collecting reliable data is part of the research process

18. Research is not … cont. Searching out published research results in libraries (or the internet) • This is an important early step of research • The research process always includes synthesis and analysis • But, just reviewing of literature is not research

19. Research is… • Searching for explanation of events, phenomena, relationships and causes • What, how and why things occur • Are there interactions? • A process • Planned and managed – to make the information generated credible • The process is creative • It is circular – always leads to more questions

20. All well designed and conducted research has potential application. • Failure to see applications can be due to: • Users not trained or experienced in the specialized methods of economic research and reasoning • Researchers often do not provide adequate interpretations and guidance on applications of the research • Researchers are responsible to help users understand research implications (How?)

21. Public good • Public research is a public good • May be more rigorous and objective because it is subject to more scrutiny • Private research may also be rigorous • But research on a company’s product may be questioned as biased.

22. Classification of ResearchBasic vs Applied Research • Basic – to determine or establish fundamental facts and relationships within a discipline or field of study. Develop theories … (examples in economics?) • Applied – undertaken specifically for the purpose of obtaining information to help resolve a particular problem • The distinction between them is in the application • Basic has little application to real world policy and management but could be done to guide applied research

23. Disciplinary • designed to improve a discipline • dwells on theories, fundamental relationships and analytical procedures and techniques • In economics, the intended users are other economists • Provides the conceptual and analytical base for other economic research • It is synergistic and complementary with subject matter and problem-solving research

24. Disciplinary… cont. • Provides the foundations for applied research • Circular as applied research reveals the shortcomings of disciplinary research • Examples of some economic theories? (supply & demand, price elasticity, consumer utility …)

25. Subject-matter research • “research on a subject of interest to a set of decision makers “ (p 22) • Tends to follow subject-matter boundaries within a discipline ( eg. resource economics, production economics, labor economics) • Inherently multidisciplinary, drawing information from many disciplines • eg. consumer economic draws from psychology, natural resource economics from biology, economic policy from political science

26. Subject-matter research … cont. • Provides policy makers with general knowledge to make decisions about various problems. • A primary source of policy applications for economics • Subject-matter research is a cornerstone in economics – it involves direct application of economics to contemporary issues.

27. Problem-solving research • Designed to solve a specific problem for a specific decision maker • Often results in recommendations on decisions or actions • Problem-solving research is holistic – uses all information relevant to the specific problem (while disciplinary research tends to be reductionist) • Disciplinary research is generally the most “durable” (long lasting); problem-solving research the least durable

28. Analytic vs Descriptive Research • Descriptive Research – the attempt to determine, describe, or identify something • The intent is often synthesis, which pulls knowledge or information together • Analytic – the attempt to establish why something occurs or how it came to be • All disciplines generally engage in both

29. Methodology Defined & Described Methodology and Method are often (incorrectly) used interchangeable • Methodology – the study of the general approach to inquiry in a given field • Method – the specific techniques, tools or procedures applied to achieve a given objective • Research methods in economics include regression analysis, mathematical analysis, operations research, surveys, data gathering, etc.

30. Contrast research methodology in economics (the approach to research) to economic methodology (the general approach to economic reasoning and economic concepts) • While these are different they are interdependent ( in the same way as science and research are related)

31. The Process of Research • The process is initiated with a question or problem (step 1) • Next, goals and objectives are formulated to deal with the question or problem (step 2) • Then the research design is developed to achieve the objectives (step 3) • Results are generated by conducting the research (step 4) • Interpretation and analysis of results follow (step 5)

32. Overview of Presentation • Overview of Quantitative/Qualitative Methodologies • Focus on Qualitative Research • Questions, Characteristics, Methods, Data Analysis, Credibility • Research Methodologies / Methods & GILD • Discussion

33. Overview of Research Methodologies • Qualitative Research • Ethnography, Case Study, Grounded Theory, Autobiography, Participatory Action Research, Phenomenology (each grounded in a specific discipline and philosophical assumptions) • Quantitative Research • Survey methods, Experiments • Mixed Methods • Draw from qualitative and quantitative methods

34. Quantitative • A quantitative approach is one in which the investigator primarily uses post-positivist claims for developing knowledge (i.e. cause and effect thinking, reduction to specific variables and hypotheses and questions, use of measurement and observation, and the test of theories). (Creswell, 2003, p.19)

35. Qualitative - Definition • … qualitative researchers study things in their natural settings, attempting to make sense of or interpret phenomenon in terms of the meanings people bring to them. (Denzin & Lincoln, 2000, p.3).

36. Qualitative - Definition • A qualitative approach is one in which the inquirer often makes knowledge claims based primarily on constructivist perspectives (i.e. the multiple meanings of individual experiences, meanings socially and historically constructed, with an intent of developing a theory or pattern) or advocacy/participatory perspectives (i.e. political, issue-oriented, collaborative or change oriented) or both. (Creswell, 2003, p.18)

37. Research Questions • Qualitative • In qualitative study inquirers state research questions, not objectives (i.e. specific goals for the research) or hypotheses (i.e. predictions that involve variables and statistical tests). (C., 2003, p.105) • Example: How do students use program development tools?

38. Characteristics of Qualitative Research • Takes place in the natural setting • Uses multiple methods that are interpretive • Is emergent rather than tightly prefigured • Fundamentally interpretive (role of researcher as interpreter) • Researcher views social phenomena holistically • Researcher systematically reflects on who he or she is in the inquiry and is sensitive to hiw or her personal biography and how it shapes the study • Researcher uses complex reasoning that is multifaceted, iterative, and simultaneous • Researcher adopts and uses one or more strategies of inquiry

39. Research Methods • Interviews • Focusgroups • Participant observation (field notes) • Video • Text and Image analysis (documents, media data)

40. Data Analysis (C., p.191) • Organize and prepare the data for analysis • Read all data, get a sense of the whole • Begin detailed analysis with coding process • Generate a description of the setting/people as well as categories or themes for analysis • Represent themes (writing, visual, etc.) • Interpret and make meaning out of data • *iterative, non-linear process

41. Credibility • Use of Triangulation • Use of Member Checking • Use of rich, thick Description • Clarification of Bias • Use of Negative or discrepant information • Prolonged field time • Peer Debriefing • (C., 2003, p.196)

42. Research Methodologies & GILD • Ethnography • An ethnography is a description and interpretation of a cultural or social group or system. The research examines the group’s observable and learned patterns of behaviour, customs, and ways of life. (C., 1998,p.58) • Rapid Ethnography

43. Research Methodologies & GILD • Case Study • … a case study is an exploration of a ‘bounded system’ … over time through detailed, in-depth data collection involving multiple sources of information rich in context. This bounded system is bounded by time and place, and it is the case being studied – a program, an event, an activity, or individuals. (C., 1999, p.61)

44. Useful Methods • Participant observation • Gains insight into understanding cultural patterns to determine what’s necessary and needed in tool development (complementary to interviews) • Interviews/Focusgroups with stakeholders • Explores how tools are used and could be used in a novice programming course • Gains insight into the meaning of tools for students for learning to program

45. Useful Methods • Data analysis • Themes arising from data would provide insight into current “learning to program” issues and see what is important to students / teachers / administrators • Survey • Useful for verifying results on a larger scale • User Testing • Useful for triangulating results

46. Choice of Methodology & Methods • Depends on • Research Questions • Research Goals • Researcher Beliefs and Values • Researcher Skills • Time and Funds

47. Discussion • (How) Can tool improvement, collaboration, ed-tech questions and learning outcomes be addressed in the same study? • What GILD research questions match which research methodologies?

48. References • Creswell, J.W. (1998). Qualitative inquiry and research design. Choosing among five traditions. Thousand Oaks, CA: Sage. • Creswell, J.W. (2003). Research design. Qualitative, quantitative and mixed methods approaches. Thousand Oaks, CA: Sage. • Denzin, N.K. & Lincoln, Y. (2000). Introduction: The discipline and practice of qualitative research. In N.K. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp.1-17). Thousand Oaks, CA: Sage.

49. Recognising Research: Approaches & Designs Introduction to Study Skills & Research Methods (HL10040) Dr James Betts FACSM J.Betts@bath.ac.uk @DrBSteamjets

50. Lecture Outline: • The Research Process • The Research Design Continuum • Experimental Designs • Sampling Methods • Scientific Reasoning • Quantitative & Qualitative Research Strategies.