Research Design in International Education: Methods and Practice Overview

1. NAFSA PRE-CONFERENCE WORKSHOP #11RESEARCH FOR INTERNATIONAL EDUCATION AND EXCHANGE: METHODS, POLICY, AND PRACTICE Washington D.C. Sunday, May 25, 2008

2. Research OverviewFrom Principles to Practice Bruce La Brack, Ph.D. blabrack@pacific.edu Presenting for Michael Paige, Ph.D. r-paig@umn.edu with thanks to Rick Sutton

3. Identify your audience! • Research findings—no matter how powerful—are useless if they don’t find the right audience. • QED, your first research step is to identify the audience that can implement/change the policy you’ve targeted. They are your readers. And, yes, politics are always involved!

4. Why are you doing this research? • To document/benchmark existing operations? • To demonstrate change/trends in IE? • To promote good ideas & initiatives? • To focus attention on key agenda items? • To influence policy?

5. Constructing a Testable Hypothesis • Frame your question • Put your question into a positive statement • If yes… • If no… • Identify Methodology to determine yes/no • Diagram each step of research, data collection, and analysis (critically important if working with other offices—e.g., IR, registrar, admissions, etc.)

6. “Research Design” typically considered as methodology • Sample size (random vs. proportional) • Control groups • Pre-tests & post-tests • Instrumentation and validation • Internal threats to validity • External threats to validity • Replication Sutton, Miller, & Rubin: “Research Design in Assessing Learning Outcomes of Study Abroad Programs” in Guide to Assessment in Education Abroad (2007)

7. Research Design Issues: The Basics • Choice of Methodology • Choice of Methods • Cost Requirements • Time Requirements • Sampling Requirements– who, how, when, how many? • Research Logistics – scheduling, travel, recordkeeping, budgets, etc. • Research Team – who, how many, what qualifications? Use internal/external consultants? • Protection of Human Subjects (Institutional Review Boards/Research Permission)

8. Data Collection Methods • Interviews (scheduled or open?) • Focus Groups • Surveys • Inventories (new or existing?) • Participant Observation • Journals - (inter-rater reliability?) • Ethnography • Case Study • Authentic Texts/Written Materials

9. Purist vs. pragmatist • If constituents are favorably inclined, quick-&-dirty research has immediate utility & benefit. • If constituents are not favorably inclined, imperfections of methodology will be focus of attack.

10. Methodology Taxonomy • Positivist • Theory Driven • Theory Testing • Hypothesis Testing • Deductive • Objective • Statistical Analyses of Data • Common Methods: Surveys, Inventories

11. Methodology, contd. • Interpretive • Theory Generating • Exploring "Lived Experience" • Deep/’Thick” Description (C. Geertz) • Exploratory • Inductive • Subjective • Co-Construction of Meaning (Researcher and Subjects) • Common Methods: Interviews, Focus Groups, Participant Observation, Ethnography, Case Study, Personal Stories, Content Analysis, Journals, Photos, Media

12. Methodology, contd. • Critical Research • Co-Construction of Research Goals • Co-Construction of Meaning • Researches Social Issues (e.g., Inequity and Inequality) • Purpose-Driven Research (Redress Social Issues) • Subjective • Common Methods: Case Study, Community-Based Research, Action Research, Action Planning

13. Research Challenges/Controversies • Objectivity/Subjectivity “Hard” Facts = Quantitative data vs. “Soft” Anecdotal = Qualitative data • Postmodernism/Deconstruction (no ultimate truth/objectivity, all relative to viewpoint of participant(s) and who controls narrative)

14. Research Ethics • Protection of Human Subjects & Protocols • Obligations to Respondents • Publication Conventions • Intellectual Property • Anonymity • Rewards • Researcher Bias • Post-Project Dissemination

15. A spectrum ***** of research dissemination vehicles • Published articles • Published research notes • Occasional papers series • Conference presentations/proceedings • Inter-agency white papers • Institutional white papers • Internal reports • Participating parties and institutions - acritical and often neglected component

16. Applied Research  Practical Research Agenda • IE research movement is growing, but still at somewhat rudimentary stages • Many fundamental questions about IE not yet addressed • Independent research projects (often small) frequently proceed without intentional collaboration • As current wave of research builds, opportunities to connect points of convergence & dissonance will emerge • Researchers need to be aware of each other and of broader contexts, but are largely focused on serving local needs • Alas, Research is currently a luxury for most international educators