1 / 26

Educational Action Research

Educational Action Research. Todd Twyman Summer 2011 Week 1. Agenda. Discuss Internal and External Validity Review Threats to Internal Validity Consider Precautions for Minimizing Threats to Internal Validity. Why are you here?.

shiri
Télécharger la présentation

Educational Action Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Educational Action Research Todd Twyman Summer 2011 Week 1

  2. Agenda • Discuss Internal and External Validity • Review Threats to Internal Validity • Consider Precautions for Minimizing Threats to Internal Validity

  3. Why are you here? • How can you use the results from your action research project to inform your practices? • Why is action research important for improving schools?

  4. Internal Validity The internal validity of a study relies on the researcher’s ability to rule out factors other than the intervention as possible causes of changes in the dependent variable.

  5. Improving Internal Validity • Maximize systematic variance • Make the groups as different as possible in regard to the intervention • Minimize error variance • Reduce the differences between the participants • Control extraneous variance • Manipulate the environment to maximize the controls

  6. Improving Internal Validity • Focus is on trying to “control” extraneous factors (or confounds). Identify threats and then either delete or reduce threats to validity through your research design.

  7. Thinking about Internal Validity… • What could have caused these findings? • What could have influenced the results besides the intervention? • What are alternative hypotheses for the results? • What decisions would you make based on these results? • What are the implications of the findings?

  8. History Maturation Testing Instrumentation Statistical regression Differential selection Experimental mortality Selection-maturation interaction Experimental treatment diffusion Compensatory rivalry by the control group Compensatory equalization of treatments Resentful demoralization of the control group Threats to Internal Validity

  9. Threats to Internal Validity • History: Events may occur that influence the observed effects. • Maturation: Physical/psychological changes that influence the observed effects. • Testing: Pre-test influences the post-test. • Instrumentation: Changes in the measurement tool influence the observed effects.

  10. Identifying Threats • A tenth-grade biology teacher institutes a new approach to the teaching of one-celled animal life based on a series of seven-minute single-concept films. Prior to starting the new unit, she develops a test covering the material and administers it to her class before and after the unit, noting with satisfaction a dramatic improvement by her pupils on the test.

  11. Threats to Internal Validity • Statistical regression: Test scores move toward the mean. • Differential selection: Participants are differentially selected into groups. • Attrition: Participants differentially drop out of the study. • Selection-maturation interaction: Participants differ based on maturation.

  12. Identifying Threats • A school principal has received a federal grant to institute a new mathematics enrichment program. Using school records, he computes the average mathematics achievement score on a nationally standardized examination for his fifth-, ninth-, and twelfth-grade pupils during each of the preceding three years. He plans to have the same tests administered for the two years following the institution of the new program. He wishes to compare the relative performance of the three classes before and after the implementation of the new program.

  13. Threats to Internal Validity • Treatment diffusion: Control group seeks access to treatment. • Compensatory rivalry: Control group works harder to make up for group placement. • Compensatory equalization: Outsiders provide other treatment to compensate. • Demoralization of control group: Control group performs worse due to discouragement.

  14. Identifying Threats • A researcher uses a table of random numbers to subdivide 15 junior high school classes into three groups of five classes each (30 pupils per class) as follows: “Treatment A,” “Treatment B,” and “Control.” The treatments are administered during the first four months of the school year, and at mid-semester a posttest is given to all of the 450 pupils involved.

  15. 3 Steps to Strengthening Treatment Fidelity • Write precise specifications for dependent and independent variables. • TRAIN extensively to ensure treatment is delivered as intended. • Collect data on fidelity of treatment implementation.

  16. Internal Validity Review

  17. 1. History (H) 2. Maturation (M) 3. Testing (T) 4. Instrumentation (I) 5. Statistical regression (SR) 6. Differential selection (DS) 7. Experimental mortality (EM) 8. Selection-maturation interaction (SMI) 9. Experimental treatment diffusion (ETD) 10. Compensatory rivalry by the control group (CR-control) 11. Compensatory equalization of treatments (CE) 12. Resentful demoralization of the control group (Rdem) Threats to Internal Validity

  18. Control group works harder to make up for group placement • Pre-test influences the post-test • Control group performs worse due to discouragement • Test scores move toward the mean • Events may occur that influence the observed effects

  19. Control group seeks access to treatment • Participants are differentially selected into groups • Physical/psychological changes that influence the observed effects • Participants differentially drop out of the study

  20. Participants differ based on maturation • Changes in the measurement tool influence the observed effects • Outsiders provide other treatment to compensate

  21. Quiz

  22. Steps to Reducing Threats to Validity • Debrief with peers: identify strengths/ weaknesses/ biases. • Align data sources with research questions. • Triangulate data sources. • Record data accurately and without bias. • Embed fidelity checks. • Thoroughly describe the setting and study. • Engage in continuous and ongoing reflection.

  23. Read with an Informed Eye Assignment 3 will ask you to critique an Action Research article of your choice. To prepare: • Look carefully at the methodology of research articles. • Evaluate the degree to which the authors have controlled threats to internal validity. • Question everything!

  24. Critiquing Research • Are the problems and their educational importance explained clearly? • Does the literature review sufficiently set up a rationale for the need for this study? • Are the research variables easily identified from the hypothesis or purpose statement?

  25. Critiquing Research • What instruments are used in the study? Are they appropriate for the research question and for the participants? • Is the research design appropriate to the research question? • How does the study control for threats to internal or external validity? • Are the conclusions logically related to the questions posed in the research question?

  26. Your own study • Identify the goals for conducting your study • Identify the types of validity that are applicable to your study (consider the purpose of the study, activities, audience, data sources) • Create a plan for reducing the threats to internal validity that is aligned with the types of validity addressed above.

More Related