Supplement 11: Testing hypothesis examples

1. Supplement 11: Testing hypothesis examples

2. Requiring a Math Skills Unit • Pozo, Susan and Charles A. Stull (2006): “Requiring a Math Skills Unit: Results of aRandomized Experiment,” American Economic Review, 96(2): 437-441.

3. Mathematics matters • Research spanning three decades supports what many experienced instructors of economics have long concluded—math matters. Students with greater mathematics preparation attain higher test scores in introductory Economics • Ballard and Johnson (2004) find “mastery of extremely basic quantitative skills is among the most important factors for success in introductory microeconomics.” • Mathematical competency reduces anxiety in economics classes (Mary Ellen Benedict and John Hoag, 2002). • To the extent that anxiety may interfere with the cognitive process, an effective mechanism to correct for math deficiencies is desirable.

4. In this paperreports on the results of acontrolled experiment with random assignment,which tests whether giving a grade incentiveto complete a math skills unit resultsin higher overall achievement in introductoryeconomics.

5. Experimental design • The experiment was performed with studentsenrolled in two sections of principles of macroeconomicstaught by the same professor duringthe spring 2004 semester at Western MichiganUniversity, a large regional university located inKalamazoo. • The assignment to treatment or controlgroup was determined by randomization andnot by class section. • The experiment included 273students.

6. Treatment vs. control • Students in both the experimental and controlgroups were asked to complete Aplia’s on-linediagnostic math test during the first week ofclass. The test consisted of 28 questions coveringnumerical calculations, graphs, units ofmeasurement, area, and simple algebra. Followingthe due date, students received their scoresand information about math deficiencies. • Studentsin the experimental group were remindedthat this test score would serve as their mathgrade, but they could improve their grade byworking through the appropriate on-line tutorialsand taking Aplia’s post-review test. Thestudents were reminded that the higher of thetwo test scores would be used in computingfinal grades. Hence, students with math deficiencieswere provided an incentive to improvemath skills, while proficient students need bearno further costs. • Students in the control group were stronglyencouraged to complete the math diagnostictest, to work through the appropriate tutorials,and to take the post-review test to gauge theircomprehension of the material. They were informedthat neither the pre-review nor postreviewscore would be used in their final grade.

7. The performance in the math test

8. The results Can we assume independence of the two populations? • H0: mE – mC 0 • H1: mE – mC> 0 Yes. Because of the randomization procedure in putting students into treatment and control group. Standard deviation of the sample mean.

9. Can we conclude that Math matters? • Due to the random allocation of students into the control and experiment group, there should not be systematic reason to see students’ background (e.g., IQ, training in economics, and math, etc.) before taking the course has any impact on the final result. So, the result is likely due to the “treatment”. • What is the treatment? Giving incentive to students to work hard in the additional math test. • Two possible channels for the treatment group to obtain better exam results. • Treatment group has improved their math skills. Better math skills leads to better performance. • Treatment group had acquire the taste of working harder in the course after working hard in the math test.

10. Supplement 11: Testing hypothesis examples - END -