Aspects of Experimental Designs

1. Aim: How can we assign experimental units to treatments in a way that is fair to all of the treatments?

2. Aspects of Experimental Designs • The design of an experiment first describes the response variable or variables, the factors (explanatory variables), and the layout of the treatments, with comparison as the leading principle. • The second aspect of design is the rule used to assign the experimental units to the treatments.

3. Unbiased Experiments • Comparison of the effects of several treatments is valid only when all treatments are applied to similar groups of experimental units. • Systematic differences among the groups of experimental units in a comparative experiment cause bias.

4. How can we assign experimental units to treatments in a way that is fair to all of the treatments? • The statistician’s remedy is to rely on chance to make an assignment that does not depend on any characteristic of the experimental units and that does not rely on the judgment of the experimenter in any way. • The use of chance can be combined with matching, but the simplest design creates groups by chance alone.

5. Example • Does talking on a hands-free cell phone distract drivers? Undergraduate students “drove” in a high-fidelity driving simulator equipped with a hands-free cell phone. The car ahead brakes: how quickly does the subject respond? Twenty students (the control group) simply drove. Another 20 (the experimental group) talked on the cell phone while driving. • This experiment has a single factor (cell phone use) with two levels. The researchers must divide the 40 student subjects into two groups of 20. To do this in a completely unbiased fashion, put the names of the 40 students in a hat, mix them up, and draw 20. These students form the experimental group and the remaining 20 make up the control group.

6. Randomization • The use of chance to divide experimental units into groups is called randomization.

7. Logic behind the randomized comparative design • Randomization produces two groups of subjects that we expect to be similar in all respects before the treatments are applied. • Comparative design helps ensure that influences other than the  cell phone operate equally on both groups. • Therefore, differences in average brake reaction time must be due either to talking on the  cell phone or to the play of chance in the random assignment of subjects to the two groups.

8. Principles of Experimental Design • The basic principles of statistical design of experiments are • Comparetwo or more treatments. This will control the effects of lurking variables on the response. • Randomize—use impersonal chance to assign experimental units to treatments. • Repeateach treatment on many units to reduce chance variation in the results.

9. Statistical Significance • We hope to see a difference in the responses so large that it is unlikely to happen just because of chance variation. • We can use the laws of probability • Looking to see if the treatment effects are larger than we would expect to see if only chance were operating. • If they are, we call them statistically significant.

10. Statistical Significance • An observed effect so large that it would rarely occur by chance is called statistically significant.