Experimental Design Workshop

1. Experimental Design Workshop Efficient and Effective Experiments Maximizing Information with Limited Resources Richard Preziosi Rob Shaw Norman Burton

2. Agenda • Why is design so important? • The path from question to hypothesis: a statistical perspective • Pitfalls and preliminary studies • Variation, sampling and replicates • Experimental designs to minimize error and improve power

3. Please DO NOT view this workshop as an isolated event • Other training sessions • Individual consultations • Books

4. Introduction • What is Experimental Design? • Statistics!

5. Introduction • What is Experimental Design? • Statistics! • Requires an appreciation of statistics

6. Introduction • What is Experimental Design? • Biological insight! • Logic • Common sense • Planning

7. Introduction • What is Experimental Design? • Biological insight! • Logic • Common sense • Planning • Note that there are different approaches to Experimental Design

8. Some myths about ED • Myth 1 • Its better to spend time collecting data than sitting around thinking about collecting data, just get on with it • Reality • A well designed experiment will save you tons of time. This belief often results in staff and post-docs sitting around while supervisors rewrite grant proposals and permit applications

9. Some myths about ED • Myth 2 • “It does not matter how you collect your data, there will always be a statistical ‘fix’ that will allow you to analyse them” • Reality • NO WAY! This belief results in people crying in my office. Big problems are non-independence and lack of control groups.

10. Some myths about ED • Myth 3 • “If you collect lots of data something interesting will come out and you will be able to detect even very subtle effects” • Reality • NO! Generally collecting lots of data without a plan wastes your time and someone’s money.

11. Costs of poor design • Time is wasted. This is something you can’t afford and its sometimes downright embarrassing. • Money and resources are wasted. This is something your supervisor (or department or company) can’t afford and tends to make them quite angry.

12. Costs of poor design • Ethical issues (when animals or humans are experimental subjects) • Experiments must minimize the stress and suffering of any animals involved • Minimum numbers must be used • Experiments must have a reasonable chance of success • Ethical issues include causing damage or excessive disturbance to an ecosystem • Using poor design in animal studies is not only wasteful and embarrassing but may also be illegal

13. Experimental Design and Statistics • Good experimental design is about more than statistics • You MUST know how you will analyse your experiment before you collect a single datum! • Once you have designed your experiment seek advice on the statistical test you will use. • Go ahead and use experienced people in your lab or department and/or a expert in statistics for this.

14. Why is ED so important in life sciences? • Biologists largely deal with complex, interacting systems (i.e. things that are alive OR composed of live parts) • Data are often non-independent • Random variation is common • Confounding factors are common

15. Independence of data • Samples tell us about Populations • This is only true if the data in a sample are drawn randomly from the population • The true difficulty of non-independent data is that we do not know how it influences the sample (could be positively or negatively correlated)

16. Random variation(AKA between-individual variation, inter-individual variation, within-treatment variation or noise) • Found everywhere in biological systems • Important enough in biological systems that most modern statistics were invented by biologists • Note that this inherent randomness is not present in the data for all sciences

17. Confounding factors(AKA Confounding variables, third variables) • These are factors that interact with both what we are manipulating and what we are measuring • These need to be either eliminated through experimental design or have their effects removed statistically (covariates)