Experimental Design Fr. Clinic II Dr. J. W. Everett
Planning • Begins with carefully considering objectives (or goals) • How do our filters work? • Which filter is best? • Performance • Cost • Ease of Use • Durability…
Variables • Variables include both inputs and outputs - i.e., factors and responses • The selection of these variables is best done as a team effort
Terms • Factor • Can be varied, e.g., temperature, pressure, material … • And may have an effect (cause a response) • Response • Results, e.g., reaction rate, flow rate, load capacity … • Caused by factor(s) • Level • Each variable (factor or response) can be “set” to or measured at different levels, e.g., 10, 20, 30 OC… • Run • Test involving a unique combination of factors / levels
What are our factors? • How do our filters work? • Type of filter cartridge, type of pump mechanism … • What effects their performance? • Stroke force, stroke rate, flow rate, filter “age”… • Comparative Assessment • The portable water filters themselves!
Water are our possible responses? • Water quality • Water quantity • Filter capacity • Cost (cartridge replacement) • Ease of use • …?
Experimental Design • Choosing an experimental design depends on the objectives of the experiment and the number of factors to be investigated
Experimental Design Objectives • Comparative Objective • 1 or several factors under investigation • Make conclusion about 1 important factor • in the presence of, and/or in spite of the other factors • Screening Objective • select or screen important effects from lesser ones • Response Surface (method) Objective • estimate response to multiple factors • find improved or optimal process settings • troubleshoot process problems and weak points • make product / process more robust against external and non-controllable influences
Typical Experiment Programs • Test • One or more factor • At various levels • With multiple runs • Control for nuisance variables • Measure all relevant responses • Number of total runs increases exponentially with number of factors
Experimental Designs • Completely Randomized Designs • Comparative objective, 1 factor • Randomized Block Designs • Comparative objective, multiple factors • Full or fractional factorial • Screening objective, multiple factors • Many more!…
Completely Randomized Designs • One factor, with multiple levels of interest • For example – effect of temperature on a chemical reaction • three levels, two runs each gives 90 unique orders to conduct experiment • T1, T1, T2, T2, T3, T3; • T1, T2, T3, T1, T2, T3… • In a completely randomized design, you would randomly select the order of runs
Randomized Block Designs • One factor or variable is of primary interest. However, there are also several other nuisance factor variables • Nuisance variables may affect result (response), but are not considered of primary interest • For example: specific operator who prepared the treatment, the time of day the experiment was run, the room temperature,… • All experiments have nuisance factors
Randomized Block Designs (cont.) • Blocking • Run every level of the primary factor with the nuisance factor(s) held the same (blocked) • Minimizes total # of runs • Random • Run with nuisance factors selected randomly. • More runs may be required • Likely to get more variability (error) in results • May be able to block some nuisance factors, but not all
Randomized Block Example • Engineers at semiconductor manufacturing facility test effect of 4 different wafer implants using 3 runs for each level • Nuisance factor is "furnace run", since each furnace run differs from the last and impacts many process parameters • Block: run all 4x3=12 wafers in the same furnace run • Completely Random: randomly select order and put each wafer on a different furnace run
Full Factorial • Run all combinations of factors and levels • Example: A basic experimental design is one with all input factors set at two levels each • These levels are called ‘high’ and ‘low’ or ‘+1’ and‘-1’ respectively. • A design with all possible high/low combinations of all the input factors is called a “full factorial design in two levels”
Fractional Factorial • A factorial experiment in which only an adequately chosen fraction of the treatment combinations required for the complete factorial experiment is selected to be run
What should we do? • For our competitive assessment, we have one factor – portable water filters • The levels are the different filters • What are our nuisance factors? • Properties of pond water? • Day of experiment? • Operators of experiment? • ??? • For some experiments will add another factor.
Comparative Assessment 1 • Which filter pumps the most water per stroke? • Experiment: Pump water through clean filter cartridge at various stroke rates • Additional Factor: strokes per minute • Response: Flow per stroke • Nuisance Factors? • 5 stroke rates, 1 minute each time, at least two runs each. Try to span range of stroke rates campers would use. Try to get high enough to demonstrate decrease at high stroke rates
Comparative Assessment 2 • Which filter creates the best water? • Experiment: Pump pond water through each filter and measure water quality • No additional factors • Responses: • Color (apparent) • Turbidity • Pathogens • Nuisance factors?
Comparative Assessment 3 • Which Filter is best aesthetically and ergonomically? • Experiment: Students rate each filter by filling out a questionnaire • No additional factors • Response: overall score • How do you create an overall score? • Nuisance Factors?
Comparative Assessment 4 • Which filter cartridge needs least pressure? • Experiment: Run clean water through clean filter cartridge at multiple flow rates • Factor: Flow Rate • Response: Pressure difference between inlet and outlet side of cartridge • Nuisance Factors? • 5 flow rates, from half of manufacturers claim to just below failure by leakage
Pump and system curves http://www.cheresources.com/centrifugalpumps5.shtml
Comparative Assessment 5 • Which filter cartridge needs the least additional pressure as it treats water? • Experiment: Run dirty water through filter at constant flow rate • Additional Factor: Quantity of Water Treated • Responses: • Pressure difference between inlet and outlet side of cartridge • Water Quality Improvement (Turbidity and Apparent Color) • Other Factors? Flow Rate • Keep flow rate constant and try to run at as high a flow rate as possible. If filter is cleanable, do it when needed.
Comparative Assessment 6 • Which filter cartridge requires least effort to pump? • Experiment: measure force required to pump water through clean filter cartridge • Additional Factor: Stroke Speed • Response: Stroke Force • Nuisance Factors? • 5 stroke speeds, spanning range used by campers (base on CA 1). No leaking!