Research Methodology

Research Methodology EPH 7112 LECTURE 7: EXPERIMENTAL DESIGN

Contents • Synthesis • Implement Solution • Design Experiments • Conduct Experiments • Reduce Results

Scientific Method

Analysis

Hypothesis • Specify detail and comprehensive solution • Assert expected results • Define factors that will be varied • Measure against performance metrics

Synthesis • Implement the solution • And experiment • To accomplish the goals • And validate the hypotheses

Synthesis Steps • Implement Solution • Design Experiments • Conduct Experiments • Reduce Results

Implement Solution • Implement solution to test hypotheses • Methods: • Acquire • Construct • Combination of both

Implement Solution • Construct • Custom made to meet requirements • Time consuming • Expensive

Implement Solution • Acquire • Quick solution • Cheaper • May not meet requirements

Implement Solution • Consider strongly to acquire the solution • Even if part of entire solution • Consultants – acquired solution?

Implement Solution • Example: Optical Amplification in S-band • Construct using Thulium doped fiber • Problem: Fusion Splicing is not possible • Solution: Use Angled Connectors • Issue: Not specified during order

Implement Solution • Is your solution right or is it the right solution? • Careful implementation • Step-by-step • Troubleshooting • Example: Constructing an optical amplifier • Problem: WDM is faulty

Design Experiments • To design a series of experiments • Results used to estimate how good solution to solve problem • An experiment acquires data to measure the performance of the solution under controlled conditions in a laboratory

Design Experiments • Always good to have a check list • Objective • Unit under Test • Inducers • Sensors • Supervisor • Channels • Domain Knowledge • Range Knowledge • Solution

Experiments • Is it really necessary? • How about theoretical or simulation work? • Experiment = verification • Example: Find solution of two-dimensional plane that satisfy certain conditions

Experiments • Simulation and modeling • Verify against experimental results • Example: Modeling of Optical Amplifier • Advantages of modeling • Optimization • Analyzing the physical phenomena

Design Experiments • Planning: • Specification Experiment Laboratory • Design of experiment blocks • Design of protocols • Acquiring and managing data

Experiment Laboratory • Laboratory is where the experiment takes place • Large room with test & measurement equipments, units under test, chemical & mechanical apparatus, computers

Laboratory • Experiments can also take place: • In an office • Field • Manufacturing Plant

Laboratory • Depends on the experiment: • Objective • Sample • Unit under Test • Inducers • Sensors • Solution

Laboratory: Safety • Watch out for moving or revolving parts (they don’t like necklaces and neck ties!) • Watch out for Electro-Static Sensitive Devices • Limit personnel into the laboratory • Maintenance and cleaning personnel can cause mishaps

Design Experiment Block • Process that takes place in the laboratory during the experiment • Important terms • Sample • Factors (independent variables) • Inducers • Sensors

Sample • Task unit consisting of objects, living plants, animals, humans that is the subject in the experiment

Factors • Condition or parameter of a task whose value is intentionally varied to measure its impact on the results of the task

Inducers • A device or mechanism that alters the task unit/ subject during the experiment

Sensors • Device that capture the results from the task/ unit or subject

Extent • Each factor is assigned a set of values • Extent of the factor space is total number of unique combination of values that may be assigned to each factor • FVa = number of values for factor a • Extent = FVa x FVb x FVc

Treatment • Each one of the unique combination of values that may be assigned to every factor is called a treatment • One instance in the entire factor space

Case Study • Factors • Typeface: 2 types (Serif and Sans Serif) • Noise Level: 12 levels • Character: 36 • Extent = 36 x 12 x 2 = 864 • Each combination = treatment

Block Design • If sample is a single object or device, then all the possible treatment must be assigned to it during the experiment • Example: Characterization of a Thulium doped Fiber Amplifier for different pump powers and wavelengths

Block Design • If sample is more than one, then the treatments may be distributed in some way among the sample • Important terms: • Experiment trial • Experiment block

Experiment Trial • Complete set of treatments applied to a sample during the experiment (sample is more than one) • Example: The combination of typeface: serif, character <A, B, C> and noise level <130, 140, 150>

Experiment Block • Set of experiment trials that provides a cover of the factor space that is appropriate and adequate for achieving the task objective

Block Design • What is the appropriate set of experiment trials • that provides an appropriate cover of the factor space • for the experiment?

Block Design • Three basic strategies: • Enumerated block design • Systematic block design • Randomized block design

Enumerated Block Design • Assigns every possible treatment to every sample • Obvious strategy if sample = 1 • If sample > 1, this is not practical • Because total number of trials = extent of factor space x number of sample • Too large !!

Systematic Block Design • Uses a deterministic algorithm to assign treatments to different sample in a systematic way • Eventually covers the entire factor space • Problem: Unintentional resonance between sample and treatment can be sparked

Systematic Block Design • Example: • A marketing survey is carried out to every 100th telephone number • The chances a treatment assigned to, say a number 03-2698 1100 belonging to a business entity • Is higher than say 03-2698 1024 • A bias towards response of business entity may occur in the survey

Systematic Block Design • This block design should be avoided • Unless this bias can be ascertained

Randomized Block Design • Similar to systematic block design • Except that the treatment assigned to the sample are sequenced randomly • This can also reduce the risk of systematic bias

Case Study • Enumerated block design is not practical • Total trials = 864 x 14! • Each sample has to respond to 864 treatments! • Fatigue • ‘Peak Performance’

Case Study • Another disadvantage: • Humans are smart • Easily guess that factor space include 10 decimal digits and 26 Latin characters • Guess from elimination process • Bias the results

Case Study • Since all the license plate inspectors had same recognition skills • Not all treatment need to be assigned to every sample/ subject • Divide 864 treatments equally • Reduce time for each subject • Can a systematic block design do it?

Case Study • Systematic block design also has setbacks • Subjects can also detect the periodicity • Biased improved performance • Randomized block design is solution • Computer generated pseudo-random assignment of treatments

Case Study • Decide how many and which sets of treatments would be randomly assigned to subjects • Combined to cover enough sets for each factor • To make up set of trials that cover entire factor space

Representation Factor • A factor that is not intended as a basis for measuring performance • However they are necessary for assigning values of a parameter • Example: • Characters

Performance factor • A factor that is used as a basis for measuring performance • Example: • Typeface • Noise Level

Case Study • Either assign each of two typefaces to half the subjects • Or assign both typefaces to all • Former solution better to avoid confusion among subject and more realistic

Case Study • Noise Level range 130 to 240 with increments of 10 • How to distribute the treatment to subjects? • Condition: • Interval must be same • Subsets different • Same average

Research Methodology