CGT 411 Research Presentation

1. CGT 411 Research Presentation Conducting Research: What Do You Need to Think About? Part 3 – Instruments and Participants

2. The Nature of Sampling • The idea is to examine elements of a population in order to draw conclusions about the population as a whole • The (experimental) population is the group of elements about which we want to make inferences • A count of all elements of a population would be known as a census • Sample elements = data points

3. Why do we sample? • Lower cost  it is normally too expensive to look at everyone • Greater accuracy  fewer data points means less to keep track of • Faster  this greatly reduces cost • Availability of population elements  every population element is often not accessible • A census is feasible when populations are small and necessary when elements are not homogeneous

4. Overview of Research • Good research should follow the standards of the scientific method • Purpose clearly defined • Research process detailed • Research design thoroughly planned • Limitations frankly revealed • High ethical standards applied • Adequate analysis for decision-maker’s needs • Findings presented unambiguously • Conclusions justified • Researcher’s experience reflected

5. Exploratory Research • Interviewing • Observation • Extant data • Document analysis • Case studies • Ethnography • Expert interviewing • Secondary data analysis • Experience surveys • Focus groups • Two-stage designs Instances of data should be analyzed that provide meaningful information within the context of the organization

6. Communicating with Participants • Observations • Interviews • Surveys

7. Getting to the Measurement Questions • What type of data is needed to answer the investigator’s dilemma? • What communication approach will be used? • Should the questions be structured, unstructured, or some combination? • Administrative: identify setting characteristics • Classification: sociological/demographic • Target: address the issues at hand • Should the questions be disguised or undisguised?  have to be careful here  potential IRB issues

8. Constructing and Refining Research Questions • Measurement questions should: • Consider subject content • Consider the wording of each question • Consider the response strategy • Question development and question sequencing are typically completed at the same time

9. Drafting and Refining the Instrument • The introduction has to hook ‘em and give them the particulars  also has to filter them • Some questions depend on other having or not having been asked already • Awaken interest and motivation • Put sensitive questions at the end • Balance complexity of questions and their location • General to specific • Group questions to minimize shifting frame of reference and subject matter

10. Drafting and Refining the Instrument • Pre-testing/pilot-testing is valuable for having a reliable instrument • Discover participants’ reactions to the instrument • Ensure meaning is conveyed • Why do participants “transform” questions they do not understand? • Improve continuity and flow of the instrument • Make sure question sequence is appropriate • Helps understand variability in population • Helps to gauge length and time

11. Interview Considerations • Should the question be asked? • Is the question of the proper scope and coverage? • Can the participant adequately answer the question as it is asked? • Will the participant willingly answer the question as it is asked?

12. Observation Considerations • Much of what we know comes from observation  but the collection process often leaves much to be desired • Scientifically based • Conducted to answer a specific question • Systematically planned and executed • Uses appropriate control mechanisms • Reliable and valid account of what happened • Observation is generally behavioral • Nonverbal analysis: body movement • Linguistic analysis: patterns/content of speech • Spatial analysis: physical relationship to surroundings • Extra-linguistic analysis: characteristics of speech

13. What is Experimentation? • The emphasis is towards discovering a causal relationship • Accepting the world as it is found (ex post fact, communication, observation) vs. systematically altering a variable and observing the change (experimentation) • Is the experimental treatment at the heart of the observed change, or are other factors influencing the outcome?

14. What is Experimentation? • Requires the intervention of a researcher beyond that required for measurement • Manipulation of independent variables to affect a hypothesized dependent variable  atleast one IV and one DV in an experiment • Must be an agreement between variables  presence/absence of one relates to the presence/absence of another • Time order of variables must make sense • Researchers must be confident that extraneous factors did not cause such a change • Often involves the presence of a contrived research scenario  standardized to aid generalizability

15. Elements of Scientific Method • Direct observation of phenomena • Clearly defined variables, methods, and procedures • Empirically testable hypotheses • Ability to rule out rival hypotheses • Statistical justification of conclusions • Self-correcting process

16. Descriptive Study • Descriptions of phenomena associated with a particular population • Estimates of the proportion of the population that would have such characteristics • Discovery of associations among different variables The simplest descriptive study concerns a statement about the size, form, distribution, or existence of a variable  no causal relationship implied

17. Causal Studies • Correlation is not causation essentially “A produces B”  an artifact of language • We cannot prove that A causes B, but we can find evidence that leads us to believe that A leads to B • Covariation of A and B • Time order of events in the direction specified • No other possible causes • Inference-making also requires: • Control no other confounding causes • Random assignment along the IV

18. Researcher’s ability to manipulate the IV Influence of extraneous variables can be controlled Convenient and cost-effective Ability to replicate across people, places, and time Can use naturally occurring events Contrived setting Generalization from non-probability samples can be problematic Budgets can be deceiving Most effective for problems in the present or the immediate future Ethical issues in the manipulation of people Advantages Disadvantages

19. Conducting an Experiment • Select relevant variables • Specify levels of treatment • Control the experimental environment • Choose the experimental design • Select and assign the participants • Pilot test  revise  test • Analyze the data

20. The Logic of Hypothesis Testing • The null hypothesis  a statement that nosignificant difference exists between a parameter and a statistic being compared to it. • Why not state this in a positive form? • Evidence would preclude a positive hypothesis from being taken as conclusive evidence for accepting the hypothesis • Could be consistent with other hypotheses as well

21. The Logic of Hypothesis Testing • The alternative hypothesis  a statement that there is a significant difference between the parameter and the statistic • This is the logical opposite to the null hypothesis • May take several forms depending on the objective of the researcher • Not equal to (not the same as) • Greater than • Less than