1. CHAPTER 12 Descriptive, Program Evaluation, and Advanced Methods

2. Descriptive Research • Descriptive research designs typically involve the researcher observing and systematically recording some particular behavior(s) for the purpose of better describing the behavior in question. • Some common descriptive designs include the following: • Large-Scale Surveys • Archival Data Methods • Naturalistic Observation • Structured Test Methods

3. Large-Scale Surveys • Large-scale surveys are conducted to describe the behaviors or characteristics of some population. • They basically involves lists of questions that participants are asked to answer. • Because of its purpose large-scale surveys must have representative samples. Why? • Advantages: • Easy to conduct • Generate lots of data in a short period of time • Can be on virtually any topic • Disadvantages: • Nonresponse bias – many participants do not respond. • Social Desirability bias: participants answer questions in ways they think are socially acceptable. • The data collected can be analyzed by a variety of methods, but most often simple descriptive statistics are used (e.g., percentages, frequencies).

4. Archival Research • Archival records are documents that can be typically found in university libraries and government buildings. • Some common types of archival information that psychologists may be interested in include: • Data on birth weights and rates • Death rates and causes of death • Weather reports • Crime rates • Sales of certain products • And many more! • Archival research often involves content analysis – the objective description, classification and evaluation of previous research. Thus, archival research is considered descriptive in nature.

5. Naturalistic Observation • Naturalistic observation is another descriptive method where a researcher observes subjects’ behavior in a “natural” or normal setting without trying to influence or interfere with the behavior. • Advantages: • Behavior is observed in a natural setting • Free from reactivity effects • Disadvantages: • Can take a very long time • No knowledge or control of events that cannot be seen. • Experimenter bias

6. Structured Test Methods • Structured test methods are descriptive methods that involve setting up or structuring some test situation to determine how individuals behave in that situation. • The structured test situation is designed to either increase or decrease the likelihood of a certain behavior. • Also structured tests help to increase the frequency of the behavior of interest – making it easier to study.

7. Case Study • We learned in chapter 11 that case studies were considered intrasubject designs because they were typically performed on one or a few participants. • Case studies can also be considered descriptive research since case reports involve careful systematic observation of the participant.

8. Program Evaluation • Program evaluation is a type of research that typically involves assessing the need for and success of a particular program. • The programs we are talking about here are generally large scale – involving institutions like school boards or governments. • The outcomes of these programs often have significant implications to various groups of people. • The goals of program evaluation are to provide administrators unbiased information and feedback pertaining to the proposed or current program. • This information can be used to begin a new program, cancel an old program, or to restructure a current program to make it more efficient.

9. Program Evaluation: Needs Analysis • A needs analysis is a study that attempts to identify: • If a problem exists • If so, how many people (or animals) are affected, • What factors will help reduce of eliminate the problem • And how many people will use the program aimed to fix the problem. • A needs analysis is almost exclusively a descriptive research strategy. Methods typically employed involve: • Archival data (census, public records) • Surveys

10. Program Evaluation: Process Analysis • A process analysis examines whether a currently operating program is: • Known to exist and being utilized by the target population • Being implemented and structured properly • Data are typically gathered through surveys, staff records, and observation of program in action. • Thus process analysis is descriptive in nature.

11. Program Evaluation: Outcome Analysis • An outcome analysis assesses the impact or results of a program. • Is the program achieving its goal? • The success of a program can be accomplished by various means: • Descriptive strategies would base evaluations on frequencies and percentages acquired before and after the program. • Quasi-experimental strategies would use t-tests and ANOVA to compared means before and after the program. • Experimental strategies would also use t-tests and ANOVA but would have a control group, and equivalent control and experimental groups.

12. Program Evaluation: Efficiency Analysis • If the outcome analysis shows that a program works, administrators now want to how efficient the program is. • Efficiency analysis is usually deals with the monetary aspect of the program and evaluates the programs costs relative to its benefits. • The data generated here is often in ratio form, e.g., cost-benefit ratio.

13. Advanced Methods: • Meta-Analysis • A meta-analysis is a statistical method that combines the results of previous studies (on any given topic) to provide an integrated overview of those previous findings. • Two common approaches used in meta-analysis include: • Combined probability • Combined estimation

14. Advanced Methods: • Multivariate Analyses • Multivariate analyses are a group of statistical procedures that typically involves either: • Two or more dependent variables • Advanced correlational methods with one dependent variable and multiple independent variables. • Some examples of multivariate analysis: • Multivariate Analysis of Variance (MANOVA) • Structural Modeling and Path Analysis