Designing quantitative studies

1. Designing quantitative studies Dr. Aidah Abu Elsoud Alkaissi An-Najah National University Faculty of Nursing

2. The research design of a study spells out the basic strategies that researchers adopt to develop evidence that is accurate and interpretable. • The research design incorporates some of the most important methodologic decisions that researchers make, particularly in quantitative studies. • Thus, it is important to understand design options when embarking on a research project.

3. TIP: If you are doing a study, you will need to make many important decisions about the study’s design. • These decisions will affect the overall believability of your findings. • In some cases, the decisions will affect whether you receive funding (if you are seeking financial support for your study) or whether you are able to publish your research report (if you plan to submit it to a journal). • Therefore, a great deal of care and thought should go into these decisions.

4. ASPECTS OFQUANTITATIVERESEARCH DESIGN • Intervention • A fundamental design decision concerns the researcher’s role vis-à-vis study participants. • In some studies, nurse researchers want to test the effects of a specific intervention (e.g., an innovative program to promote breast self-examination). • In such experimental studies, researchers play an active role by introducing the intervention. • In other studies, referred to as nonexperimental studies, the researcher observes phenomena as they naturally occur without intervening.

5. Comparisons • In most studies, researchers develop comparisons to provide a context for interpreting results. • The most common of types of comparison are as follows: • 1. Comparison between two or more groups. For example, suppose we wanted to study the emotional consequences of having an abortion. • To do this, we might compare the emotional status of women who had an abortion with that of women with an unintended pregnancy who delivered the baby. • 2. Comparison of one group’s status at two or more points in time.

6. For example, we might want to assess patients’ levels of stress before and after introducing a new procedure to reduce preoperative stress. • Or we might want to compare coping processes among caregivers of patients with AIDS early and later in the caregiving experience. • 3. Comparison of one group’s status under different circumstances. For example, we might compare people’s heart rates during two different types of exercise.

7. 4. Comparison based on relative rankings. If, for example, we hypothesized a relationship between level of pain of cancer patients and their degree of hopefulness, we would be asking whether patients with high levels of pain feel less hopeful than patients with low levels of pain. • This research question involves a comparison of those with different rankings—high versus low—on both variables.

8. Comparison with other studies. Researchers may directly compare their results with results from other studies, sometimes using statistical procedures. • This type of comparison typically supplements rather than replaces other types of comparisons. • In quantitative studies, this approach is useful primarily when the dependent variable is measured with a widely accepted approach (e.g., blood pressure measures or scores on a standard measure of depression).

9. Example of using comparative data from other studies: • Beckie, Beckstead, and Webb (2001) studied quality of life and health of women who had suffered a cardiac event. • Women in their sample were administered standard scales for which there were national comparison data, enabling the researchers to evaluate their sample’s outcomes relative to national norms in the United States.

10. Comparisons are often the central focus of a study, but even when they are not, they provide a context for understanding the findings. • In the example of studying the emotional status of women who had an abortion, it would be difficult to know whether their emotional status was of concern without comparing it with that of others.

11. In some studies, a natural comparison group suggests itself. • For example, if we were testing the effectiveness of a new nursing procedure for a group of nursing home residents, an obvious comparison group would be nursing home residents who were exposed to the standard procedure rather than to the innovation

12. In other cases, however, the choice of a comparison group is less clearcut, and the researcher’s decision about a comparison group can affect the interpretability of the findings. • In the example about the emotional consequences of an abortion, we opted to use women who had delivered a baby as the comparison group. • This reflects a comparison focusing on pregnancy outcome (i.e., pregnancy termination versus live birth). An alternative comparison group might be women who had a miscarriage. • In this case, the comparison focuses not on the outcome (in both groups, the outcome is pregnancy loss) but rather on the determinant of the outcome.

13. Thus, in designing a study, researchers must choose comparisons that will best illuminate the central issue under investigation.

14. Controls for Extraneous Variables • the complexity of relationships among human characteristics often makes it difficult to answer research questions unambiguously unless efforts are made to isolate the key research variables and to control other factors extraneous to the research question. • Thus, an important feature of the research design of quantitative studies is the steps that will be taken to control extraneousn variables. • Familiarity with the research literature often helps to identify especially important variables to control.

15. Timing of Data Collection • In most studies, data are collected from participants at a single point in time. • For example, patients might be asked on a single occasion to describe their health-promoting behaviors. • Some designs, however, call for multiple contacts with participants, usually to determine how things have changed over time.

16. Thus, in designing a study, the researcher must decide on the number of data collection points needed to address the research question properly. • The research design also designatesn when, relative to other events, data will be collected. • For example, the design might call for interviews with pregnant women in the sixteenth and thirtieth weeks of gestation, or for blood samples to be drawn after 10 hours of fasting.

17. Research Sites and Settings • Research designs also specify the site and setting for the study. • sites are the overall locations for the research, and settings are the more specific places where data collection will occur. • Sites and settings should be selected so as to maximize the validity and reliability of the data. • In designing a study, it may be important to consider whether participants are influenced by being in settings that may be anxiety-provoking or foreign to their usual experiences.

18. Communication With the Subjects • In designing the study, the researcher must decide how much information to provide to study participants. • As discussed in the previous chapter, full disclosure to subjects before obtaining their consent is ethically correct, but can sometimes undermine the value of the research. • The researcher should also consider the costs and benefits of alternative means of communicating information to study participants.

19. Among the issues that should be addressed are the following: • How much information about the study aims will be provided to (and withheld from) prospective subjects while they are being recruited and during the informed consent process? • How will information be provided—orally or in writing? • What is the reading and comprehension level of the least skilled participants? • Who will provide the information, and what will that person be expected to say in response to additional questions participants might ask?

20. Will there be debriefing sessions after data are collected to explain more fully the study purpose or to answer questions? • The nature of the communication with participants can affect their cooperation and the data they provide, and so these issues should be given careful consideration in designing the study.

21. OVERVIEW OF RESEARCHDESIGN TYPES • Between-Subjects and Within-Subjects Designs • As previously noted, most quantitative studies involve making comparisons, which are often between separate groups of people. • For example, the hypothesis that the drug tamoxifen reduces the rate of breast cancer in high-risk women could be tested by comparing women who received tamoxifen and those who did not.

22. In this example, those getting the drug are not the same people as those not getting it. • In another example, if we were interested in comparing the pain tolerance of men and women, the groups being compared would obviously involve different people. • This class of design is referred to as between-subjects designs.

23. It is sometimes desirable to make comparisons for the same study participants. • For example, we might be interested in studying patients’ heart rates before and after a nursing intervention, or we might want to compare lower back pain for patients lying in two different positions. • These examples both call for a within-subjects design, involving comparisons of the same people under two conditions or at two points in time. • The nature of the comparison has implications for the type of statistical test used.

24. The Time Dimension • Although most studies collect data at a single point in time, there are four situations in which it is appropriate to design a study with multiple points of data collection:

25. Studying time-related processes. • Certain research problems specifically focus on phenomena that evolve over time (e.g., healing, learning, recidivism, and physical growth).

26. 2. Determining time sequences. • It is sometimes important to determine the sequencing of phenomena. • For example, if it is hypothesized that infertility results in depression, then it would be important to determine that the depression did not precede the fertility problem.

27. 3. Developing comparisons over time. • Some studies are undertaken to determine if changes have occurred over time. • For example, a study might be concerned with documenting trends in the smoking behavior of teenagers over a 10-year period. • As another example, an experimental study might examine whether an intervention led to both short-term and long-term effects.

28. 4. Enhancing research control. • Some research designs for quantitative studies involve the collection of data at multiple points to enhance the interpretability of the results. • For example, when two groups are being compared with regard to the effects of alternative interventions, the collection of data before any intervention occurs allows the researcher to detect—and control—any initial differences between groups. • Studies are often categorized in terms of how they deal with time. • The major distinction is between cross-sectional and longitudinal designs.

29. EXPERIMENTS • A basic distinction in quantitative research design is that between experimental and nonexperimental research. • In an experiment, researchers are active agents, not passive observers. • Early physical scientists learned that although pure observation of phenomena is valuable, complexities occurring in nature often made it difficult to understand important relationships.

30. This problem was handled by isolating phenomena in a laboratory and controlling the conditions under which they occurred. • The procedures developed by physical scientists were profitably adopted by biologists during the 19th century, resulting in many achievements in physiology and medicine. • The 20th century has witnessed the use of experimental methods by researchers interested in human behavior.

31. Characteristics of True Experiments • The controlled experiment is considered by many to be an ideal—the gold standard for yielding reliable evidence about causes and effects. • Except for purely descriptive research, the aim of many research studies is to understand relationships among phenomena.

32. For example, does a certain drug result in the cure of a disease? Does a nursing intervention produce a decrease in patient anxiety? • The strength of true experiments lies in the fact that experimenters can achieve greater confidence in the genuineness of causal relationships because they are observed under controlled conditions. • hypotheses are never proved or disproved by scientific methods, but true experiments offer the most convincing evidence about the effect one variable has on another.

33. A true experimental design is characterized by the following properties: • Manipulation—the experimenter does something • to at least some subjects • Control—the experimenter introduces controls over the experimental situation, including the use of a control group • Randomization—the experimenter assigns subjects to a control or experimental group on a random basis

34. Each of these features is discussed more fully in the following sections. • Manipulation involves doing something to study participants. • The introduction of that “something” (i.e., the experimental treatment or intervention) constitutes the independent variable. • The experimenter manipulates the independent variable by administering a treatment to some subjects and withholding it from others (or by administering some other treatment). • The experimenter thus consciously varies the independent variable and observes the effect on the dependent variable

35. For example, suppose we hypothesized that gentle massage is effective as a pain relief measure for elderly nursing home residents. • The independent variable (the presumed cause) in this example is receipt of gentle massage, which could be manipulated by giving some patients the massage intervention and withholding it from others. • We would then compare patients’ pain level (the dependent variable) in the two groups to see if differences in receipt of the intervention resulted in differences in average pain levels.

36. Control • Control is achieved in an experimental study by manipulating, by randomizing, by carefully preparing the experimental protocols, and by using a control group. • This section focuses on the function of the control group in experiments.

37. Obtaining evidence about relationships requires making at least one comparison. • If we were to supplement the diet of premature infants with a particular nutrient for 2 weeks, their weight at the end of 2 weeks would tell us nothing about treatment effectiveness. • At a bare minimum, we would need to compare their posttreatment weight with their pretreatment weight to determine if, at least, their weight had increased. But let us assume that we find an average weight gain of 1 pound.

38. Does this gain support the conclusion that the nutritional supplement (the independent variable) caused weight gain (the dependent variable)? No, it does not. • Babies normally gain weight as they mature. • Without a control group—a group that does not receive the nutritional supplements—it is impossible to separate the effects of maturation from those of the treatment. • The term control group refers to a group of subjects whose performance on a dependent variable is used to evaluate the performance of the experimental group or treatment group (the group that receives the intervention) on the same dependent variable

39. Randomization • Randomization (also called random assignment) • involves placing subjects in groups at random. • Random essentially means that every subject has an equal chance of being assigned to any group. • If subjects are placed in groups randomly, there is no systematic bias in the groups with respect to attributes that could affect the dependent variable.

40. Let us consider the purpose of random assignment. • Suppose we wanted to study the effectiveness of a contraceptive counseling program for multiparous women who have just given birth. • Two groups of subjects are included—one will be counseled and the other will not.

41. The women in the sample are likely to differ from one another in many ways, such as age, marital status, financial situation, attitudes toward child-rearing, and the like. • Any of these characteristics could affect a woman’s diligence in practicing contraception, independent of whether she receives counseling.

42. The women in the sample are likely to differ from one another in many ways, such as age, marital status, financial situation, attitudes toward child-rearing, and the like. • Any of these characteristics could affect a woman’s diligence in practicing contraception, independent of whether she receives counseling.

43. We need to have the “counsel” and “no counsel” groups equaln with respect to these extraneous characteristics to assess the impact of the experimental counseling program on subsequent pregnancies. • The random assignment of subjects to one group or the other is designed to perform this equalization function. • One method might be to flip a coin for each woman (more elaborate procedures are discussed later). If the coin comes up “heads,” the woman would be assigned to one group; if the coin comes up “tails,” she would be assigned to the other group.

44. Although randomization is the preferred scientific method for equalizing groups, there is no guarantee that the groups will, in fact, be equal. • Asan extreme example, suppose the study sample involves 10 women who have given birth to 4 or more children. • Five of the 10 women are aged 35 years or older, and the remaining 5 are younger than age 35.

45. We would expect random assignment to result in two or three women from the two age ranges in each group. • But suppose that, by chance, the older five women all ended up in the experimental group. • Because these women are nearing the end of their childbearing years, the likelihood of their conceiving is diminished.

46. Thus, follow-up of their subsequent childbearing (the dependent variable) might suggest that the counseling program was effective in reducing subsequent pregnancies; • however, a higher birth rate for the control group may reflect only age and fecundity differences, not lack of exposure to counseling.

47. Despite this possibility, randomization remains the most trustworthy and acceptable method of equalizing groups. • Unusual or deviant assignments such as this one are rare, and the likelihood of obtaining markedly unequal groups is reduced as the number of subjects increases.

48. You may wonder why we do not consciously control those subject characteristics that are likely to affect the outcome. • The procedure that is sometimes used to accomplish this is known as matching.

49. For example, if matching were used in the contraceptive counseling study, we might want to ensure that if there were a married, 38-year-old woman with six children in the experimental group, there would be a married, 38-year-old woman with six children in the control group as well.

50. There are two serious problems with matching, however. • First, to match effectively, we must know (and measure) the characteristics that are likely to affect the dependent variable, but this information is not always known. • Second, even if we knew the relevant traits, the complications of matching on more than two or three characteristics simultaneously are prohibitive