Some Central Issues in Social Research

1. Some Central Issues in Social Research • Description versus Explanation (notes follow Punch, Ch. 2) • Goal of scientific inquiry is not just to describe phenomena, but to explain them and to make predictions based on the patterns that are uncovered and their putative causes or at least their antecedents • The purpose of description is to give an account of the phenomenon in order to improve understanding and reduce complexity by extracting the features necessary to achieve understanding. The “what” • The purpose of explanation is to locate the causal underpinnings of events; to create a “story” in which the reasons for things or the enabling or antecedent conditions which give rise to them are laid out. The “why” and/or the “how”.

2. Description versus Explanation • Presumably if we know what happened, why it happened, and how it happened, we will be able to • Predict under what circumstances it might happen again and • Possibly prevent/encourage it from happening again by removing/encouraging motivating or enabling conditions. • Although generally speaking purely descriptive studies have less appeal to journal editors, descriptive research is important • When the phenomena of interest are new and unexplored, or • When the researcher is attempting to isolate causal factors for testing with confirmatory methods • Description is of course the essential work of ethnography

3. Description versus Explanation, continued • How does explanation relate to theory? Theory is a systematic, sometimes axiomatic effort to explain a phenomenon or group of phenomena. Both descriptive and explanatory studies have a role in the development of theory. • Theory verification vs. theory generation • Theory verification: from theory we derive hypotheses which are then tested. More likely to be quantitative • Theory generation: theory derived from the data we have collected and described, using a defined strategy such as grounded theory. More likely to be qualitative • Punch presents a model of the structure of scientific knowledge based on a “nomothetic” view (nomothetic refers to disciplines characterized by the search for universally applicable scientific laws). • In the model empirical generalizations are based on observed regularities in raw data, and the generalizations are “covered” by an explanatory theory.

4. Question-Method Connection • The choice of research method or combination of methods is tied to the type of question asked (although in practice the question is often shaped to the available resources including the researcher’s training and area of expertise, e.g., the “law of the hammer”, see also “methodolatry”) • Questions like “what sorts of messages are conveyed in ads featuring female athletes” are exploratory in nature and call for qualitative methods. • A question like “what is the relationship between gender of celebrity endorser and product type” calls for quantitative, correlational methods • A question like “which type of celebrity (athlete, musician, actor) is most effective in promoting a Democratic senatorial candidate” calls for quantitative, experimental methods with random assignment of subjects to conditions and systematic rotation of endorser type across messages

5. Question-Method Connection, continued • Other questions such as “what is the process by which celebrity endorsement leads to product purchase” might be best pursued through interview methods with structured recall leading to a narrative account of the trajectory from exposure to purchase. • The important thing to remember is that the questions come first and the methods follow (although methods can sometimes limit what can be studied) • The hard part is to figure out what the question is, precisely, that you want answered. When the question has been properly formulated the way to study it becomes much more clear.

6. Tight versus Loose Structure • How much structure is applied to the research ahead of time, with respect to • Research questions • Can range from explicitly stated hypotheses to research questions to completely exploratory work with no formulated research questions and only a loose conceptual framework, if any, for guidance • Research design • Can range from experimental designs with planned comparisons among conditions and control of potentially confounding variables, to quasi-experimental design to case studies and ethnographies • Data • Can range from ratio level measurement where the obtained data is coded into pre-established categories based on valid, reliable, established measures to inductive categorization based on categories which emerge from the researcher’s immersion in the data

7. Tight versus Loose Structure, continued • Typically a quantitative study will introduce the most explicit forms of these three sorts of structure (research questions, research design, data) , at the beginning or very early in the research process • Typically a qualitative study will seek to keep structure to a minimum, at least at the beginning, so that less obvious and more elusive patterns and structures can emerge • Similarly, theory verification research will tend toward the highly structured and theory generation research will be more loosely structured

8. Introduction to Research Methods • Notes from Williams and Monge, Chapters 1,2,3 • A research method “refers to the strategy, plan and activities undertaken to accomplish the research” • Often both narrative/descriptive or qualitative approaches and quantitative methods using statistics can be profitably employed to provide answers to questions about human communication. • In a study of “The Telegarden” done some years back one of our questions was how the members dealt with people who violated the social norms of the group which had formed. Both quantitative methods involving numerical comparisons of coded message posts and historical/descriptive analysis involving a critical incident helped us to further our understanding.

9. Uses of Quantitative Research • It doesn’t make much sense to talk about one approach as being superior to the other. • Both have their uses and can complement each other and lead to further interesting questions when the methods yield conflicting findings. • The important thing is to decide what circumstances might make application of quantitative methods appropriate. • Quantitative methods are appropriate when measurement can offer a useful description • We can say that someone is 6 feet tall, or we can say that she is “the tallest in the group”, or that she is “one of the tall ones” (interval, ordinal, nominal), or we could say that she is an “Amazon”, or someone who “towers over this researcher” It depends on what we want to do with the information. Sometimes we want to use very precise metrics, and sometimes we want to make comparisons, and sometimes we want to provide a subjective impression of an object of study in reference to ourselves.

10. Is Measurement Relevant, Useful, Possible? • Is measurement relevant and possible? • One of the questions we have to ask is at what level it might be possible to measure something. • Suppose we have a group of ten cat owners. What measures would we use to sort them out by how much they love their cats? Amount spent on vet bills? Time per day spent petting, grooming? • Could we order them from 1 to 10, where number 10 loves her cat ten times as much as number 1? • Could we order them but say only that number ten loves her cat more than number 9 by the same amount that number 2 loves her cat more than number 1? • Could we merely order them, but not say anything about how much they differ, pairwise? • Could we merely classify them into high, low and moderate love? Would a narrative description gained from interviews and/or observations be more useful?

11. Are there Generalizations to be Made and or Hypotheses to be Tested? • Quantitative methods are appropriate when there are statistical generalizations to be made/hypotheses to be tested • One of the reasons for using statistics is that we want to be able to generalize about the attitudes, beliefs, and behaviors of people on the basis of a set of observations that will probably be limited by various constraints including financial ones. • So we have to find out the answers to various questions, such as, • Am I able to describe precisely the population to which I want to generalize? • Do I have confidence that I can draw a sample that is large enough and representative enough to stand in for the population? • Is the variability within the sample so great that it doesn’t make sense to talk about central tendencies?

12. Are there Observable Differences? • Another reason for using statistics that we want to answer questions like, “is the tendency I observed in this group something distinctive, different from what one would expect just by chance?” or, are these two, or three, or some larger number of groups different in ways that exceed chance expectations? • Suppose the question were about the efficacy of a new drug. What sort of confidence would you have to have that the difference between a treatment and a control group not taking the drug was not due to chance before you would recommend the drug? Ten percent? Five percent? One percent? Less than that? • How much confidence would you have if the report about the drug’s efficacy were based on methods of interviews and participant observation (anecdotal reports of “feeling better”)? Would statistical hypothesis testing increase your confidence? Would it depend on how “feeling better” was measured?

13. Some Problems with Quantitative Studies • Typical problems in quantitative studies: • Validity; we may not be measuring what we think we are measuring. Subjects don’t see it as we do • Reliability; we may not have confidence that our “yardstick” is measuring in a consistent way • Convenience samples

14. Common Terms in Quantitative Research • Some terms that reflect the application of the scientific method in communication research practice: • Phenomenon: the object of study; the behaviors, beliefs, attitudes, characteristics, and their purported inter-relationships that we seek to describe, explain, and/or predict (e.g. How competent people are at using computers) • Variable: an observable property or characteristic that can be measured (operationalized) and is expected to vary across cases or observational instances, such as what people report about their ability to use computers, or, what computer skills people can demonstrate, etc. • Independent variable • Dependent variable • Measurement: a plan for operationalizing the variable, such as using a particular scale or observational technique (e.g. Durndell and Haag self-report measure of computer self-efficacy) • Data: the obtained values which the researcher ascribes to individual measured instances of the variable (the numbers researchers assign to the answers study subjects give to items on the self-efficacy scale)

15. Terms in Quantitative Research, continued • Population: the totality of cases which constitute the sphere within which the phenomenon is to be observed. Could be people in general, people in the US, college students, children under 12, could also be states, cities, animal shelters, department stores, countries, etc. • Sample: some portion of the population which is believed to be representative • Descriptive statistics: statistically derived values that represent the central tendencies and variability with a body of data • Sampling statistics: values used to make inferences about the characteristics of the population from which they were drawn, including the variation of the sample characteristics from corresponding population parameters

16. Elements of a Quantitative Study: The Problem • Typical sort of quantitative study as reported in a journal will feature the following elements: • Literature review leading to problem statement leading to research question or hypothesis • Research obliged to “follow the conversation”, summarize what is known, what is controversial or not known, and what remains to be done. • May formulate a general question or make specific predictions about what relationships among variables may be found. • Hypothesis ordinarily will be stated in terms of a specific operationalization of the variables of interest as well as the nature of their relationships

17. Elements of a Quantitative Study: Method • Method, including subjects, measures and procedures • Descriptive methods, where variables and their relationships are observed and described (seniors who have high scores on computer self-efficacy have high scores on social support) • Quasi-experimental (e.g. seniors who take a computer class have higher scores on social support after one year than those who don’t sign up (no random assignment, based on self-selection; no experimenter manipulation) • Experimental methods, where one or more of the variables are manipulated by the researcher to provide for systematic examination of their interrelationships. (same as above except half of the seniors have been randomly assigned to a six-month long computer class and half have not) • Importance of control, avoiding of confounding factors

18. Elements of a Quantitative Study, Method, continued • Subjects (Respondents, Participants); who constitutes a “case” in the study. Some studies may have other cases such as web sites, tv episodes, etc. • Materials: questionnaires, interview protocols, coding schemes, hardware, environmental manipulations, etc. measures, operationalizations, material circumstances of experimental conditions. • Procedures: Instructional set, repeated measures design, debriefing, etc.

19. Elements of a Quantitative Study: Results • Results • Reliability and validity assessments • Reliability : internal consistency of measure; alternatively, consistency of measurement over time with the same subject, case, instance. Test-retest; alpha coefficient • Validity: does the measure really assess what it claims it does? Internal validity (does the research design represent what it says it does?) External validity (can the study’s results be said to apply to the real world? Face, concurrent, predictive, construct (convergent validity and discriminant validityare two subtypes of construct validity-measures that should be related are, and that should not be related, theoretically, are not). • Validity implies reliability; but reliability does not imply validity

20. Elements of a Quantitative Study: Results, continued • Data analyses (performing calculations on obtained data, conducting statistical hypothesis testing). • Researcher is assumed to have made decisions beforehand as to the level of statistical error that will be tolerated, what sort of “tail” (one or two) the test will have if appropriate, etc.

21. Elements of a Quantitative Study: Discussion • Discussion • Conclusions • Limitations • Implications for future research

22. Fundamental Assumptions of Quantitative Research Steps in the research process (notes from Kendrick) • Specify research goals and devise one or more research questions • Review the literature • Formulate hypotheses: reduce problem to be studied to a testable and preferably falsifiable statement of a set of relationships among variables of interest • Measure and record • Analyze the data • Invite scrutiny

23. Sampling from Populations • Populations, elements of populations, and units of analysis • Population --all registered voters in US • Elements within populations – the individual registered voter • Units of analysis or units of observation—the individual, or perhaps the individual’s household, spouse, etc. • Sample: subset of population selected to represent the population from which you draw it • Probability sample: selected so that each element in the population has a known probability of being included in the sample • Sampling frame: list of elements in the population (e.g. list of all persons paying property taxes in Los Angeles County in 2008)

24. Sampling from Populations, cont’d • Types of sampling • Simple random sampling (random number table, computer program) • Systematic sampling with a random start; only the first element is selected at random; then every nth • Multistage sampling processes • Stratification; partitioning the sampling frame by factors such as gender, ethnicity etc before random or systematic sampling • Clustering, for ex., in absence of good sampling frame, obtain list of clusters of elements in the frame, e.g., neighborhoods, zip codes, city blocks, then randomly select clusters, then randomly select within clusters

25. Variables and their Relationships • Variables: properties that vary from person to person, such as age, gender, attitudes toward politics, self-monitoring, etc., or that vary within the same element or unit of analysis over time. • Independent variable: assumed to be causal (logic, time order, ascribed vs. achieved characteristics) • Dependent variable: assumed to be an effect or result of one or more other variables • Variables have categories (e.g., M/F; Caucasian non-Hispanic/African-American, Hispanic, etc.;) and numbers or values can be assigned to these • Relationships between variables • Control variables; some variable other than the independent or dependent variable which is thought to influence their relationship; e.g., effect attributable to ethnicity may actually be due to the correlation of religion and ethnicity • Symmetrical relationship; variables are correlated but it is not possible to specify the direction of a causal relationship

26. Independent and Dependent Variables: Which is Which? • Independent variables must be chronologically prior to dependent variables • Ascribed characteristics (inherited, properties over which one has little/no control, such as height or eye color or ethnicity are usually independent variables (although not always) • Achieved or acquired characteristics (attitudes, values, beliefs, etc) are frequently dependent variables, though not always (consider health behaviors, for example, in which attitudes and beliefs play a major role in health information seeking, compliance with physician’s instructions, etc.)

27. Levels of Measurement • Categories, Values, Data (e.g., Category =Gender: Values: 1=Female, 2=Male; Data: respondent #22=2) • Levels of Measurement (Kendrick, Chapter 2; Williams and Monge, Chapter 3). The level at which a variable is measured determines the kinds of statistics which can be used to describe or make inferences about the variable. Generally, researchers use numbers to classify or categorize, to rank or order, or to assign a score or rating • Discrete vs. Continuous Variables • Discrete: variables countable in quantities that can’t be reduced to smaller units, such as no. of children, jobs you hold, movies you attend weekly. • Continuous: countable in quantities that can be reduced indefinitely, such as time. A variable may be theoretically continuous such as age but in practice it will be measured as discrete because of available metrics • (smallest age measurement usually in hours or days); same with attitudes or other ratio type variables that are treated in practice as discrete

28. Levels of Measurement, continued • Categorical vs. Numerical • Categorical: categories predetermined by researcher • Exclusivity: answer categories should not overlap • Exhaustivity • Examples (demographic (ethnicity, marital status, income categories), Likert-type attitude scales, semantic differential), frequency measures (once a week, once a month, etc.) • Types of categorical variables • Nominal: no rank order implied; categories simply discrete (gender, ethnicity, religion, etc). Considered weakest level of measurement • Ordinal: rank order implied, for example, when age, income, education level, etc. are measured not numerically but in categories where one end of the range is “higher” for the researcher’s purposes. There is no information about the magnitude of differences. Often ordinal scaled data like Likert or semantic differential data is treated in practice as if the intervals between response categories were equal so more complex statistical analysis can be applied • Numerical: direct measurement in terms of numbers, amounts, frequencies with no pre-conceived categories • Examples: hours spent on the Internet weekly, absolute salary, age in years, miles between cities, scores on some aptitude, IQ and personality tests, etc.

29. Levels of Measurement, continued • Types of numerical variables: • Ratio level; numbers are assigned to identify ordered relations with respect to some property ; there is an absolute zero amount of the property; proportions are meaningful (e.g. travel in miles between two points, where zero means the two points are the same, and 500 miles is twice as far as 250 miles, thus being isomorphic with relations in the real, physical world). (But- on a test, 0% correct may not mean no amount of the property measured by the test unless the test has a true random sample of all possible indicators of the property) • Interval level; numbers are assigned to identify ordered relations with respect to some property, but no absolute zero; proportions not meaningful, but distances are assumed to be equidistant between scale intervals. For many communication variables, it is not reasonable to think of “zero” amount of the property. What is zero amount of self-monitoring? Of shyness? etc

30. Levels of Measurement, cont’d • Categorical data can be treated numerically if the researcher believes the categories are roughly equidistant (e.g. strongly agree, agree distance is same as agree, neutral, and same as strongly disagree, disagree) • Numerical data can be treated as categorical. For example, income, age, hours watching TV data can be collected categorically for various reasons • The level of measurement determines the type of statistical tests and procedures which can be applied to the data. Statistics appropriate for numerical data (ratio, interval) are not appropriately applied, for example, to nominal or ordinal data. But data can be “scaled back” (e.g. taken back to a lower level of measurement) and statistical procedures appropriate to, say, ordinal data can be applied (e.g. change hours worked per week to categories such as none, some, lots, many)

31. Hypotheses and Research Questions • Hypotheses: hunches or expectations that social scientists have about relationships between or among variables, commonly but not always expressed as the expectation that variation in an independent variable will “cause” or be associated with variation in a dependent variable. Some examples from recent papers: • “Local Mexican Web pages will reflect a higher level of collectivism than the Mexican Web pages of American companies” • “Internet users' concern for privacy online negatively influences their trust of a commercial Web site” • Hypotheses look different from research questions: • RQ1. Does subject gender interact with medium in the development of relational intimacy, and if so, how do these variables interact?