Methods of Observation

1. Methods of Observation PS 204A, Week 2

2. What is Science? Science is: • Based on natural laws/empirical regularities. • Makes predictions. • Collections of laws that generate predictions that are empirically confirmed constitute “explanations.” • Must be falsifiable. • Is always tentative (move from grossly wrong to more subtly wrong theories).

3. The Scientific Enterprise

4. Theory Analogy • Theories always possess a “theoretical notion” or analogy that simplifies reality. • This analogy is embodied in the assumptions or premises of the theory. Assumptions are themselves unobservable – and known to be simplifications (e.g., individuals are rational, states are unitary actors). Prefer plausible over less plausible premises. • Since premises are never “true” or, at least, are unobservable, theories are never true, only more or less useful. • Utility is defined by the number of empirically supported propositions the theory generates.

5. Plausibility of Premises • If in theories, premises are things we do not agree on and are unobservable, how do we assess their plausibility? • Utility of their predictions (Friedman). • Accordance with natural laws. • Transform premises into objects of investigation that are themselves the subjects of theories. • Theory of rationality. • Theory of unitary states.

6. Science is a series of “boxes within boxes” • Balance of power theory: international system is anarchic and composed of unitary states wishing only to survive. Within any given “box,” we take the premises as “given.” But any premise may itself become an object of investigation in another “box.” Internal hierarchy: state “speaks” with a single voice Unitary states wishing to survive Anarchy: no common authority If testing leads to a revision of a premise, then still need to plug back into original theory and retest. Act to check the power of other states.

7. Hypotheses • Propositions are general statements that follow logically from the premises. • Hypotheses are propositions that contain only observable variables (i.e., if X, then Y, when both X and Y can be observed). • Central issue is deductive validity: Does the hypothesis follow logically and axiomatically from the premises? Deductive Validity

8. Tests • We test theories by examining whether the hypotheses they generate are supported by the evidence. We make the observations the theories imply. • Conclusion validity: is there a relationship between X and Y? • Internal validity: is the relationship causal? • Construct validity: do the observable measures capture concepts in the theory appropriately? 2. Internal Validity Test 3. Construct Validity 1. Conclusion Validity

9. Explanation v. Prediction • Theory offers an explanation for observed facts and predicts new facts that, once confirmed, are also explained. • Theories must be potentially falsifiable. Popper/Hempel insist that known facts cannot falsify a theory. Therefore, prediction is the goal of all science. • Alternatively, Snyder argues that if scientific evidence is objective, evidence is evidence independent of the timing of it’s discovery relative to the theory.

10. Who’s Right? • All evidence helps corroborate a theory, even known facts. • Predictions are more “valuable” than explanations in providing evidence for a theory.

11. Generalization Possible Refinements • Can we generalize our observations to larger populations? • Key issue here is external validity (i.e., will conclusions hold for other people at other times). • Testing may lead us to refine our theories further, propelling the cycle another round. • Science is interactive. Tests suggest refinements to theories, which then generate new predictions and tests. Conversation between theory and evidence. 4. External Validity

12. Deductive v. Inductive Reasoning • Hypothetico-Deductive method begins with theory, then generates tests. From observations, draw inferences about theory, which in turn lead to generalizations about unobserved populations. • Inductive approach begins with observations and draws draw inferences about unobserved populations. May or may not lead to theory. • Induction can be science: body of replicated and confirmed laws that are predictive. But, falsifiablility is always an issue.

13. What can we learn by “observing”?

14. Observation Inference • Observation central to both deduction and induction. • How do we draw inferences about unobservable phenomena, premises, or populations from observable phenomena? • How do we learn about what we can’t see, from what we can? • Applies equally to inherently unobservable traits (what goes on in people’s heads), future events (predictions), and true populations (alternative worlds).

15. Descriptive Inference: Or how do we know what we saw? • What is this a case of? What is the class of which you observe one or more members? • Many categories are question or theory dependent. • If observation is unique, no generalizations are possible. • If observation is exhaustive (all members of class), what can you generalize to?

16. Descriptive Inference II • Probabilistic v. Deterministic Events: all events have systematic and non-systematic components. • Probabilistic events occur with some probability < 1.0; if replayed under identical conditions, the observed result would differ (more or less). On average, would get same result (non-systematic component is random). • Deterministic events occur with certainty (p = 1.0). Special case in which non-systematic component is zero.

17. Descriptive Inference III • With a probabilistic event, how are we to classify a particular instance? • What should we infer from a war in which country A loses? We observe the loss, but can we generalize to other similar cases? If the probability of victory was .25, what can we infer from the actual loss? • Inference is harder the smaller the number of cases. • Problem arises when we mistake a probabilistic event for a deterministic event. • Incorrect to infer any “pathology” or “mistake” in such instances.

18. Induction I: Empirical Laws • Analysis limited to observable phenomena only. • An empirical law is a robust “regularity” (e.g., the democratic peace) • By extending empirical laws, we can make predictions (inferences) about future events. • But, • Concepts do not exist independent of theory. • Correlations may be spurious. • Correlation does not equal causation. We may “explain” events by empirical laws but such laws do not imply cause or constitute a causal test.

19. Induction II: Thick Description • Inferring unobservable from observable phenomena. • “Thick description” uses observables to learn about unobservables. • Geertz in Negara studies puppet theatre in Bali to gain insight into political culture in that society. • “Justifications” as means of learning about social norms. • At this level, does not aim to explain outcomes, only to describe unobservable traits.

20. Induction III: Interpretation • As in thick description, use observables to infer unobservable phenomena, typically motives, intent, purpose. Then explain observed outcomes in terms of the actor’s self-understanding. • Geertz: “not an experimental science in search of law but an interpretive one in search of meaning.” • When the choice by the actor is rendered appropriate or required by the observable traits of the actor and its environment, claim to have explained its behavior. • Risk of circularity: observables used to infer unobservables, which are then used to explain observables. • Not falsifiable. • External validity? Geertz: object is “not to generalize across cases but to generalize within them.” • Interpretation is not science.

21. Conclusion • Observation is central to the scientific enterprise. There can be no science without observation. • Observation by itself can never demonstrate cause. To explain a phenomenon requires an empirically supported theory. • Nonetheless, much of what we do in political science is observe.