SSCI E-100b Graduate Research Methods and Scholarly Writing in the Social Sciences: Government and History (23667) Joe

1. SSCI E-100b Graduate Research Methods and Scholarly Writing in the Social Sciences: Government and History (23667)Joe & Doug Bond2/24/2014

2. 1st papers (to be returned 3/3) • Facilitation on 3/3 • Bullying and Divorce • Research Methods • Literature Reviews • Illustration of Applied Research • Discuss Readings • Fourth In-Class Writing Assignment

3. Authoritative Numbers: Bullying in School Journal of the American Medical Association (JAMA) study The Study (random sample of approximately 16,000 students) • Nearly 1 in 3 U.S. children in 6th through 10th grades have been bullied, or bully other students. SHOCKING, I SAY! Ask yourself: how do they define bullying? Definition (buried in an appendix): “We say a student is being bullied when another student, or a group of students, say or do nasty things and unpleasant things to him or her. It is also bullying when a student is teased repeatedly in a way he or she doesn’t like. But it is not bullying when two students of about the same strength quarrel or fight.” (emphasis in original)

4. Your Risk of Divorce • What proportion of marriages end in divorce? • No one official agency keeps track • Filing a marriage license and obtaining a divorce are both legal steps. • Divide the number of divorces during a particular year by the number of marriages during that year to get a rough measure of the likelihood of marriage ending in divorce. • Since the 1960s, the number of divorces has been nearly half that of marriages (referred to as the divorce rate).

5. The Problem with Divorce Rates • Example: crime rates • Crime rate calculated per 100,000 people (murder rate is 5.5 per 100,000 in the U.S., according to the FBI). • But who makes up the population at risk when we try to calculate divorce rates? • Obviously, it doesn’t just include those married during the same year. • In fact, we know that relatively few couples get divorced during the calendar year in which they marry.

6. Divorce, Continued • The population at risk for divorce is all married couples. • Say we record 2 marriages in a year on the Isle of Fictitiousand one of those marriages ends in divorce, yielding a 50% divorce rate. • But cohorts matter. • Divorce data derived from a sample rather than a population as a whole paints a more accurate picture. • There are important cohort differences that reveal how society has changed. • Basically, people in each cohort were likely to have remained married longer than those in the cohort that followed.

7. Divorce Rates by Cohort • 35% of 60-year-old men had their first marriage end in divorce • 40% of 50-year-old men had their first marriage end in divorce • Of women who first married during 1945-1949, 70% were married 30 years later • Only 55% of those married for the first time during 1960-1964 remained married • Key Point: estimating divorce rates is a fairly complicated matter

8. Examples of Five Research Methods (far from exhaustive) • Experimental • Correlation • Natural Observation • Survey • Case Study

9. Experiments • Researcher manipulates a variable (anything that can vary) under highly controlled conditions to see if this produces change in a second variable • The variable, or variables, that the researcher manipulates is the independent variable(s) while the other variable, the one measured for changes, is the dependent variable.

10. Strengths: Experiments • Experiments help us in understanding relationships (about as close to cause and effect as one can get) BUT • Must be sure that manipulation of an independent variable is the only variable having an effect on the dependent variable. S/he does this by holding all other variables constant or equivalent (control variables) • See quasi-experimental design in Creswell

11. Weaknesses: Experiments • Experiments can only be used when it is practical and ethical for the researcher to manipulate the antecedent conditions (e.g. Tuskegee experiments - 399 black sharecroppers with syphilis (1932 -1972), The Tearoom Trade (1970) by Laud Humphreys, Stanley Milgram’s experiments, etc.) • Experiments are usually done in the highly controlled setting of the laboratory. These conditions are artificial and may not reflect what really happens in the less controlled and infinitely more complex real world (counterfactual thought experiments an exception) • Randomization is often problematic, particularly in medical studies

12. Can one's biochemistry be, at least in part, responsible for aggressive or violent behavior? • A biologist at Yale, attempted to answer this question by manipulating the biochemistry of group prison inmates • Previous studies suggested that lower levels of serotonin was associated with increased aggression (also depression) and, conversely, higher levels of serotonin caused decreases in aggressive responses • Sheard(1995) injected a group of prisoners (the experimental group) with lithium carbonate, known to enhance serotonin brain activity • Another group (the control) of similar prisoners were given a placebo • The lithium injections dramatically reduced aggressive behavior in prisoners over a four month trial

13. Correlation Approach • Correlation is classified as a non-experimental method because variables are not directly manipulated • It is more of a statistical tool. • These studies are designed to determine the strength and direction of a relationship between two or more variables

14. Strengths: Correlation • They can be used to determine if there is a relationship between two variables without having to directly manipulate those variables • They can be used when it is impractical and/or unethical to manipulate the variables • They can be used as a basis for prediction. For instance, if we know that two variables are highly correlated, say +.85, we can predict the value of the dependent variable by knowing the value of the independent variable (e.g. SAT scores and success in college)

15. Weaknesses: Correlation • Correlation does not tell researchers whether or not the relationship is causal [e.g. drowning and ice cream consumption or children’s shoe size and math skills] • Since correlation does not, cannot, prove causation, we never know for sure • If we can’t disprove it, it supports [but does not prove] our hypothesis

16. Natural Observation • Field studies; non-experimental approaches used in the field or in real-life settings • The researcher very carefully observes and records some behavior or phenomenon, sometimes over a prolonged period, in its natural setting (inductive approach) • This usually involves observing humans or animals as they go about their activities in real life settings • Nickel and Dimed: On (Not) Getting by in America (Barbara Ehrenreich; 2002)

17. Strengths: Natural Observation • It allows the researcher to observe behavior in the setting in which it normally occurs rather than the artificial and limited setting of the laboratory • It might validate some laboratory finding or theoretical concept

18. Weaknesses: Natural Observation • This is a descriptive method, not an explanatory one. The behavior can only be described, not explained • It can take a great amount of time • It is sometimes difficult to observe behavior without disrupting it and the difficulty of coding results in a manner appropriate for statistical analysis

19. What is the impact of having a label (stigma) attached to one's self. How does this label influence others' perceptions of and responses to the stigmatized person? • The label of schizophrenia • A doctor (Rosenhan) had himself and eight other volunteers admitted to mental hospitals throughout the United states by faking symptoms of schizophrenia • Once admitted, the pseudo-patients immediately began behaving "normal“ • Despite the normal behavior, none of the patients were recognized as "sane" • Their stays ranged from 7-52 days with an average stay of 19 days • Normal behaviors were interpreted to fit the label (e.g. the pseudo-patients took extensive notes while they were on the wards and in the open. This behavior was seen as an aspect of their pathological behavior)

20. Survey Research • Usually not an end-product • Typically used as input for statistical analyses (correlation analysis) • Administering Surveys (refer to Trochim): • Telephone • Mail • Online surveys • Personal in-home survey • Personal mall intercept survey • Response Rates • Items (the fewer, the better, avoid negations, etc.) • Constructing and administering a good survey is not simply a matter of coming up with a series of questions.

21. Case Studies (much more on this later) • In-depth exploration of an event, an activity, a process, of one or more individuals. • Psychobiography: The Colonial House by George & George) • Phineas Gage • Anna: Case study of a Feral child (Kingsley Davis)

22. Literature Reviews • Lit reviews are guided by one or more general questions • By the time you are finished with your review, you will have [most likely] the answer to your question(s) • You will also have one or more new questions • These questions will [hopefully] serve as the focal point of your ALM thesis

23. How many sources? It depends. Universe Discipline Y We know a lot Little is known Discipline Z We know less Discipline X

24. Begin with a dozen sources, if possible • Try not to cite everything under the sun related to your topic • Start broad and shoot for specificity as your review progresses • If little is known, branch out (e.g. interdisciplinary) • If the topic/question has been thoroughly investigated, go for more specificity • Try to stick with “scholarly” books and refereed journal articles • While internet sources are fine for ideas, try to cite a hardcopy, if available (e.g. some online reports are also available in hardcopy; UN documents). • Do not cite an internet source that refers to someone else’s study; rather, cite the study.

25. A “Researchable” Topic • In the Extension School, just about anything is fair game • Most ALM theses proposals start out overly ambitious • DO NOT CHOOSE A TOPIC BASED ON WHAT YOU THINK WILL MAXIMIZE YOUR CHANCES OF GETTING A PARTICULAR FACULTY MEMBER TO SERVE AS YOUR ADVISOR • If it doesn’t interest you, you will never finish • AND IF YOU ARE SATISFIED WITH YOUR THESIS WHEN YOU FINISH, IT PROBABLY ISN’T VERY GOOD

26. Should I bother? • A research topic should add to the pool of research knowledge available on the topic • Question to ask: • Does the study address a topic that has yet to be examined, extend the discussion by incorporating new elements, or replicate a study in new situations or with new participants? • Is the topic salient? Does it appeal to a broad audience? Is the topic timely? Is the topic non-trivial? • In the context of the Extension School, your research topic should have something to do with government and/or history broadly defined

27. Purpose of a Literature Review • To share with the reader the results of other studies that are closely related to an area of interest • Relate your research to the larger ongoing dialogue in the literature, filling gaps and extending prior studies • Provide a framework for establishing the importance of your study with other findings SYNTHESIS IS KEY

28. What it is NOT • The literature review is not the place to analyze your research questions - those adopting an historical approach are particularly susceptible to falling into this trap • Only review what has already been reported and/or is known about the topic • By the time that you finish your literature review, you may find that your preliminary questions have already been addressed by others but additional, more interesting questions have been left unanswered

29. A Lit Review IS NOT an Annotated BibliographyThis is an annotated bibliography and yes, it is 204 pages long:http://www.teachingterror.com/bibliography/CTC_Bibliography_2004.pdf

30. # 1 • Identify key words useful in locating materials using Hollis, for example • Key words may help you identify a suitable topic of interest and will assist you in finding preliminary books in the library or e-journals

31. # 2 • Focus initially on refereed journals and books • Search databases typically reviewed by social science researchers include ERIC (http://www.eric.ed.gov/ERICWebPortal/Home.portal), the Social Science Citation Index, etc.

32. The Social Sciences Citation Index • Covers 1969 through the present • Available in most academic libraries • Covers 5700+ journals that represent virtually every discipline in the social sciences • Useful in locating studies that have referenced an important study • Allows the user to “trace” all studies since publication of a “key” study that contain the cited work. • Allows the user to develop a chronological list of references that document the historical evolution of an idea or a study

33. # 3 • Locate a dozen books, journal articles, reports, etc. related to your topic • Avoid shortcuts! Start now! Reading material on the web may be convenient but it is rarely adequate • Start with the most recent publications and work backwards

34. # 4 • Identify an initial group of books and articles that are central to your topic • Review abstracts and skim the articles or chapters • Get a sense of whether the article or chapter will make a useful contribution to your understanding of the literature • Don’t reinvent the wheel! • Use the bibliographic information (i.e. references) contained in the articles and books to extend your search

35. Abstracting Studies A Good literature review summary might include the following: • Mention the problem being addressed. • State the central purpose of focus of the study. • State the underlying assumptions. • Briefly state information about the sample, population, or participants. • Review the key results. • Point out any technical or methodological flaws. • Be sure to jot down full citations even if you do not ultimately incorporate the piece into your review Questions?

36. Assessing State Failure: An Illustration of Applied Research

37. State Failures: Using Structural Data to Predict Political Instability • State Failure project (1994) • objective: then VP Gore asked the CIA to predict which states will fail 5 years out • analyzed thousands of [structural] variables • found that 3 variables could predict failures 85% of the time looking out 5 years • infant mortality (a proxy? for what?) • level of democratization • openness to trade • other salient factors: youth bulge, religious distributions, etc.

38. Africa Prospects: Predicting State Failure with Structural Data

39. Africa Prospects • Purpose:to assess the vulnerability of countries to conflict escalation based on its profile or set of structural indicators. • Overall Accuracy:is defined as the ratio of correct classifications (C) to all classifications (A). Accuracy = C/A * 100%. • Recall: is defined as the ratio of correct classifications (C) to the observed classification (O). Recall = C/O * 100% and represents the ability of the algorithm to classify the conflicts as they were observed. Illuminates the algorithms false negatives. • Precision: is defined as the ratio of correct classifications (C) to correct (C) and incorrect classifications (I). Precision = C/(C+I) *100%. Illuminates the algorithm’s false positives; specifically, the higher the ratio the lower the false positives.

40. Forecasting Performance Metrics: Definitions and Illustrations Recall Precision Overall Accuracy # of correctly predicted conflicts # of conflicts that occurred # of correctly predicted conflicts # of conflicts predicted to occur # of correct predictions # of predictions made • Bad forecast model #1 (almost every country will be unstable) • High recall= 99% (1% miss rate) • Low precision = 5% (95% false positive rate) • Low accuracy =40-50% • NET IS CAST TOO BROADLY • Bad forecast model #2 (few countries will be unstable) • Low recall = 5% (95% miss rate) • High precision =100% (0% false positives) • Low accuracy =40-50% • NET IS CAST TOO NARROWLY • Near-perfect forecast model • High recall = 99% • High precision = 100% • High accuracy =99.5% Countries forecast to be unstable at some level of intensity False positives Countries that experience instability “Misses” misses Countries that DO NOT experience instability

41. 5-15 Year Validation of Forecasting Average Performance Scores For Different Training Sets / Forecast Periods Forecast Period Accuracy Recall Precision However, high recall scores indicate the net is cast wide enough to correctly forecast conflicts that DO occur (errors fall on the side of caution). Low precision scores (high false positive rates) in the out years indicate that the world was more stable than would have been expected given macro-structural conditions.

42. Independent Variables • Caloric Intake: Estimate of the average number of calories consumed per person, per day. • GDP per Capita: Annual gross domestic product per person measured in constant 1995 U.S. dollars. • Male/Female Infant Mortality: Number of deaths of male and female children under 1 year of age per 1,000 live births. • Life Expectancy: Average life expectancy (males and females combined). • Youth Bulge: Ratio of population aged 15-29 to those aged 30-69. • Among others……..

44. Dependent Variable: Index of Instability • Maximum level/intensity of conflict per country-year; source: KOSIMO Data Project, Heidelberg Institute of International Conflict Research (HIIK),1975-2003. http://www.hiik.de/de/index_d.htm • Represents a high threshold of instability 3 Levels of instability intensity Instability Levels High intensity (if combined probability > 67% Moderate intensity (if combined probability > 67% None/Low intensity (if combined probability > 67% Key Assumption: country is unstable if (and only if) the government or its opponent(s)threatens or initiates a conflict to restore equilibrium or harmony with respect to its internal or external relations.

45. Steps • 1. Compile a time series data set of the selected target variable’s intensity • 2. Compile a time series data set of candidate indicators associated with the target’s intensity • 3. Train an algorithm that explains the historical target intensities with the candidate indicators • 4. Calculate performance measures of the explanation from a time series of historical test data • 5. Generate vulnerability scores based on projections from the current value of the indicators • 6. Calculate a confidence level for the forecasts using the likelihood of occurrence at each intensity • 7. Iterate from step #1 for experimentation with alternative target variables • 8. Iterate from step #2 for experimentation with alternative explanatory variables or indicators

46. Using Dynamic Data to Predict Political Instability(MACD: Egypt)

47. Using Dynamic Data to Predict Political Instability, Continued Bollinger Bands – Country Stability for Egypt, 2011

48. What other data type, other than dynamic and structural, might be used to predict political turmoil?

49. Discuss Readings • Chandra & Martin (Voodoo Science & Doing a Literature Review)

50. In-Class Writing Exercise 4February 24, 2014