Research Methods

1. Research Methods Surveys, Field Research & Secondary Data 1

2. Week 6: Lecture outline • Survey data collection methods • Types of Surveys • Field Research • Secondary Data • Class Round on Methods 2

3. Topics appropriate to survey research • Counting crime – asking people about victimization counters problems of data collected by police • Self-reports – dominant method for studying the etiology of crime • Frequency/type of crimes committed • Prevalence (how many people commit crimes) committed by a broader population 3

4. Topics appropriate to survey research • Perceptions and attitudes – to learn how people feel about crime and CJ policy • Policy proposals – search for ways to respond to crime that are supported by the general public • Targeted victim surveys – used to evaluate policy innovations & program success • Other evaluation uses – e.g., measuring community attitudes, citizen responses, etc. 4

5. Open- and close-ended questions Open-ended – respondent is asked to provide his or her own answer • What were your initial expectations of the program? ________________________________________________ Closed-ended – respondent selects an answer from a list or scale • On a scale of 1 to 5, where 1 is low and 5 is high, rate your initial expectations of the program. 5

6. Open versus closed? • Choice between these question types will depend on the aim of your research, the study design, the population being study, etc. • Advantages and disadvantages to both. • Possible gains in reliability (closed-ended) balanced with possible losses in breadth of information and validity. • One strategy is to combine open- and close-ended questions. 6

7. Designing questions… • Make items clear – avoid ambiguous questions; do not ask “double-barreled” questions • Short items are best – respondents like to read and answer a question quickly • Avoid negative items – leads to misinterpretation • Avoid biased items and terms – do not ask questions that encourage a certain answer • Closed question response categories must be exhaustive and mutually exclusive. 7

8. Survey data collection methods 1. Self-administered 2. In-person interview 3. Telephone interviews Focus: Australia’s National Crime Victimization Survey Methodology 8

9. Australia’s Crime Victimization Survey • The MPHS was conducted as a supplement to the monthly LFS. • Each month one eighth of the dwellings in the LFS sample were rotated out of the survey. • In 2012-13, all of these dwellings were selected to respond to the MPHS each month • . In these dwellings, after the LFS had been fully completed for each person in scope and coverage, a person aged 15 years and over was selected at random (based on a computer algorithm) and asked the various MPHS topic questions in a personal interview. • If the randomly selected person was aged 15–17 years, permission was sought from a parent or guardian before conducting the interview. If permission was not given, the parent or guardian was asked the crime questions on behalf of the 15–17 year old. • Questions relating to sexual assault, alcohol or substances contributing to the most recent physical or face-to-face threatened assault were not asked of proxy respondents. Only those persons aged 18 years and over were asked questions on sexual assault. • Data was collected using Computer Assisted Interviewing, whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, usually during a telephone interview. 9

10. A Link to the USA National Crime Victimization Survey • http://www.bjs.gov/index.cfm?ty=dcdetail&iid=245 10

12. UCR and NCVS • UCR data are based on reported criminal acts (offender characteristics) • NCVS data based on individuals actually victimized (characteristics of victims)

13. Assessment of NCVS • Document a massive amount of crime that goes unreported • Underestimate crime rate • Insignificant crimes tend to be forgotten • Victims of several crimes may also forget about all the crimes • Females do not report victimization if her abuser live in the same household • Whites and college graduates are more likely to report being victimized

14. Assessment of NCVS • NCVS respondents are interviewed every six months (7 interviews) • Reported victimization rates usually decease with each interview (awareness of victimization) • Overestimation of some crimes • Respondents might mistakenly interpret some noncriminal events as crimes • “Telescoping “ effect

15. Self-reports data • Created to complement UCR and NCVS • Interviews or questionnaires • Demonstrate the prevalence of offending (the proportion of respondents who have committed a particular offence) • Incidence of offending (the average number of offences per person in the study)

16. Samples for self-reports • Target: Adult inmates of jails and prisons: PREA findings on sexual assault • Target: Adolescents, usually high school students • The most important finding: delinquency is very common • Middle-class youth commit as much crime as working-class youth • Testing criminological theories: Travis Hirschi’s Research Study testing Control Theory( compared to subculural & Strain)

17. Assessment of self-report studies • Focus on minor and trivial offenses (truancy, running away from home, minor drug and alcohol use) • Although recent studies (NYS) asked subjects about rape and robbery • Respondents might not to tell the truth (reliability issues)

18. If respondents lie…. • Self-report data can be checked against police records, school records, interviews with teachers and parents • The use of, or threat of , polygraph validation (20% change their initial responses when threatened with a “lie detector”) • Subsequent interviewing of subjects permits probing regarding the details and context of acts • Use of “lie scales”

19. Focus: Self Reported Substance Use Among Detainees in Australia • http://www.aic.gov.au/media_library/conferences/evaluation/mcgregor.pdf • http://192.190.66.70/documents/7/E/8/%7B7E8D4A8E-A5AF-4D3B-8821-ED8A1BA489B6%7Drpp93.pdf 19

20. UCR, NCVS, and self-reports • None of the three is perfect • For the best estimates of the actual number of crimes, NCVS data are preferable • For the best estimates of offender characteristics, self-reports and NCVS are preferable • UCR are superior for understanding the geographical distribution of crime

21. Self-administered questionnaires • Can be home-delivered • Researcher delivers questionnaire to home of sample respondent, explains the study, and then comes back later • Mailed (sent and returned) survey is most common • Researchers must reduce the trouble it takes to return a questionnaire • Goal is High Response Rate 21

22. 22 Computer-based self-administration • Via email, website • Issues: • representativeness • mixed in with, or mistaken for, spam • requires access to Web • sampling frame?

23. Week 7: More Survey Research What we will Cover: 1.Interviewing techniques and Focus Groups 2. A Comparison of Survey Methods: phone/ computer( survey monkey), mail, and in-person 3. Field Research: The good, the bad, and the ugly 4. Agency Records, Content Analysis and Secondary Data 23

24. 24 In-person interview survey • Typically achieve higher response rates than mail surveys (80-85% is considered good) • Demeanor and appearance of interviewer should be appropriate; interviewer should be familiar with questionnaire and ask questions precisely • When more than one interviewer administers, efforts must be coordinated and controlled • Practice interviewing

25. 25 Specialized interviewing • Two variations: • General interview guide – less structured, lists issues, topics, questions you wish to cover; no standardized order • Standardized open-ended interview – more structured, specific questions in specific order; useful in case studies, retrieves rich detail in responses

26. 26 Telephone surveys • 94% of all households now have telephones • Random-Digit Dialing • Obviates unlisted number problem • Often results in business, pay phones, fax lines • Saves money and time, provides safety to interviewers, more convenient • may be interpreted as bogus sales calls; ease of hang-up

27. 27 Comparison of the three methods • Self-administered questionnaires are generally cheaper, better for sensitive issues than interview surveys • Using mail – local and national surveys are same cost • Interviews – more appropriate when respondent literacy may be a problem, produce fewer incompletes, achieve higher completion rates • Validity low in survey research; reliability high • Surveys are also inflexible, superficial in coverage

28. 28 Tips on self-report items • Convince subjects you will guarantee confidentiality and anonymity • Minimize possible social undesirability you are asking respondents to admit • Phrase questions in non-judgmental manner • Bear in mind “fading memory” when setting time frame

29. 29 Focus groups • 12-15 people brought together to engage in guided group discussion of some topic( e.g. addicts& recovery) • Members are selected to represent a target population, but cannot make statistical estimates about population • Most useful when precise generalization to larger group is not necessary • May be used to guide interpretation of questionnaires following survey administration • Examples: Drug addicts and reentry, homeless sex offenders, teenagers and sexting

30. Getting Out Field Research 30

31. Field Research • Field research encompasses two different methods of obtaining data: • Direct observation • Asking questions • May yield qualitative and quantitative data • Often no precisely defined hypotheses to be tested • Can be used to make sense out of an ongoing process

32. Topics Appropriate to Field Research • Gives comprehensive perspective – enhances validity • Go directly to phenomenon, observe it as completely as possible • Especially appropriate for topics best understood in their natural setting • How street-level drug dealers distinguish customers • Studies of ‘vice’, e.g., prostitution and drug-use. • Aspects of physical settings, Disney World, social control

33. Various Roles of the Observer (Gold, 1969) • Complete participant – participates fully; true identity and purpose are not known to subjects • E.g., posing as a bar patron; becoming a police officer; or corrections worker • Participant-as-observer – make known your position as researcher and participate with the group • E.g., study of active drug users; Julie Mueler and the Guardian Angels • Observer-as-participant – make known your position as a researcher; do not actually participate • E.g., Observational study of police patrol—”ride-along” ; research on gangs • Complete observer – observes without becoming a participant • E.g., court observation, Chicago neighbourhood study( windshield studies)

34. Observer status • Be aware of, and document role of researcher (extent of participation) • Be aware that all observation is subjective. • Be aware of the possible “effect of participation”. • Be aware of often competing ethical and ‘scientific’ values related to all observational studies. 34

35. Asking Questions • Field research is often a matter of going where the action is and simply watching and listening • Also a matter of asking questions & recording answers • Field research interviews are much less structured than survey interviews • Ideally set up and conducted just like a normal, casual conversation

36. Preparing for the Field • Access to formal organizations • Find a sponsor, write a letter to executive director, arrange a phone call, arrange a meeting • Access to subcultures • Find an informant (e.g., person who works with offenders), use that person as your “in” • Snowball sampling is useful as informant identifies others, who identify others, etc.

37. Sampling in Field Research • Controlled probability sampling used rarely; purposive sampling is common • Bear in mind two stages of sampling: • To what extent are the situations available for observation representative of the general phenomena you wish to describe and explain? • Are your actual observations within those total situations representative of all observations?

38. Recording Observations • Note taking, tape recording when interviewing and when making observations • Videotaping or photographs can make records of “before” and “after” some physical design change • Field notes – observations are recorded as written notes, often in a field journal; first take sketchy notes and then rewrite your notes in detail • Structured observations – observers mark closed-ended forms, which produce numeric measures

39. Linking Field Observations and Other Data • Useful to combine field research with surveys or data from official records • Baltimore study of the effects of neighborhood physical characteristics on residents’ perceptions of crime problems (Taylor, Shumaker, & Gottfredson, 1985) • Perceptions – surveys; • Physical problems – (1) observations, (2) actual population and crime information - census data & crime reports from police records

40. Strengths and Weaknesses of Field Research • Provides great depth of understanding • Flexibility (no need to prepare much in advance) • More appropriate to measure behavior than surveys • High validity; quant. measures – incomplete picture • Low reliability – often very personal • Generalizability – personal nature may produce findings that may not be replicated by another • Precise probability samples can’t normally be drawn

41. Other Sources to Consider Agency Records, Content Analysis and Secondary Data 41

42. Secondary Data • Data from agency records – agencies collect a vast amount of crime and CJ data • Secondary analysis – analyzing data previously collected • Content analysis – researchers examine a class of social artifacts (typically written documents)

43. Topics Appropriate for Agency Records • Most commonly used in descriptive or exploratory studies • Content analysis often center on links between communication, perceptions of crime problems, individual behavior, CJ policy

44. Types of Agency Records • Published Statistics – gov’t organizations routinely collect and publish compilations of data (e.g., ABS, BOCSAR, AIC) often available in libraries and online • Nonpublic Agency Records – agencies produce data not routinely released (e.g., police departments, courthouses, correctional facilities) • New Data Collected by Agency Staff – collected for specific research purposes; less costly + more control

45. Units of Analysis in Criminal Justice Data • Criminal Activity • Incidents • Crimes violated • Victims • Offenders • Court Activity • Defendants • Filings • Charges and Counts • Cases • Appearances • Dispositions • Sentences • Apprehension • Arrests • Offenders • Charges • Counts • Corrections • Offenders • Admissions • Returns • Discharges

46. Sources of Reliability and Validity Problems • Virtually all CJ record keeping is a social process – “social production of data” • Records reflect decisions made by CJ personnel as well as actual behavior by juveniles and adults • Discretion factors in to recordkeeping • CJ organizations are more interested in keeping track of individual cases than in examining patterns • Potential for clerical errors due to volume of data

47. Content Analysis • Systematic study of messages – can be applied to virtually any form of communication • Decide on operational definitions of key variables • Decide what to watch, read, listen to & time frame • Analyze collected data • Well suited to answer “who says what, to whom, why, how, and with what effect?”

48. Aspects of Sampling and Coding in Content Analysis 2 • Reminders: • Remember operational definition of variables, and their mutually exclusive & exhaustive attributes • Pretest coding scheme • Assess coding reliability via intercoder reliability method and test-retest method

49. Secondary Analysis • Sources – websites, libraries • AIC, http://www.aic.gov.au/ • BOCSAR, http://www.bocsar.nsw.gov.au/ • BJS, NCVS, ICPSR, NACJD • Advantages – cheaper, faster, benefit from work of skilled researchers • Disadvantages – data may not be appropriate to your research question; least useful for evaluation studies (which are designed to answer specific questions about specific programs)

50. Class Round What’s the most appropriate method(s) for investigating your research evaluation question(s)?