Measurement of Social Capital: Recall Errors and Bias Estimations

1. Measurement of Social Capital: Recall Errors and Bias Estimations Kuo-hsien Su, National Taiwan University Nan Lin, Academia Sinica and Duke University

2. Change in number of positions accessed from wave I to wave II (N=2,707 respondents) No change : 12% Decrease : 52.9% Increase : 35%

3. Differences between the sets of accessed positions during two interviews may reflect…

4. Motivations • Measurement instability poses a serious challenge to the study of network changes. • Need a clear measurement or better understanding of the possible sources of error. • The two periods panel survey provided an opportunity (1) to model factors associated with changes in accessed position (2) to detect whether the respondent forgot a subsequently/previously named contact .

5. Prior research • Forgetting is a pervasive phenomenon in the elicitation of network contacts. • Research on forgetfulness has been disproportionately based on name generator instrument. • Little research on the reliability of position generator.

6. Tasks

7. Data • Social Capital Project: the Taiwan Survey, conducted in late 2004 and 2006 • Consists of 1,695 men and 1,585 women aged 20-65.

8. Problem of Non-response Wave I 2004 N = 3,280 Wave II 2006 N = 2,710 Re-interview = 82.6% Non-response = 17.4%

9. Table 1. Characteristics of the follow-up and non-response sub-sample

10. Three types of research designs (Brewer, 2000).

11. Limitations of our data • Our survey was not designed to examine forgetting specifically. • No recognition data or objective records to compare with. • Two years interval is too long: Test-retest design is usually within a very short time interval.

12. Revised method C: Comparison of accessed positions elicited in two separate interviews How many years have you known this person？ 2004 2005 2006 Wave I Wave II Whether the respondent forgot a subsequently named contact？ Forgetting = (Contact mentioned in wave II but not mentioned in wave I) AND (duration >= 3 years) Assumption: durations reported in wave II are more or less accurate.

13. Coding scheme for tie changes

14. The distribution of length of relationship of forgotten ties (N=4,332 dyads, 7.3%) The average duration of ties forgotten is 13 years

15. How much does the respondent forget？ Unique = 51.1% approximately 15% of forgetting

16. Distribution of respondents by number of ties forgotten (N=2707 respondents) 35.6% of the respondents did not forget any ties 64.4 %of the respondents failed to mention at least one contact, with an average of 1.6 forgotten ties per respondent. These numbers suggest that forgetting a contact was not a rare occurrence.

17. Analytical Strategies Model predicting “forgetting” Analysis for the effect of forgetting on estimates of accessibility • What factors are associated with forgetting？ • Unit of analysis: person-contacts dyads • Model : Multilevel logit • Whether “forgetting” affects estimates of network resources ？ • Unit of analysis: person

18. Sample • A multi-level logit approach • The models estimate the odds of “forgetting” versus “not forgetting”; the reference population consisted of all contacts mentioned in the first interview (2004).

19. LEVEL 2 LEVEL 1 Data structure The multi-level approach requires us to transform the individual-based data to person-contacts observations. Positions nested within individuals The final sample consists of 2,682 respondents and 28,343 person-contact dyads.

20. Variables • Level 2 (respondent level): • Age • Years of schooling • Marital status (married) • Employment status (employee) • Occupational prestige score • Size of daily contact

21. Variables • Level 1 (ties level): • Type of relationships • Group into six categories: kin, neighbor, school tie, work-related ties, friends, indirect tie • Length of relations (in years) • Closeness • Gender homophily • Status difference • Status distance = absolute difference between respondent’s prestige score and contact’s prestige scores • Status disparity = respondent’s prestige score – contact’s prestige score

22. Descriptive statistics (individual level)

23. Descriptive statistics (dyad level)

24. Multi-level model predicting “forgetting”(level-2 model)

25. Multi-level model predicting “forgetting” (Level-1 model)

26. Multi-level model predicting “forgetting” (Level-1 model)

27. Findings • Recall error may not be random. • Forgetting is more likely among weak ties. • How does recall error affect the estimation of network-driven indices？

28. Table 4. Discrepancy between “true” (corrected) and “observed” (raw) network resources indices Because forgetting is more likely among weak ties, position-generator underestimate embedded network resources.

29. Table 5. Correlations between “true” (corrected) and observed (raw) network resources indices at wave I (N=3,272)

30. Conclusions • Forgetting a contact was not a rare occurrence; • Recall error is largely nonrandom. • Status difference appears to govern the recall process. • Position generator systematically underestimates network-driven resource indices.