Social Settings and Randomized Cluster Trials: Journey of the William T. Grant Foundation

1. William T. Grant Foundation www.wtgrantfoundation.org Social Settings and Randomized Cluster Trials: Journey of the William T. Grant Foundation Edward Seidman Event: National and International Perspectives on Place-Randomized Trials in Education Location: Institute for Human Development and Social Change, New York University Date: Friday, October 3rd, 2008

2. William T. Grant Foundation www.wtgrantfoundation.org Overview of the Presentation • Shifting Paradigms • WTGF’ RFPs and Research Interests in the 21st Century • Social Settings Framework • Evolving Research Design and Analysis Challenges • Evolving Setting-Level Measurement Challenges • Future Challenges and Directions 2

3. William T. Grant Foundation www.wtgrantfoundation.org Shifting Paradigms • Individualism (or person-centered) foci • Preoccupation of western style thought • Well developed instrumentation • Population-centered prevention/promotion • Policy perspective • Context • Seemingly all inclusive • Hard to get a handle on for: • Research, including measurement • Intervention • The proximal Social Settings of daily experience • A “place to stand”

4. Chronology of RFPs Re. Social Setting Research and Lessons Learned William T. Grant Foundation www.wtgrantfoundation.org • 2002 RFP: Intervention Research • Naïve conceptions of intervention, research design and analysis, limited understanding of power when “program” was the unit of analysis > need for capacity-building • Need for further specification of our interests in contexts/settings • 2005-7 RFP: Intervention Research to Improve Youth-Serving Organizations • Need for the development and improvement of the measurement of social setting processes • 2007-? RFP: Development and Improvement of the Measurement of Classroom Quality (in collaboration with the Spencer Foundation)

5. William T. Grant Foundation www.wtgrantfoundation.org Why Social Settings? • Need to understand the proximal settings of daily experience – families, peers, classrooms and schools, youth-serving organizations, neighborhoods • Need to understand how policies work through proximal settings of daily experience to effect youth • Social settings as loci of intervention > more “bang for the buck,” especially in classrooms and schools and other youth-serving organizations in terms of prevention and promotion

6. William T. Grant Foundation www.wtgrantfoundation.org Social Setting Theory and Measurement References • WTG Annual Report Essays: • 2005 Social Setting Theory and Measurement (website) • 2006 Why try to understand and improve social settings? (website) • 2006 Doing social setting research (website) • 2007 Measuring social settings (website)

7. William T. Grant Foundation www.wtgrantfoundation.org Social Setting Theory and Measurement References (continued) Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. American Journal of Community Psychology, 39(3-4), 217-228 Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25, 119-142. Pianta, R.M. (in press). Classroom management and relationships between children and teachers: Implications for research and practice. In Good, T. & Evertson, (Eds.), Handbook of Classroom Management. Shinn, M., & Yoshikawa, H. (Eds.) (2008). Towards Positive Youth Development: Transforming Schools and Community Programs. New York: Oxford Press.

8. Macro Forces Schools (Classrooms) Macro Forces Youth Outcomes Families Neighborhoods Youth Programs O.S.T. Macro Forces Macro forces are policies, the economy, the environment, secular trends in demography, and the like. 2

9. William T. Grant Foundation www.wtgrantfoundation.org Macro Forces Resources Social Processes Macro Forces Youth Outcomes Social Settings Organization of Resources

11. William T. Grant Foundation www.wtgrantfoundation.org Evolving Research Design & Analysis Challenges • Designing Research for Place-Randomized Trials to increase precision (Raudenbush, Bloom, and colleagues): Old issues in search of new answers • N of nested unit – asymptotes quickly • N of randomly assigned units • Baseline covariates, pretests at individual- and school-level • Intra-class correlation coefficients • Matching of covariates • Products: • Optimal Design Software • Consultation Service

12. William T. Grant Foundation www.wtgrantfoundation.org Evolving Setting Measurement Challenges • How can we effectively measure setting-level processes? • Observations, logs, artifacts, etc? • Multi-method, multi-source, multi-construct? • Norms: consensus measures? • Reliability of measuring setting-level processes = Generalizability studies, e.g., CLASS, YPQA, & others • Linking setting-level change to individual-level change

13. William T. Grant Foundation www.wtgrantfoundation.org Future Challenges and Directions • Using new knowledge accrued to better plan and reduce costs of future place randomized trials • Disentangling dynamic nature of setting-level constructs • Discovery of new or improvement of analytic methods to establish causality between setting-level and individual-level change