Using Data to Guide Decision Making at the Pre-school Level

1. Using Data to Guide Decision Making at the Pre-school Level Tim Lewis, Ph.D. & Susan Brawley University of Missouri

2. Pre-school / EC Settings • Program-wide • Classroom focus • Reliance on “informal” decision making • Need for better data to a) conduct systems analyses (problem solve), b) identify at-risk students sooner, c) guide intervention and environmental support, and d) progress monitor

3. Data-Based Decision Making • Needs Assessment • System/Intervention Evaluation • Identifying at-risk students for more intensive supports

4. Data Decision Logic

5. Data-Based Decision Making • Determine what questions you want to answer • Determine what data will help to answer questions • Determine the simplest way to get data • Put system in place to collect data • Analyze data to answer questions

7. 1. Determine what questions you want to answer Examples Can we predict problems/success? When/where/who? Possible “function” of problem behavior? What environmental changes/supports are needed?

9. 2. Determine what data will help to answer questions • Existing data set(s) • Current data collection • Additional / new data • Confidence in accuracy? • Complete picture?

11. 3. Determine the simplest way to get data • Agreement on definitions • Standard forms / process • Frequency of collection • Target “Multi-purpose” data/use Train ALL staff on use & provide on-going TA

13. 4. Put system in place to collect data • Build on existing systems • Add components over time • Central entry point

15. 5. Analyze data to answer questions • Trends • Instruction & supports in place/not in-place • Pre/post “big outcomes” • Comparisons (norm / local) • Relative growth • Absolute growth

17. Needs Assessment Pre-school SAS Pre-school SET

18. Pre-school SAS

20. PRE-SET(Horner, Benedict, & Todd, 2005) • Adaptation of an assessment tool called the School-wide Evaluation Tool (SET) used in K-12 educational settings to measure critical features of school-wide PBS. • The Pre-SET assesses classroom and program-wide variables across 9 categories: A. Expectations Defined B. Behavioral Expectations Taught C. Appropriate Behavior Acknowledged D. Organized and Predictable Environment E. Additional Supports F. Family Involvement G. Monitoring & Decision-Making H. Management I. Program & District-Wide Support

21. Pre-SET Administration • Information necessary for completion of the Pre-SET is gathered from multiple sources including review of permanent products, observations, and staff and child interviews. • A Pre-SET should be conducted for each classroom within an early childhood program if the implementation status or practices (e.g., classroom rules) are different for each classroom. • The Pre-SET may be conducted at the program level if all classrooms within the early childhood program are at the same implementation status and use the same practices (e.g., have same classroom rules).

23. System/Intervention Evaluation Pre-school ODRs?

26. EC Behavioral Report

30. Identifying at-risk students Tier II / III

31. At-risk Students • Behavioral incident reports • Create data decision rules – capture 10-15% • Teacher Referral • Clear process with supporting data • Screening

32. Proactive multiple-gated screening: Stage One: Teacher ranking of externalizing & internalizing behaviors Stage Two: Teacher ratings of the 5 highest ranked children Stage Three: Direct observations & parent questionnaires of children exceeding Stage Two criteria Early Screening Project(Walker, Severson, & Feil) Teacher Ranking Teacher Ratings Observations

33. Missouri SW-PBS

34. Missouri Regional Professional Development Centers

36. MO SW-PBS Early Childhood PD • Annual Statewide Early Childhood Summit • Topics: • Problem-Solving with Class-wide Data • Family Involvement • Maintaining Developmentally Appropriate Practices with Program-wide PBS • 15 Programs that Responded to Informal Survey on Data Collection Systems Indicated: • 4 use Excel • 4 use SWIS • 7 use nothing

37. EC Summit Feedback • Barriers to Implementing PW-PBS • Time • Buy in • Lack of data system • To Improve Data Collection and Use Need • Time to record • Handy and practical forms • Training about data • Consistent definition of terms • Staff buy in • Communication and collaboration between EC and Kindergarten • Time for purposeful data collection

38. Pilot of EC Data Collection Tools • Sample fields • Name • Date • Age • Challenging Behavior (vs. Problem Behavior) • Routine (vs. Location) • Teacher Response • Graphs generated • Per day per month • Behavior • Time • Location (routine) • Student

39. Rationale for Data Focus • Increasingly schools are using three-tiered models for prevention and intervention in K-12 (Ardoin, Witt, Connell, & Koenig, 2005). • Early childhood programs are beginning to use three tiered models but challenges exist when trying to implement the key features(Hawken, Vincent and Schumann, 2008). • One of the critical parts of successfully implemented tiered models for social behavior and academic skills is the collection of data for decision making (Hawken, Vincent and Schumann, 2008).

40. Rationale for Data Focus • Practitioners need data to identify the type and level of support that children need (Barnett, et al., 2006; Hojnoski, Caskie, Gischlar, Key, Barry, & Hughes, 2009). • Early childhood educators tend to rely on intuition and informal observations of children rather than the collection and systematic use of data to guide decision making(Sandall, Schwartz LaCroix, 2004). • Early childhood programs do not, and should not, use office discipline referrals, so there is a less formalized method of tracking and reporting challenging behavior (Stormont, Lewis, Beckner, 2005).

41. Rationale for Data Focus • There is a need to understand more about the current practices of data in in order to build capacity for more early childhood programs to collect and use data (Fulignia, Howesa, Lara-Cinisomob & Karolyb, 2009; Kincaid, Childs, Blasé, & Wallace, 2007; Schwartz & Olswang,1996). • Potential barriers to data collection and use in decision making are time,lack of knowledge and lack of experience with data (Hojnoski, et al., 2009).

42. Rationale for Data Focus Potential factors that increase the likelihood of data collection and use in decision making include: • Additional professional development for all staff, terminology changes suitable to EC programs and streamlined data collection systems (Muscott, Pomerleau, and Szczesiul, 2009). • Simplified methodology of data collection, reduction in number of variables on which to collect data and reduction of time required for data collection (Gunter, Callicot, Denny, & Gerber, 2003).

43. Research Study: Status of Data Collection and Use in Early Childhood • Study of Head Start, early childhood special education and early childhood general education programs in Missouri • Administrators and teaching staff • On-line survey

44. Research Questions • To what extent are preschool programs currently collecting and using data for decision-making? • What barriers exist indata collection and use in decision-making? • What factors mayincrease the likelihood that programs collect and use data? • What relationships can be found between administrator and teacher views on data practices?

45. Preliminary Study Findings • Data collection • Analyzing and Using Data • Data systems • Barriers • Positive Factors • Beliefs

46. Data Collection: Importance of Practice

47. Data Collection: Frequency of Practice

48. Data Analysis: Importance of Practice

49. Data Analysis: Frequency of Practice

50. Barriers: Level of Agreement