Developing Deeper and More Meaningful Evaluation Systems

1. Developing Deeper and More Meaningful Evaluation Systems Funded by a grant through the McCormick Foundation Lisa Hood, Center for the Study of Education Policy

2. Teacher Evaluation in the Early Grades: Developing a PreK-3 Support Services for Danielson Framework for Teaching

3. Danielson Early Supports Project Staff Lisa Hood, lhood@ilstu.edu Deb Kasperski, dkasper@ilstu.edu Erika Hunt, elhunt@ilstu.edu Pam Rosa, pam.rosa@cecillinois.org Website http://leadershiplinc.illinoisstate.edu/

4. District Level – Teacher Evaluation in the Early Grades • Danielson Framework for Teaching has been validated for Grades 4 and above, yet many districts were planning to use the Danielson FfT for early childhood teachers (PreK-grade 3) • Is this a valid and reliable tool for PreK-3 teachers? • What tiered assessments are appropriate for PreK-3 classrooms?

5. Developing Early Learning Supports for Danielson Framework for Teaching (FfT) • Fall 2012:Review and adaptation by PreK-3 administrators, teachers, and stakeholders • Critical Attributes and Examples • Winter 2012: NBCT’s developed examples and coded PreK-3 videos using adapted Framework • Winter 2013: Training developed for pilot principal and teachers • March 15, 2013: Feedback from early childhood stakeholders • April 26, 2013: Second round of feedback from early childhood stakeholders • June 2013: National feedback of framework supports solicited at NAEYC conference, San Francisco • Fall 2013: Pilot of framework supports and training

6. Piloting the 2013 FfT with Early Learning Supports • Pilot will occur in seven selected schools throughout the region (Northern, Central, and Southern Illinois) in fall 2013. • Prior to pilot, selected principals and teachers will undergo a three day pilot training that will utilize three inter-connected modules.

7. Module 1: Understanding and using the 2013 Danielson FfT in the context of early learning (PreK-3rd grade) • Present the need for piloting the 2013 FfT Early Learning supports to enhance early learning teaching practice growth opportunities • Identify the structure/priorities of the 2013 FfT Domains/Components and Critical Attributes • Understand shifts in professional practice for different levels of performance and the link to the 2013 FfT Framework Components/Critical Attributes • Explore and connect to Early Learning Grade Band examples at four different levels of practice

8. Module 2: Applying the 2013 Danielson FfT instrument through observation-based evidence collection in early learning teaching settings • Understand observation-based data collection within the Plan-Teach-Reflect-Apply (PTRA) Professional Improvement Cycle for supporting and improving teaching practice • Develop/enhance observation process skills for collecting, sorting, and scoring evidence focused upon early learning Classroom Environment and Instruction • Determine Master Scores for early learning Classroom Environment and Instruction teaching practices using 2013 FfT Components and Critical Attributes

9. Module 3: Applying the 2013 Danielson Framework for Teaching instrument through conversation-based evidence collection in early learning teaching settings • Understand data collection and artifacts of practice and student learning within Professional Improvement Cycle • Enhance skills focused upon early learning Planning and Preparation practices, review of Classroom Environment and Instruction, and Responsibilities • Determine Master Ratings for Planning/Preparation and Responsibilities using FfT Components and Critical Attributes and protocols to confirm inter-rater reliability • Extend knowledge of professional learning-focused coaching language, conversations, and protocols of practice to support planning and reflection conversations

10. New Proposals • February 2013: Submitted a proposal to validate the 2013 Danielson Framework for Teaching for PreK-3rd grade teachers • April 2013: Submitting a proposal to validate the training and professional development modules for principals and teachers associated with the early learning supports for the 2013 Danielson FfT.

11. Purpose of Proposed Validation Study • To ensure that FfT provides accurate and reliable data that PreK-3rd grade teachers and their supervisors can use to identify strengths and weaknesses • To help teachers and supervisors use the data from the FfT to identify appropriate professional development mechanisms to promote teachers’ growth. • To utilize data from the early learning Framework’s observation along with other with student growth data so that teachers who are observed to be “Unsatisfactory, Needs Improvement, Proficient, or Excellent” have appropriate student growth that correlates with the observation ratings

12. Other Benefits of Proposed Validation Study • Construct a student/parent survey • Explore other student growth measures • Increase principals’ understanding of different teaching pedagogies related to early childhood education

13. Purpose of Proposed Professional Development Project • Develop and validate professional development training and supplemental materials to support Danielson FfT for PreK-3rd grade teachers. • To use the data from validation study to identify appropriate professional development mechanisms to promote teachers’ growth and principals’ understanding of early childhood education (PreK-grade 3). • Training geared to principals, other persons evaluating teachers, the PreK-3rd grade teachers

14. Group Activity • Question One: What do evaluators of PreK-grade 3 teachers need to know about early learning (PreK-grade 3) classrooms? • Question Two: What do principals need to know in general about doing effective evaluations that will grow teacher’s practice?

15. Building Evaluation Capacity Within P-12 Principal Programs and Partners

16. Approval of new Principal Endorsement Programs – a beginning not an end • Annual reports required by Illinois Educator Licensure Board • program data (e.g., number of faculty, number of graduates, etc.) • formative data that is internal to the programs and varies by institution • 2011 - Recommendations by P-20 Council Committee on Teacher and Leadership Effectiveness for Principal Preparation Performance Indicators\ • Tied to new P-20 Data Longitudinal System

17. Principal Preparation Program Performance Indicators

18. Principal Preparation Program Continuous Improvement Process Project • Project Staff • Lisa Hood, Project Director • Erika Hunt, Project Staff • Alicia Haller, Evaluation Coach • Joe Pacha, Evaluation Coach • Brad White, Illinois Education Research Council, Team Consultant • Project Partners: • University of Illinois at Urbana-Champaign • Southern Illinois University at Edwardsville • North Central College • Loyola University • New Leaders for New Schools

19. Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process Outputs: Outcomes-Impact Inputs: (What to invest) Faculty Time Money Research Materials Technology Partners Activities: (What we do) Review Principal Preparation Programs (Quality Program Assessments and Feedback. Support university personnel in their continuous improvement processes (Process Improvements) Help faculty implement best practices for improving programs (Knowledge Building) Participation: (Who we reach) Principal Preparation Program Faculty members and ISBE University Partners, Faculty, and ISBE Faculty Members and ISBE Short Term: (What the short term results are) Improved candidate selection Improved courses and programs Improved internship Improved graduates of programs Medium Term: (What the medium term results are) Candidates prepared through quality coursework Candidates ready to perform internship assessments successfully Candidates prepared to lead schools in improving learning for all students Long Term: (What the ultimate impacts are) Improved learning for ALL students Improved teacher practices Improved school leadership

20. Logic Model: Illinois Principal Preparation Program Continuous Improvement and Review Process Evaluation: 1. Focus – 2. Collect Data – 3. Analyze and Interpret – 4. Report – Repeat the Cycle Focus Program Requirements Internship Requirements Internship Assessments Coursework Requirements Staffing Requirements Candidate Selection Collect Data Data Collected for Program Improvement Analysis of the Program Improvement Data Program Changes Based on Data Analysis Summative INTERNSHIP Assessment Data: (Number of candidates; listing of each and all assessments with the scoring of each candidate on the assessment and number of times each candidate needed to complete the assessment successfully; length of the internship for each candidate) Listing of Courses and Course Syllabi Number of candidates and Faculty (FTE) Number of Candidate Applications Number of Candidates Selected Number of Candidate Completers Graduate’s School Data Analysis over time Analyze the results Analysis and planning for program improvement based on data 1, 2, &3. Analysis and implementation for improvement for Internship and candidate success based on data 4, 5, 6, 7, 8 and 9. Analysis and planning for improvements based on data 10. Report Program Improvement Plan Internship Improvement Plan Candidate Selection Improvement Plan Coursework and Staffing Improvement Plan

21. Project Objectives • Implementation of a Program Theory Evaluation (PTE) model to identify how policies, practices, and processes have changed as a result of the process of redesigning; • Creation of an evaluation plan of Principal Preparation Programs; • Alignment of current data and creation of needed data sources to continuously inform evaluation plan process; • Coaching, technical assistance, and tools to inform the evaluation process • Collaboration with the ISBE to provide input to the annual reporting process and requirements for Principal Preparation Programs.

22. Project Activities • Programs will receive technical assistance through evaluation coaches to help build internal capacity • Programs will be provided with: • Tools to inform the evaluation process (e.g., Self-Assessment Rubric, Sample Data Instruments) • Technical assistance (e.g., Coaching and Program Team Meetings) • Meeting facilitation • Timeline planning • Data analysis • Reporting framework

23. Two Project Phases • Phase One: Initial Assessment Phase • Phase Two: Continuous Improvement Phase

24. Phase One: Initial Assessment Phase • Beginning March 2013, evaluation coaches will convene program faculty and partners. • Step One: Completion of a self-assessment process developed by project staff • Step Two: Use self-assessment instrument to capture baseline information on: • partnerships with districts and other community members • content and pedagogy • assessments • internship • Step Three: Share results with program team and develop action plan

25. Phase Two: Continuous Improvement Phase • Evaluation coaches and program faculty/partners will begin testing the program’s theory of change and whether it is having the intended effects on principal candidates and new principals. • Assessment of the program’s “continuous improvement evaluation plan” will be conducted to determine appropriate level of outcomes • Center staff and evaluation coaches, in partnership with program faculty and partners, will document the process and any on-going program modifications made.

26. Next Steps • Interest with funding a multi-phase statewide evaluation of principal preparation programs • Proposed development of a framework for a statewide evaluation of principal preparation • Advisory committee to assist with development of Evaluation Framework • Statewide forum to collect input into draft conceptual framework • Development of an Request for Proposal (RFP) to select an external evaluator for the statewide evaluation

27. Group Activity • Question One: What are the highest priority issues to keep in mind when developing a statewide evaluation framework for principal preparation? • Question Two: What do you foresee will be the benefits and challenges to doing a statewide evaluation of principal preparation programs?

28. Questions?

29. Thank you! • Please remember to complete an evaluation form before you leave. • Evaluation form includes questions on challenges and successes of the new P-12 Principal Endorsement programs.