Principal Evaluation: The National Story

1. Principal Evaluation: The National Story CCSSO April 2011 Joseph Murphy Vanderbilt University

2. I. SETTING THE STAGE: EVALUATION IN CONTEXT

3. State & district policy Standards District & state policy evaluation Conditions of Work governance incentives professional development l i c e n s u r e Career Line preparation internship induction (residency) accreditation/program approval professional development teacher leadership preparation relicensure mentoring District & state policy Education State & district policy Standards

4. II. PRINCIPAL EVALUATION: THE NATIONAL STORY

5. PROBLEMS WITH CURRENT EVALUATION SYSTEMS

6. Content • Little evidence that systems evaluate what is important, i.e. not valid • Insufficient attention to leadership for learning, especially curriculum and instruction

7. Process • Limited architecture – focus on one approach (e.g. goals) • Perfunctory – not a deep process

8. Impact • Principals not receiving useful feedback • Not promoting professional growth of principals • Not promoting organizational improvement

9. III. BUILDING A NEW PRINCIPAL EVALUATION SYSTEM

10. A. Building: Developing Guiding Principles of the Evaluation System • Foundation • Based on ISLLC Standards • Process • Be valid, reliable, and equitable • Be transparent • Have objectivity, evidenced through multiple measures and supported by collection, analysis and application of appropriate data and evidence

11. Underscore both formative and summative components • Be comprehensive yet manageable • Allow for flexibility at local level • Include multiple indicators of academic, social, and emotional growth for all students • Promote collaboration between supervisor and evaluatee • Define and prioritize expectations and areas of emphasis • Provide frequent and specific feedback • Insure due process

12. Outcomes • Promote personal accountability • Promote continuous professional growth • Promote school improvement that results in student achievement • Provide a process for the district to connect principals to resources and growth opportunities

13. B. Building a Principal Evaluation System: An Overview • Step 1: Anchor the System • Create the system around two elements: • research on principal impact on student learning • recommendation: ISLLC Standards • organizational outcomes • recommendation: measures of student learning

14. Step 2: Select Components • Choose components that will tap into or shed light on the two elements noted above • Recommendation: more than one component, no more than five components • Recommendation: include one component on student learning and one on 360 degree feedback on performance

15. Step 3: Value the Components • Provide weights to the components • Example #1: • organizational goals 15% • student achievement 40% • 360º assessment (VAL ED) 30% • professional developmental goal 15% • total 100% • Example #2: • 360º assessment (VAL ED) 50% • student achievement 50% • total 100%

16. Step 4: Set Performance Levels • Create performance levels for each component • Recommendation: use four levels • Below basic (inadequate) • Basic (satisfactory) • Proficient • Distinguished

17. Step 5: Set Parameters • Develop borders for each performance level • Example—student achievement • Below basic ↓.9 years growth • Basic .9 - 1.1 years growth • Proficient 1.2 - 1.3 years growth • Distinguished ↑.1.3 years growth • Example—360º (VAL ED) • Below basic 1:00 - 3:28 • Basic 3:29 - 3:59 • Proficient 3:60 - 3:99 • Distinguished 4:00 - 5:00

18. Step 6: Assign Scores • Provide an assessment score for each component and a total score • Example—components and raw total score • organizational goals (15%) basic (2) 30/60 • student achievement (40%) basic (2) 80/160 • 360º assessment (VAL ED) (30%) proficient (3) 90/120 • professional dev. goal (15%) proficient (3) 45/60 • 245/400 (÷ 4 = 61/100) • example—total score: • below basic 25-50 • basic 51-65 • proficient 66-84 • distinguished 85-100

19. Step 7: Determine Consequences • Ensure that results are used for: • shaping professional development • retention and promotion • contract length • salary/bonuses • Example—used for contracts: • Distinguished: 3 year contract • Proficient: 2 year contract • Basic: 1 year contract • Below basic: 1 year contract and improvement program

20. Example—used for bonus payments • Rules in example:  • Pool of money = $15,000 per principal (20 principals x 15,000 = $300,000 pool); i.e., maximum bonus = $15,000 • Set bonus guideline so no bonus money for less than “proficient” work • Bonus allocated by component • Three component system in use

21. Principal Williams Bonus: 8000

22. Principal Smith Bonus: 2000

23. C. Principal Evaluation Work: Detailed Explanation Components States

24. PERFORMANCE GOAL • Source of Goal • National, state, and local assessment • Improvement plans • Standards • Principal self reflection • District goals • School audits • Surveys

25. Essential Characteristics of Goals • Be linked to ISLLC Standards • Be organizationally grounded and emphasize the direct contributions of the leader – organizationally grounded goal(s) may include district, superintendent, and/or community-based goals and priorities • Be anchored in an analysis of multiple sources around relevant data – sources of data may reflect cognitive or non-cognitive measures

26. Be specific, measurable, attainable, realistic, timely, and challenging • Have a longitudinal focus – evaluation goals should address the hard work that will be required over time to promote change and improvement • Be mutually determined through collaborative dialogue • Be collaboratively reviewed with frequent and specific feedback

27. STUDENT ACHIEVEMENT • Source of Goal • Norm-referenced (state/district) • Proficiency • End of course • SAT/ACT • AP • Locally developed • Focus of Goal • Level • Equity • Value-added

28. 306 DEGREE ASSESSMENT • Anchored on ISLLC standards • Valid and reliable • Helpful for individual and school

29. PROFESSIONAL GROWTH GOAL • Source of goal • Previous evaluations • School improvement plans • Demographic information • District goals • Essential characteristics • Directly linked to Standards and indicators for School Improvement and ISLLC Standards • The principal should engage in personal and collegial reflective practice that promotes continuous professional growth

30. The principal and supervisor will determine how the acquisition of knowledge and skills will be demonstrated • The principal’s professional growth goal should be differentiated and based on the need determined by collaborative discussion around formative and/or summative evidence • Supervisors shall provide specific and timely feedback, mentoring, and coaching through regularly scheduled collaborative meetings and ongoing communication • The supervisor should provide resources that will aid in this process • It is the responsibility of the principal to collect and maintain data that evidences the impact of personal growth and school improvement

31. PARENT SATISFACTION

32. Drill down on ISLLC Standard

33. IV. VAL-ED: Assessing Learning-Centered Leadership

34. Acknowledgments • The VAL-ED instrument was published and distributed by Discovery Education Assessment starting in July 2008. • We are authors of the VAL-Ed, and while we have made every effort to be objective and data-based in my statements about this instrument in this presentation, readers should judge the facts and related information materials for themselves and make independent decisions regarding the use of the instrument. 2008

35. Outline of Presentation • Background on VAL-ED • The VAL-ED instrument • Psychometric properties • Score reports • VAL-ED and professional development 2008

36. Background on VAL-ED

37. The Case for Leadership Assessment • Most school leadership improvement focuses on professional development, mentoring, licensing policies, and standards. • Minimal attention has been paid to assessment, feedback, and subsequent action. • Leadership assessment and feedback is an important missing link to improving and strengthening school leadership. 2008

38. Learning-Centered Leadership • Leaders should be assessed on leadership behaviors associated with student learning. • Learning-centered leadership is leadership for student performance. • Learning-centered leadership is the framework for our leadership assessment system. 2008

39. Our Conceptual Model 2008

40. Leadership Behavior Framework 2008

41. Definitions of Core Components • High Standards for Student Learning—There are individual, team, and school goals for rigorous student academic and social learning. • Rigorous Curriculum (content)—There is ambitious academic content provided to all students in core academic subjects. • Quality Instruction (pedagogy)—There are effective instructional practices that maximize student academic and social learning. • Culture of Learning & Professional Behavior—There are integrated communities of professional practice in the service of student academic and social learning. There is a healthy school environment in which student learning is the central focus. • Connections to External Communities—There are linkages to family and/or other people and institutions in the community that advance academic and social learning. • Performance Accountability— Leadership holds itself and others responsible for realizing high standards of performance for student academic and social learning. There is individual and collective responsibility among the professional staff and students. 2008

42. Definitions of Key Processes • Planning—Articulate shared direction and coherent policies, practices, and procedures for realizing high standards of student performance. • Implementing—Engage people, ideas, and resources to put into practice the activities necessary to realize high standards for student performance. • Supporting—Create enabling conditions; secure and use the financial, political, technological, and human resources necessary to promote academic and social learning. • Advocating—Promotes the diverse needs of students within and beyond the school. • Communicating—Develop, utilize, and maintain systems of exchange among members of the school and with its external communities. • Monitoring—Systematically collect and analyze data to make judgments that guide decisions and actions for continuous improvement. 2008

43. The Development of VAL-ED • The development of VAL-ED has been supported by a 3-year grant from The Wallace Foundation. • Three phases of our work: • Phase 1 – Leadership conceptualization and assessment system development • Phase 2 – Field testing the behavior rating scale and exploring its properties • Phase 3 – Dissemination of results and products 2008

44. The VAL-ED Instrument

45. The VAL-ED Instrument • The instrument consists of 72 items defining six core component subscales and six key process subscales. • Principal, Teachers, & Supervisor provide a 360-degree, evidenced-based assessment of leadership behaviors. • Respondents rate effectiveness of 72 behaviors on scale 1=Ineffective to 5=Outstandingly effective. • Each respondent rates the principal’s effectiveness after indicating the sources of evidence on which the effectiveness is rated. • Two parallel forms of the assessment facilitate measuring growth over time. • The instrument will be available in both paper and online versions. 2008

46. Purpose & Uses • The VAL-ED can be used as part of a comprehensive assessment of the effectiveness of a leader's behaviors. • The VAL-ED reports principal performance through • Norm-referenced scores and • Criterion-reference scores. • VAL-ED can be used annually or more frequently to: • Facilitate a data-based performance evaluation, • Measure performance growth, and • Guide professional development. 2008

47. Implementation • Identify respondents and invite participation. • Discuss use of results & confidentiality. • Decide paper or online version. • Time and Timing • Average respondent requires 20 to 25 minutes. • Schedule completion after respondents have had a reasonable time to observe/experience the leader’s work and its effects on the school. • Designate person(s) to manage collection and submission of response forms, if paper version used. • Ensure teacher confidentiality. 2008

48. Directions for Completing Rating Scale 2008

49. An Example Set of Responses 2008

50. Psychometric Properties