Alabama Professional Education Personnel Evaluation Program

1. Alabama Professional Education Personnel Evaluation Program Field Test of Central Office System July 2000

2. Statewide Performance Standards • Summative decisions (optional) Central Office Evaluation System • Purpose of Evaluation is for Improvement • Criteria • Central Office system was developed from a Framework • Framework contains Competencies, Tasks, and Knowledge/Skills • Currently being field tested Alabama Department of Education Central Office Field Test Evaluation

3. 1. Communication 2. Collaboration 3. Assessment/Evaluation 4. Organization 5. Planning 6. Laws/Policies 7. Problem Solving 8. Innovation 9. Technology Management 10. Program Operations/Management 11. Fiscal Management/Leadership 12. Professional Responsibilities 13. Leadership of Human Resources Central Office Evaluation System • 13 Knowledge/Skill Areas Alabama Department of Education Central Office Field Test Evaluation

4. Central Office Evaluation System • Scoring • 4-point rating scale with rationale statements (rubrics) • Holistic scoring cross-cutting 13 knowledge/skill areas • Evaluators • Trained in conducting evaluations • Must meet reliability standards • Central Office Administrators • Evaluation content varies with years of experience in position • Evaluation content addresses only areas of responsibility • Evaluation process is same for all Alabama Department of Education Central Office Field Test Evaluation

5. 1. Portfolio developed by central office administrator 2. Surveys of various system administrators, principals, and school staff, when applicable 3. Interviews (written or oral) 4. Supervisor’s Review Form 1. Evaluation Summary Report (shows scores and score rationales by knowledge/skill area) 2. Professional Development Plan (becomes a data source in next cycle) Central Office Evaluation System • Instrumentation/Data Sources • Summary Report Sources • Optional Instrumentation 1. Self-Assessment Form (completed by central office administrator, but not considered in evaluation scoring) Alabama Department of Education Central Office Field Test Evaluation

6. · Central Office Data: evidence of competence produced, assembled and submitted by the central office administrator Portfolio · Written interview · Interpretation of survey responses · Evaluator Data: evidence of central office administrator competence developed by the evaluator · Oral Interview · Supervisor's Review Form · Clarifying interview · · Line of Evidence: all data/information supporting the score awarded in a knowledge/skill area, whether produced by the central office administrator or the evaluator Central Office Evaluation System Alabama Department of Education Central Office Field Test Evaluation

7. Central Office Evaluation System • Steps in Evaluation Process 1. Orientation (LEA responsibility) 2. Self-assessment (central office administrator) 3. Prepare portfolio (central office administrator) 4. Distribute and analyze surveys (central office administrator) 5. Supervisor Review Form (evaluator/supervisor) Alabama Department of Education Central Office Field Test Evaluation

8. Central Office Evaluation System • Steps in Evaluation Process (cont.) 6. Review portfolio (evaluator) 7. Conduct interviews (evaluator/central office administrator) 8. Prepare Evaluation Summary Report (evaluator) 9. Summary conference and PDP (evaluator/ central office administrator) Alabama Department of Education Central Office Field Test Evaluation

9. Generates data for all areas of knowledge and skills · Optionally used by the central office administrator and shared with evaluator only if central office administrator agrees · Used by central office administrator to: · 1) Identify areas that need improvement 2) Compare the central office administrator’s perceptions of performance with the results of the evaluation 3) Determine areas for professional growth activities to be included in a professional development plan. Central Office Evaluation System Self-Assessment Alabama Department of Education Central Office Field Test Evaluation

10. A Clarifying Interview with probing questions focused on clarifying or bringing understanding to the portfolio and/or written interview (questions differ from central office administrator to central office administrator) · Conducted after the evaluator analyzes central office administrator’s portfolio · Modified scripting is used to record responses Responses can be banked for one year · · Central Office Evaluation System Interviews Two types of interviews: · A Structured Interview with 6 questions commonly asked of central office administrators (oral or written response format) · Alabama Department of Education Central Office Field Test Evaluation

11. · Generates data for seven areas of knowledge/skills: · Communication (oral) · Collaboration processes and skills · Assessment/measurement/evaluation · Organizing for results · Planning · Problem solving · Technology management · Oral Option: A discussion between the central office administrator and the evaluator (or his/her designee) about particular areas of the central office administrator’s knowledge, skills, and performance. Central Office Evaluation System Structured Interviews Alabama Department of Education Central Office Field Test Evaluation

12. · Each of the structured interview question sets is intended to cover one knowledge and skill area (although responses may cut across more than one area). Communication skills are considered as part of the oral presentation. · The structured interview itself is not scored, but responses are added to other evidence provided through other sources. · Written Option: Central office administrator’s written responses to interview questions (will have clarifying discussion). Central Office Evaluation System Structured Interviews (cont.) Alabama Department of Education Central Office Field Test Evaluation

13. Conducted after the structured interview is completed with a focus on the written interview and, optionally, the central office administrator’s portfolio · No set number of questions, but key questions specific to the written interview and portfolio (optional) are developed by the evaluator to bring clarity and understanding of their contents · Responses are added to other evidence provided through other sources · Central Office Evaluation System The Clarifying Interview Alabama Department of Education Central Office Field Test Evaluation

14. Generates data for six areas of knowledge and skills: · Professional Responsibilities · Fiscal Leadership/Management · Laws/Policies · Innovation · Program Operations/Management · Leadership of Human Resources · Provides data where a supervisor should have objective, experiential information · Central Office Evaluation System The Supervisor’s Review Form Alabama Department of Education Central Office Field Test Evaluation

15. Generates data for 12 areas of knowledge and skills: · Communication Collaboration · · Assessment/Evaluation Organizing for Results · · Planning Laws and Policies · · Problem Solving Innovation · · Technology Management Program Operations/Management · · Fiscal Leadership Human Resources Leadership · · Central office administrator is responsible for providing some evidence of his/her knowledge and skills · The totality of the evidential material (lines of evidence for all knowledge/skill areas) will constitute central office administrator’s evaluation portfolio · Central Office Evaluation System The Central Office Portfolio Alabama Department of Education Central Office Field Test Evaluation

16. There is no one specific form of packaging required for the portfolio, but there are organizational requirements · · Each entry (piece of evidence) is accompanied by a Portfolio Entry Form · Total portfolio requires approximately 27 pieces of evidence · One piece of evidence might serve 2-3 purposes (e.g., Program Improvement Plan) · With multiple purposes, a portfolio may have 10-12 entries (pieces of evidence) Central Office Evaluation System The Central Office Portfolio Alabama Department of Education Central Office Field Test Evaluation

17. Explicit guidelines for developing a line of evidence in each area are provided in the evaluation manual · Evaluated on the quality of materials rather than the quantity of materials in the portfolio · "Extra" entries will not be considered by the evaluator · Central Office Evaluation System The Central Office Portfolio Alabama Department of Education Central Office Field Test Evaluation

18. · Generates data for 12 areas of knowledge and skills: · · Communication Collaboration · · Assessment/Evaluation Organizing for Results · · Laws and Policies Problem Solving · · Innovation Program Operations/Mgt. · · Fiscal Leadership Technology Mgt. · · Professional Responsibilities Human Resources Leadership · Three groups of respondents: Other Central Office Administrators · Principals · School staff, when applicable · · Respondents are to remain anonymous and represent a cross-section within the group Central Office Evaluation System Surveys Alabama Department of Education Central Office Field Test Evaluation

19. Numbers for administration: · Total for groups: 20 or all, if fewer in school system · Both system administrators and principals must be in respondent group · Analyses and reporting are responsibilities of central office administrator: · Information/report will be used with other evidence from other sources · Procedures and methodologies used by the central office administrator will in itself be used as part of evidence for assessing the knowledge/ skill area of assessment/evaluation · Analyses and reporting: · Number of respondents by subgroup surveyed · Patterns and interpretation · Any problems with data · Conclusions by knowledge/skill area · Central Office Evaluation System Surveys (cont.) Alabama Department of Education Central Office Field Test Evaluation

20. · Scores are not generated by instrument · There will be a total of 13 scores, one for each knowledge/skill area · At end of process, evaluator(s) will: 1. Organize evidence by knowledge/skill area 2. Read through all evidence for a knowledge/skill area 3. Award a score (1, 2, 3, 4) based on the patterns in the evidence in relation to the score level rationales for that particular knowledge/skill area 4. Place score with comments/justification on the Evaluation Summary Report 5. Use the scores and evidence for determining what should be addressed in the Professional Development Plan Central Office Evaluation System Scoring Alabama Department of Education Central Office Field Test Evaluation

21. · Used to present the scores developed from the lines of evidence for each of the 13 knowledge and skill areas To complete the Evaluation Summary Report, the evaluator: · 1. Completes demographic information requested 2. Records the scores developed from the lines of evidence for each knowledge and skill area 3. Writes any comments, explanations, or justifications in the space provided that support the score for the specific area 4. Identifies three areas for focus in planning professional development for the coming year(s) Central Office Evaluation System Evaluation Summary Report Alabama Department of Education Central Office Field Test Evaluation

22. Activities focus on areas of improvement as identified in the Evaluation Summary Report · Activities focus on the improvement of student achievement and school improvement · Used to generate portfolio entry for Human Resources Leadership after first year of evaluation · Professional Development Plan Central Office Evaluation System Professional Development Plan Alabama Department of Education Central Office Field Test Evaluation

23. QUESTIONS ?

24. Administrator Evaluation Systems Instrument Superintendent Central Office Principal Asst. Principal X X X X Self Assessment X X X X Portfolio X X X X Surveys X X X X Interviews Supr. Review X X X X Board Quest X Observation X X X X Eval. Summary X X X X Prof. Devel. Alabama Department of Education Administrator Evaluation

25. Administrator Evaluation Systems Instrument Superintendent Central Office Principal Asst. Principal Self Assessment All know/skills All know/skills All know/skills All know/skills 12 know/skills 12 know/skills 11 know/skills 12 know/skills Portfolio Cntrl Off/Prin/ Faculty Faculty/Parents Students/Comm Faculty/Parents Students Surveys Cntrl Off/Prin Interviews 7 know/skills 7 know/skills 8 know/skills 7 know/skills 6 know/skills 4 know/skills 6 know/skills Supr. Review All know/skills Board Quest 5 know/skills Observation All know/skills All know/skills All know/skills All know/skills Eval. Summary 2-3know/skills 2-3know/skills 2-3know/skills 2-3know/skills Prof. Devel. Alabama Department of Education Administrator Evaluation

26. Central Office Portfolio Knowledge Skill Area Evidence Communication 3 pieces of written communication · · Collaboration 3 artifacts (documents, tapes, etc.) · · Assessment/Evaluation 1 personnel evaluation file (if applicable) · · 1 document/disk synthesizing and presenting student achievement results · 1 document related to program evaluation · Organizing for Results 2 documents/artifacts representing marketing knowledge/skills, customer focus and/or change efforts · · 1

27. Central Office Portfolio Knowledge Skill Area Evidence Planning School improvement planning efforts · · 1 other document/artifact · Laws/Policies 2 artifacts/documents · · Problem Solving 2 artifacts/documents · · Innovation 2 artifacts/documents · · Technology Management · 2 artifacts/documents · 2

28. Central Office Portfolio Knowledge Skill Area Evidence Program Operations/Management · 2 artifacts/documents · Fiscal Management/Leadership 2 artifacts/documents · · Leadership of Human Resources 2 artifacts/documents · · 3