Maine Department of Education Update on Learning Results Implementation Issues

1. Maine Department of Education Update on Learning Results Implementation Issues Commissioner’s Conference Bar Harbor June 20-22, 2004

2. Organizational Culture: High Expectations LEARNER DEVELOPMENT & WELLNESS Resources & Commitment Technology & Methodology • Cognitive • Physical • Emotional • Ethical Standards & Accountability Shared Leadership Life-long Adult Learning

3. Breaking News!!! • 88.3% of responding districts (n = 112) report that their work to implement ELA assessments in grades 9-12 is either partially complete or complete; • 85.9 % math; • 77.6 in science; • 76.8 in social studies • 71.4% in H/PE

4. Breaking News!!! • What do districts report needing help with? • Developing strategies for using and reporting data; • Developing policies; • Managing the administration and scoring of assessments.

5. Informational Letters on Learning Results Implementation Issues • “Batch” distribution of informational letters on related topics designed to avoid “death by a thousand cuts” approach. • Opportunities for deeper discussion will be provided at Commissioner’s Conference and at summer Curriculum Coordinator workshops. • Each informational letter begins with “key points covered in this informational letter.”

6. Coming Attractions: Additional Updates Forthcoming • Update and findings from the Local Assessment Implementation Study (LASIS) • A guidance paper on establishing reliability in the LAS: Considering Consistency • Detailed implementation plan for MEA Online for 2004-05 • Detailed update on Department plan to review performance standards for the MEA and the GLE assessment.

7. Informational Letters on MLR Implementation Issues • Learning Results Implementation Timelines (#117) • Using the MEA in Local Assessment System (# 118) • GLE Assessment Plan (Letter # 119) • MEDMS Local Assessment System Module (#120) • Learning Results Review Process (#121) • 2004-05 MEA Administration Update (#122)

8. Learning Results Implementation Timelines • ELA and mathematics graduation requirement delay until 2007-08. • Now ELA, math, social studies, science, and health/ PE, will now be the basis for graduation in 2007-08. • The Legislature did not, however, extend the timeline for Local Assessment System (LAS) completion. • Specific expectations have been outlined on what superintendents should use as a basis for signing this year’s blue form. • Superintendents will be able to request a waiver if minimal expectations have not been met.

9. Using the MEA in the Local Assessment System • Policy Advisory Committee recommends method for using MEA scores in the Local Assessment System (LAS). • Since the MEA has no replacement, it is recommended that the test score be used only if it helps a student’s overall achievement record. • In order to achieve the proper balance of purposes, the MEA score should count no less than 15% and no more than 20% of the overall LAS body of evidence.

10. GLE Assessment Plan • GLEs in reading and math are posted on the Department website. • The GLEs represent a subset of the MLR, which will result in a more compact assessment than the 4th and 8th grade MEA. • The GLE assessment will be similar to the Measured Progress “Progress Toward Standards” (PTS) test. • The GLE assessment will provide a national comparison score. • The initial field test and pilot of the GLE assessment will take place in March 2005, but will not be used in AYP calculations until 2006.

11. MEDMS Local Assessment System Module • The Department will soon release an RFP for the MEDMS local assessment system module. • The MEDMS LAS module will be made available at little or no cost to the SAUs. • MEDMS LAS will permit aggregation of statewide data in all five content areas required to be implemented this year. • Local class data from SIS systems will be exportable to the MEDMS LAS module. • Our current expectation is that MEDMS LAS will be delivered to SAUs by December 2004.

12. Learning Results Review Process • Statute and rule require review process for Maine Learning Results to begin this year. • Review process will include all eight content areas, content standards and performance indicators for each, but not the Guiding Principles. • Review process will take 2-3 yearsand will then require action by the Maine Legislature prior to implementation of any revisions, which will not begin until 2006-07 at the earliest.

13. Learning Results Review Process(continued) • Any changes to the standards will be phased in over a period of years to ensure fairness to students and to allow districts ample time to make modificationsto curriculum, instruction and assessment. • A Distinguished Educator and the Maine Learning ResultsReview Committee will guide review process.

14. 2004-05 MEA Administration Update • MEA 2003-04 School and Student Reports will be released July 15, 2004 • Local validation of MEA data will precede calculation of AYP status • AYP status should be announced by August 15. • MEA Test Dates are March 7 – 18, 2005 • MEA Format Unchanged in 2005 • 8th grade MEA Online Certification by May 28, 2004 • 8th grade MEA Online to include writing, reading, mathematics and science in 2005

15. Local Assessment System Implementation Study (LASIS): Methodology: • Research team has guided work • collaboration of MDOE, MMSA, UMaine • 18 sites across Maine • fall and spring surveys, phone interviews, site visits, peer-review (of LAS materials)

16. Local Assessment System Implementation Study (LASIS): Preliminary findings: • data being analyzed and summarized as we speak • salient themes evident nevertheless • final report due in August

17. Local Assessment System Implementation Study (LASIS): How do SAUs organize for LAS development? An SAU that is making progress toward LAS development tends to report some or all of these features:

18. Local Assessment System Implementation Study (LASIS): • Strong central coordination with clearly delineated responsibilities. • Broad participation: administrators (SAU, school), curriculum specialists, teacher leaders, and teachers. • Committees that hold regularly scheduled meetings and report on these meetings to administrators and staff.

19. Local Assessment System Implementation Study (LASIS): • The provision of time for teachers to learn about and work on LAS is an organizational priority. • Formal communication (including a variety of media) to teachers and stakeholders about LAS development.

20. Local Assessment System Implementation Study (LASIS): • What are the perceived “barriers” to LAS development? • Lack of administrative support • Lack of organizational structure • Administrative turnover • Lack of time—for professional development and the LAS work itself • Lack of money

21. Local Assessment System Implementation Study (LASIS): • Perceived Barriers: • Cynicism about LAS process and/or requirements • Overwhelming scope of LAS work • Confusion about requirements, unclear definitions, history of DOE rule changes, lack of DOE formal guidance • Resistance to change

22. Local Assessment System Implementation Study (LASIS): • What are the perceived “boosters” that facilitate LAS development? • Strong leadership (administration) regarding LAS development • clearly articulated vision • support for those doing the work • strong centralized coordination • leadership continuity • knowledge of curriculum

23. Local Assessment System Implementation Study (LASIS): • Perceived boosters: • Dedicated teachers • Positive attitude toward LAS, among teachers and administrators alike • Believing in the intrinsic value of LAS (e.g., “We should be doing this anyway.”) • Collaboration with other SAUs

24. Local Assessment System Implementation Study (LASIS): • Perceived boosters: • MDOE • LAS Guide • MAP & LAD tasks • MDOE-sponsored training events • personnel • Obtaining grants to support LAS development

25. Local Assessment System Implementation Study (LASIS): • Next steps: • final report in August • MDOE commitment to continuing it’s study of LAS implementation over the next three years (reliability, MEA, performance standards, and graduation decisions) • Independent review of LAS (possible consultation from NRC or researchers from Nebraska evaluation)

26. Legal Considerations of the LAS and Graduation Decisions • Conversation with Sarah Forster, Assistant AG: • Courts have typically used the following: • Are the standards for graduation clearly articulated and students given fair notice? • Do students have fair opportunity to learn the standards? • Are tests or assessments used to determine graduation fair measures of student achievement of the standards? • Are sub-groups of unfairly disadvantaged due to bias?

27. Legal Considerations of the LAS and Graduation Decisions • Additions thoughts: • Educators are not expected to be statisticians • Systems of evaluation need not repudiate subjective judgment • Build an educationally sound system and the courts will follow • Technical standards should provide confidence in decisions about students

28. Keeping Local Assessment System Development in Perspective • Ensure that we invest in acting on local assessment data to inform teaching and learning. • Communicate with our publics to explain why the investment in standards-based reforms are important for our young people and for Maine’s future. • Keep the whole child and the whole organization in mind.

29. Organizational Culture: High Expectations LEARNER DEVELOPMENT & WELLNESS Resources & Commitment Technology & Methodology • Cognitive • Physical • Emotional • Ethical Standards & Accountability Shared Leadership Life-long Adult Learning

30. Additional Supportive Initiatives: • Regional professional development centers with focus on mathematics • Title II A and B secondary mathematics project • MLR and LAS policy work with MSMA • Peer review tool from LASIS • Regional rounds of workshops this summer • VPA, MCL, and CP assessment design work • Coalition public information campaign

31. Rethinking Accountability Events • Series of four events spanning 2004-05 • Designed for school or district teams: board, superintendent, administrator, teacher, and parent • Course credit offered through USM • Purpose: to assist local teams to build internal, data-driven accountability systems, develop new understandings about a balanced view of accountability, and take action.

32. A Final Thought Our current system is perfectly designed….. To produce the results were currently getting.

33. Department Contact Information • Learning Results: patrick.phillips@maine.gov • LAS: pam.rolfe@maine.gov • MEA: brud.maxcy@maine.gov • MEDMS: bill.hurwitch@maine.gov • Learning Results Review: don.reutershan@maine.gov • GLE Assessment: brud.maxcy@maine.gov