OCRI Symposium

1. O.C.R.I Oregon Collaborative Research Initiative OCRI Symposium “Investigating the efficacy of teacher work sample methodology in teacher preparation” AACTE February 25th, 2007 OCRI is a joint effort between the Oregon Association of Colleges of Teacher Education (OACTE) and the Oregon Association of Teacher Educators (ORATE).

2. This Symposium is Dedicated to the Memory and Legacy of Dr. Del Schalock Dr. Del Schalock, Professor, The Teaching Research Institute, Western Oregon University Lead Developer and Advocate of the Teacher Work Sample Methodology (TWS)

3. Symposium Overview • Introduction • Paper 1: Collaborative research design in teacher preparation: Blueprints and processes • Paper 2: Evolution of policy implementation of teacher work samples in Oregon • Paper 3: Perceptions of teacher educators and candidates on the use of teacher work samples • Paper 4: Teacher work sample effects on the learning of K-12 students • Discussion

4. Introduction

5. Paper 1 Collaborative research design in teacher preparation: Blueprints and processes Ronald A. Beghetto (University of Oregon) Linda Samek (Corban College)

6. Imagine… • A focused, coherent multi-institutional collaborative research initiative • Nearly a dozen institutions (public and private) • Engaged in meaningful and sustainable teacher education research. • This vision is a reality • With several partnerships across the nation • And, most recently, a new network in the state of Oregon • The Oregon Collaborative Research Initiative (OCRI).

7. Blue Prints and Processes • Under what conditions do such partnerships develop? • How do collaborative groups develop and carry-out their research agendas? • What seems to support such efforts? • What challenges do such efforts face?

8. Historical Context

9. The Diversity of OACTE Institutions • 19 institutions • 6 public universities • 3 research and/or urban • 3 regional • 12 private non-profit colleges and universities • 6 religious • 6 secular • 1 for profit university • Small to large • From newly approved with no completers to • Hundreds each year • New to old • Multnomah Bible College program to begin fall 2007 • Western Oregon University, preparing teachers since 1882

10. Precursors to OCRI • All subject to rules governing programs from the Oregon Teacher Standards and Practices Commission • Mid 1990s – Oregon Collaborative for Excellence in Preparation of Teachers, a 5-year, $5 million NSF collaborative that included public, private, and community colleges • 1998-2001 – Oregon Quality Assurance in Teaching, a 4-year, multi-million dollar grant that involved the 16 four-year institutions with teacher preparation programs • Common interests in the education of children through OACTE

11. Perfect Storm that Spawned OCRI • Leaders from all institutions have common goals • Oregon TSPC requires evidence of competent performance and student learning • All student teachers are required to produce Teacher Work Samples • More Oregon institutions interested in NCATE accreditation • Decentralized education enterprise leaves everyone under-funded • OACTE had treasury funds that needed to be used

12. Sowing the Seeds of Collaboration

13. Building a Flexible Blueprint • Approach • Listen, synthesize, and verify • Workflow • An iterative, multistage process of planning and activity Research Designer CORT Project Strand Leaders Local Participants & Partner Sites

14. Primary Goals of OCRI • The Oregon Collaborative Research Initiative(OCRI) was established with the purpose of: • Engaging Oregon's teacher preparation institutions in the collaborative study and dissemination of a meaningful and sustainable set of research projects.

15. Meaningful • Meaningful in that the projects address questions of common interest, are aimed at improving teacher preparation, and have real-world implications for teacher development and PK-12 student learning.

16. Sustainable • Sustainable in that the research projects do not overburden participants but rather offer feasible and rewarding opportunities to engage in and disseminate the findings of collaborative inquiry.

17. Collaborative Focus • Our initial collaborative endeavor has focused on examining the efficacy of teacher work sample methodology in preparing teachers in the state of Oregon.

18. TWSM: An Initial Project • Three interrelated questions: • How has the TWSM evolved? • What is the experience of teacher educators and candidates with the TWSM? • What effect has the TWSM had on PK-12 students?

19. Example Planning Tools

20. Evolution Study • Goal: Examine how TWSM first became a mandate and how implementation has evolved over time. • Brief overview: Using a “Theory of Action” framework, juxtapose the enactment of TWSM with its intended and espoused theory.

21. Experience Study • Goal: Document the perceptions of teacher educators and candidates regarding TWSM. • Brief overview: Participants’ perceptions (representing public and private institutions) regarding the meaningfulness, usefulness, and applicability will be examined and reported on.

22. Effect Study • Goal: Examine how TWSM has impacted PK-12 learners. • Brief overview: Analysis of completed work samples to examine how work samples are impacting K-12 learning. This effort also focuses on the development and refinement of research/measurement tools most appropriate for analyzing the impact of TWSM.

23. Supports and Challenges

24. Key Metaphors • “Cyclical scaffolding” and “loose coupling” of parallel efforts as metaphors for supporting collaborative research design • Key challenges: • Anticipating and providing “just in time” assistance • Helping to navigate research strand “autonomy” and “relatedness” • Issues and challenges with respect to intellectual property (e.g., datasets), secondary analyses, and dissemination of findings.

25. Concluding Thoughts • Even though the Oregon group is still in the early stages of this process, • the project has already attained no small measure of success. • Only time will tell what the Oregon group is able to accomplish in this ambitious endeavor. • W are highly optimistic and, indeed, energized by the idea that • multiple institutions can come together • To forge a strong link between the research and practice of teacher education, and • Work towards enhancing teacher preparation and, in turn, PK12 student learning.

26. Evolution study

27. Paper 2 – Evolution Study Evolution of Policy and Implementation of Teacher Work Sample in Oregon Presenter: Hilda Rosselli - Western Oregon University Research Team: Dew Anna Brumley - Warner Pacific University Maria Cirielllo - University of Portland Hilda Rosselli - Western Oregon University

28. Research Questions • What led to teacher work samples becoming mandated? • How have teacher work samples evolved over time within the state of Oregon? • To what extent do dominant patterns of implementation run counter to the major intended premises of the state’s policy? (espoused theory and theory in use)

29. Research Design • Phase 1 • Historical--4 qualitative interviews • Key players involved with the early stages of teacher work samples policy development • Phase 2 • Survey data collected from 8 institutions • Early implementation, evolution of practice, challenges that have evolved • Phase 3 • Document analysis

30. Methodology • Four individuals selected for interviewing representing varying perspectives • Del Schalock—Teacher researcher • Dave Myton— Teacher education policymaker • Kathy Wiper—Teacher on state commission • Holly Zanville—Higher education official

31. Methodology • Conducted 4 semi-structured interviews • Transcribed interview notes • Conducted membership checks • Coded transcripts individually • Compared coding schemes • Reviewed and refined themes

32. Theme 1: Original Intent • What were teacher work samples designed originally to accomplish? What were they a reaction to or remedy for? • An evidence-based process designed to demonstrate that student teachers can teach and that students learn under the guidance of a student teacher. • Legislators were looking beyond teacher mandated standardized tests for more accountability from teacher education programs about their graduates.

33. Theme 1: Original Intent • What were teacher work samples designed originally to accomplish? What were TWSs a reaction to or remedy for? • Oregon Teacher Standards and Practices Commission were looking to output standards for program review purposes • Going beyond what do teachers know and do to producing evidence that indicated student learning.

34. Theme 1: Original Intent “Document teacher performance beyond knowledge and connected to the standards” Del Schalock “…analogy of the driver’s test. Not enough to just take a written driver’s test. Need to demonstrate the skill to drive.” Holly Zanville

35. Theme 1: Original Intent [It also] “ arose out of a dissatisfaction with the practice of using tests to demonstrate teacher competence” Holly Zanville “At Western it was clearly to go beyond the wall we had hit with competency based teacher ed where the issue is not what teachers know and can do.By 1974 we knew we had to go somewhere else because that mode didn’t help us so we realized we needed to connect teaching and learning.” Del Schalock

36. Theme 2: Roots and Context • What else was happening at the same time that influenced the development and adoption of TWSM? National, regional, state contexts? • Intersection of key events • Development of competency based programs • Creation of state curriculum standards in K-12 • Increased accountability for teacher prep

37. Theme 2: Roots and Context • Other factors • Elementary Secondary Act and engagement with schools (late 60’s and 70’s) • Com-Field- a study funded through NWREL offered a model for competency based, field-centered, and personalized teacher preparation (65 - 74) • Formation of TSPC, the nation’s first independent commission for teacher education (1971-72) • Commitment to context for research: Teaching Research Institute and College of Education at WOU • TSPC revision of programs to standards-based (1973)

38. Theme 2: Roots and Context • Other factors • Mid-Valley Consortium to assist districts with goal-based education and integration of Curriculum, Instruction, and Assessment (1975 - 1984) • Publication of Nation at Risk • Report to Joint Boards referencing work samples (1986) • 21st Century Schools Act in Oregon (1991) • Analysis of this act and its implications for teachers. • Recasting of teacher education standards to outcome based (1997)

39. Theme 2: Roots and Context “ The school context in which teacher ed was being fashioned became much more outcome oriented, e.g. what we want our kids to know and be able to do and hold ourselves to responsibility for assessing that. Wasn’t that big a jump to apply this to teacher ed which led to TWSM.” Del Schalock

40. Theme 2: Roots and Context “ComField was field centered—a graduated set of field experiences—moving from observations, to small group teaching, to full responsibility for teaching. That was a frame of reference that was firmly in the faculty minds so we married that with the goal based curriculum instruction demands for Oregon’s standards based movement.” Del Schalock

41. Theme 2: Roots and Context “Behind closed doors [they] were able to pinpoint right to the classroom door where students were meeting the expectations….what they were getting at with the student test data that they were beginning to be able to look at effective schools.” Dave Myton

42. Theme 2: Roots and Context Differing opinion “I don’t think TWSMs were intended to address the credibility of the institutions but it did allow conversation with legislators in which Colleges could say we are using multiple measures to assess quality.” Holly Zanville

43. Theme 3: Instrumental Players • What individuals or agencies or political issues were instrumental in the start up and then evolution of TWSM? • Del Schalock • Theory and practice context at WOU • TSPC

44. Theme 3: Instrumental Players • Who sat down and crafted the first draft? “I think I did.” Del Schalock “…75% probability that the language of TWSM came out of the research team in the mid 70s. The last half of the 70s was where it was being formulated. Tested at WOU in the late 70s or early 80s. Had sufficient testing within WOU before it went to TSPC.” Del Schalock

45. Theme 3: Instrumental Players “They [TRI/WOU] built and still maintain a huge database of student test results…got the superintendents at the table looking at the data and it was all over the place. So their concern was how to ensure that kids are getting not only a good education. Gerald????so he created a formula. It filled up the whole front of a classroom…but what it was to see if there were correlations with any of the inputs.” Dave Myton

46. Theme 3: Instrumental Players “The 21st Century Schools Act didn’t mention teacher licensing but the Board of Ed and the Board of Higher Ed both indicated that they were going to do something, they were going to take Teacher Ed back, if the Commission didn’t do something pretty drastic.” Dave Myton

47. Theme 3: Instrumental Players “So in the end, it kind of turned out to be the commission taking hold of it and saying we want them [student teachers] to have a work sample, you can waive anything else.” Dave Myton

48. Theme 4: Evolution of Intent • What other purposes did teacher work samples attempt to serve over time? • Stronger clinical supervision model • Follow-up on teacher candidates performance • Quartile learning gains for groups of students • Shift of emphasis from learning gains to also more focus on student teacher performance

49. Theme 4: Evolution of Intent “Maybe unintended but one of the purposes was to create a closer link between student teachers and mentor teachers in the classrooms. Became a tool for tightening the expectations for student teaching and an opportunity to work together with the school personnel. Mentoring a student teacher couldn’t be a time to go sit in the lounge.” Holly Zanville

50. Theme 4: Evolution of Intent “The responsibility for the evaluating of work samples never became the CTs [cooperating teachers’] responsibility because of the OEA opposition.” Kathy Wiper