NCLB AND VALUE-ADDED APPROACHES ECS State Leader Forum on Educational Accountability June 4, 2004 Stanley Rabinowitz,

1. NCLB ANDVALUE-ADDED APPROACHESECS State Leader Forum on Educational Accountability June 4, 2004Stanley Rabinowitz, Ph.D.WestEdsrabino@wested.org

2. GUIDING QUESTIONS • What are the goals/assumptions/values of Value-Added accountability models? • What are the pros and cons of Value-Added accountability models? • What are the requirements of Value-Added accountability models (data, technical, political)? • Are there alternatives that provide “similar” information?

3. ACCOUNTABILITY MODELS • Status • Cohort • Same students (unmatched) • Same students (matched)—“Value Added” ------------------------------------------------------ • All models are defensible (assumptions, values) • All models can be interpreted as “growth”

4. ACCOUNTABILITY MODELS (cont.) • Multiple (Simultaneous) Models - two tier (funnel into state system) - federal and state system (side-by-side) - combine status and growth - formal and informal systems (program improvement) • Advantages of multiple systems - differing assumptions may be appropriate for different audiences and purposes - approval • Disadvantages of multiple systems - need to administer multiple systems - interpret/explain differing results

5. CHALLENGES OF A VALUE-ADDED SYSTEM • Values • Assessment - content - technical • Data systems • Political

6. CHALLENGES OF A VALUE-ADDED SYSTEM: VALUES • Concept of “one-year’s growth” (across grades, content areas, students) • Individual student growth most important measure of school effectiveness (student by student) • Classroom/teacher effect most important factor in student success (vs. school, district, home, community) • Student test scores are a proxy for effective teaching • Changes in standardized test scores are what is valued

7. CHALLENGES OF A VALUE-ADDED SYSTEM: ASSESSMENT • Test questions of varying difficulty in order to adequately discriminate among the range of achievers typically found in a classroom, school, district, state • Sufficient validity and reliability of the underlying instrument(s) (reliability of difference scores) • A set of scaled tests, vertically equated, aligned to the curriculum • NRT vs. CRT

8. CHALLENGES OF A VALUE-ADDED SYSTEM: DATA SYSTEMS • Longitudinally merged student data base (unique student identifiers) • Student data need to be traceable back to the focus of accountability (teacher, classroom, school, district…) • Data to be controlled for other factors (e.g., socioeconomic indicators, disabilities) • Unmatched students? • Mobility across state and across schools? • Multiple teachers?

9. CHALLENGES OF A VALUE-ADDED SYSTEM: POLITICAL • NCLB status • Black box vs. transparency of system • Specific attribution of blame/responsibility • Expected opposition • Large year-to-year swings in value-added scores that administrators could not explain • Assumption that all learning takes place in the classroom, although the model "has not provided adequate evidence to support this contention"

10. PSEUDO VALUE-ADDED MODEL • Fully aligned content standards • Fully aligned performance standards • Sufficient variability within performance categories (e.g., break out basic into low, medium, high basic) • Accountability based on growth of individual students or groups (within and across categories) • Assign point values for different types of growth - value of reaching proficient - value at other points (advanced, at least basic) - value of lowest category (0?) • Focus attention at performance labels, especially Proficient • Lower technical requirements • Easier political discussion