Using Data to Improve Student Achievement, Graduation Rates and College- and Career-Readiness Illinois HSTW Assessment Data Workshop The Parke Hotel Bloomington, Illinois October 12-13, 2010
Welcome! Do Now! • Turn to page 10 of your workbook • What actions has your school/district/state taken in the last two years to advance student achievement ? Consider these areas: • Engaging the faculty • Having more students complete a rigorous curriculum • Having higher expectations and/or providing extra help • Providing quality CT studies and/or work-based learning • Literacy across the curriculum • Numeracy across the curriculum • Engaging science practices • Providing guidance and advisement • Select two or three actions and jot them down to share out later.
Workshop Objectives and Deliverables • Understand the 2010 HSTW Assessment and report • Use data to analyze your school/district/state’s challenges and develop an action plan for addressing those challenges • Develop a plan for engaging others in analyzing assessment results and using data to make change
The Southern Regional Education Board (SREB) • Founded in 1948 to improve the plight of the south through a focus on education • Nonprofit, nonpartisan organization • Works with state leaders and policymakers in the 16 member states • Provide data to legislatures and state boards of education for decision making • Focus on improving education pre-K through postsecondary • Network states
High Schools That Work (HSTW) • The largest and oldest of SREB’s school improvement initiatives • 1987 – 27 sites • 2010 – 1,200+ sites in 30+ states • Additional initiatives • Making Middle Grades Work for middle grades schools • 1998 – 25 sites; 2010 – 450+ sites in 22 states • Technology Centers That Work for shared-time technology centers • 2010 – 150+ sites in 15 states
HSTW Key Practices • High expectations • Program of study • Academic studies • Career/technical studies • Work-based learning • Teachers working together • Students actively engaged • Guidance • Extra help • Culture of continuous improvement
What will it take? • Improvement at the same time • Student achievement • Graduation rates • College- and career-readiness • Using data for improvement • Process • Performance
Data Walk • The facilitator will direct your group to a data poster • We will rotate clockwise to each poster (10 total) • Information about the posters will be added after each rotation • Go to the first poster
Poster 1 • How is this poster organized?
Poster 2 • There are multiple measures of data on these posters. What type(s) of data was collected and analyzed to provide this information?
2010 HSTW Assessment • NAEP-like subject tests in reading, mathematics and science • Scale of 0-500 • Student survey • Course history • School and classroom experiences • Teacher survey • Data/survey linked to Key Practices
Poster 3 • What is the difference in the type of data on these posters? • What is the value of using both types of data?
Poster 4 • Where are college- and career-readiness indicators located on the posters? • Discuss your definitions of college- and career-readiness.
HSTW Readiness Goals • Reading = 250; Mathematics = 257; Science = 258 • Students performing at this level are able to enter postsecondary studies without needing additional preparation and/or are able to pass most employer exams for entry-level jobs.
Poster 5 • How does this data relate to other data your school typically reviews? • How can this data and your other school data be used together?
Poster 6 • What is the relationship between the level of emphasis students experience on the key practice and their achievement?
Understanding the Indices • Based on the HSTW Key Practices • Example – Literacy Across the Curriculum • A series of indicators (and frequency) are identified for each practice • Example – Students often revised their essays or other written work several times to improve their quality • The number of indicators experienced determines the level of emphasis • Example – Intensive = 8-10 indicators
Poster 7 • Predict for your site the two indicators on which your students will score highest. • Why did you pick them?
Poster 8 • Predict for your site the two indicators on which your students will score lowest. • Why did you pick them?
Poster 9 • Place a check mark next to the indicators for which it will cost money for more students to experience.
Poster 10 • Place a star next to the indicators over which adults in the school community have control.
Data Walk Debrief • Relationship between school and classroom practices and student performance • What can we control? • What will it cost to make changes?
Data Walk Debrief • Effort, not ability • Focus on improving students’ experiences • Focus on things we can control • Take ownership of what we can control
Essential Question • Why do students at most-improved schools/centers make greater gains in achievement than students at least-improved schools/centers?
The Detailed Answer • More students at most-improved schools: • Completed the HSTW-recommended curriculum in English/Language Arts, mathematics and science • Experienced high expectations • Experienced reading, writing and mathematics across the curriculum • Experienced challenging and engaging authentic projects and assignments • Experienced high-quality career/technical studies with integrated academics • Had access to extra help and guidance • Understood the importance of learning and doing well in high school
The Short Answer • Most-improved schools more fully implemented the HSTW design • They took action to increase student achievement
Why is data important? • Replaces hunches with facts • Enables us to address root causes instead of symptoms • Enables us to determine the effects of implemented practices • Assists us in understanding how school and classroom practices relate to student achievement
Data cannot help when… • It is not valid or reliable • The appropriate questions are not asked • Analysis is not used for making wise decisions • There is no commitment by teachers and administrators to change school and classroom practices when data indicates the need for change
Interpreting the 2010 HSTW Assessment Topic Two
2010 HSTW Assessment • NAEP-like subject tests • Reading, Mathematics, Science • Student Survey • Course history • School and classroom experiences • Teacher Survey • Administered to all or a random sample of 60+ seniors in January/February 2010 • Comprehensive report disaggregating achievement by school and classroom practices
2010 Participation Illinois HSTW schools 916 students 1, 014 teachers, 821 academic and 164 CTE • HSTW • 757 schools • 40,428 students • 34,865 teachers • TCTW • 111 centers • 6,106 students • 2,706 teachers
2010 HSTW Assessment ResultsAll HSTW-TCTW Sites 56% 53% 52%
2010 HSTW Assessment ResultsAll HSTW-TCTW Sites HSTW Goal = 85% ? ? ?
Report Format • Executive Summary (pg. 1-2) • Overview (pg. 3-24) • Full Report • Indices (pg. 25-36) • Benchmarks (pg. 37-60) • Student survey results (pg. 62-183) • Teacher survey results (pg. 184-214) • Appendix (pg. 215-230)
Test Development • Why revise the assessment? • Refresh the subject tests • Add utility • Purpose • Continue measuring continuous school improvement • Add measure of college- and career-readiness
Test Development • Determine broad guidelines • Specify content to be included • Develop and review test questions • Write performance level descriptors • Review performance level descriptors • Administer assessment • Standard setting workshop • Research and validity studies
ETS Concurrent Validity Study • 2008 assessment data • Correlated HSTW scores with SAT, ACT, state tests and GPA • Georgia, Kentucky, Ohio, South Carolina, Texas and West Virginia • 51 schools participated, including almost 2,600 students • A second study is being conducted with 2010 data