Making Meaningful Use of Data

1. Making Meaningful Use of Data Maximizing Hawaii’s Assessment Item Bank

2. Desired Outcomes • Revisit the purposes of the Assessment Item Bank • Understand the value of the Assessment Item Bank and how it supports other DOE efforts • Review quality classroom assessment, four types of data and a sampling of data analysis processes • Examine two critical process steps to make meaningful use of data • Organize Data and Analyze Data • Explore practical ideas, strategies, and resources to organize data and analyze data

3. Purposes of theData for School Improvement Assessment Item Bank • Support for teachers and principals by developing a common bank of robust and varied assessment items aligned to HCPS III • Allows teachers to use benchmark aligned assessment items for diagnostic, formative, or summative purposes at the student level • Provides a tool to help monitor and track student progress

4. Assessment Continuum • Determines prior knowledge and skills • Identifies misconceptions, interests, or learning style preferences • Provides information to assist planning and guide instructional decision making Pre-Assessment Formative • Ongoing, during the learning process • Provides information to assist planning and guide instructional decision making Involves students as partners • At the end, to evaluate and communicate learning • Informs future instruction as well as programmatic and overall curricular decisions Summative

5. Assessment Item Bank • Value • Provides teachers with a bank of assessment items aligned with standards for use in day to day instruction • Supports monitoring and tracking of student progress in the classroom • Aids in the collection of evidence for data-based instructional decisions • Serves as a resource to support the implementation of other DOE efforts

6. Standards Based Implementation Model Teacher collaboration throughout the process. Student involvement throughout the process. 1. Identify relevant content standards 2. Determine acceptable evidence and criteria 3. Determine learning experiences that will enable students to learn what they need to know and to do Reteach, or repeat the process with the next set of relevant standards 4. Teach and collect evidence of student learning 5. Assess student work to inform instruction or use data to provide feedback 6. Evaluate student work and make judgment on learning results and communicate findings MMann/SAS Adapted from WestEd’s Learning from Assessment

7. Rigor/Relevance Framework C D Evaluation 6 5 4 3 2 1 Synthesis Analysis Assimilation Adaptation RIGORKnowledge Taxonomy Application A B Comprehension Acquisition Application Knowledge/ Awareness 1 2 3 4 5 Apply to real-world predictable situations Knowledge in one discipline Apply in discipline Apply across disciplines Apply to real-world unpredictable situations RELEVANCEApplication Model www.leadered.com/rigor.shtml

8. Literacy for Learning Guiding Principles The Hawaii State Department of Education will embrace and promote the following guiding principles: Assessment of and for learning drives instruction 2. Evidence-based instructional practices 3. Aligned policies and resources at school, complex area and state levels 4. Instructional leadership and professional learning 5. Accountability 6. School, home and community partnerships

9. CSSS and RtI

10. Assessment Item Bank • Value • Provides teachers with a bank of assessment items aligned with standards for use in day to day instruction • Supports monitoring and tracking of student progress in the classroom • Aids in the collection of evidence for data-based instructional decisions • Serves as a resource to support the implementation of other DOE efforts

11. Using Assessment Data

12. Activity: Why is Data Analysis Important for Student Learning? • As a trio, review and discuss the quotes in relation to the prompt above. • Identify a quote or part of a quote that • Squares with your group’s thinking • Is pointed and stuck out to your group • Be prepared to share your selections and why. Something that squares with our thinking. Something pointed that stuck out

13. Classroom Assessments

14. Quality Classroom Assessment Clear Targets Clear Purpose Sound Design Student Involvement throughout the process Student Involvement throughout the process Use of assessment Effective Communication Adapted from: Classroom Assessment for Student Learning: Doing It Right – Using It Well

15. Quality Classroom Assessment Key Elements for Meaningful Data • Clear Purpose • Know why you are giving the assessment • Clear Targets • Know what information the assessment will provide about student learning • Sound Design • Know how you will assess for accurate results

16. Assessment Continuum • Determines prior knowledge and skills • Identifies misconceptions, interests, or learning style preferences • Provides information to assist planning and guide instructional decision making Pre-Assessment Formative • Ongoing, during the learning process • Provides information to assist planning and guide instructional decision making Involves students as partners • At the end, to evaluate and communicate learning • Informs future instruction as well as programmatic and overall curricular decisions Summative

17. Assessments can be. . . “Quick” & “On the Run” to “In-Depth” & “Deep” • The breadth and depth of your assessment depends on your needs.

18. A Classroom View of the Assessment Continuum 3 1 2 Summative Assessment Learning Activity #2 – teacher observation Learning Activity #5 – written response assessment Learning Activity #1 – exit pass Learning Activity #3 - quiz Learning Activity #4 – learning log Learning Activity #6 – white boards Clear Learning Target Pre-Assessment – KWL Formative Assessments are given throughout the learning process.

19. Connection to Assessment Item Bank • Clear Purpose • Reports generated in the Assessment Item Bank can provide data in an organized fashion linked to the teacher’s purpose for the assessment. • Clear Target • Items in the bank are aligned to benchmark easing the development of an assessment linked to specific learning targets. • Sound Design • Multiple items are available for each benchmark to support development of valid and reliable assessments.

20. Types of Datato Analyze

21. Four Types of Data • Demographics • Perceptions • Processes • Student Learning (student performance or outcome) data Dr. Victoria L. Bernhardt

22. What information does demographic data provide? • Clarify who our students are • SpEd, ELL, 504, Speech, etc. • Ethnicities • Socio-economic status • Gender • Number of years at the school • Learning style • Interests

23. What information does perceptual data provide? • Awareness and experience • Existing values, attitudes, beliefs, and opinions

24. What information does process data provide? • What is done to achieve a purpose • Procedures • Methods • Practices • Progressions • Courses of action • Examples • Standards Based Implementation Model • I do, we do, you do • Stages of writing development • Moving from concrete to representational to abstract

25. What information does student learning data provide? • How students are performing and what students know as a result of instruction • performance in relation to learning targets • Areas of strength and need • student learning • classroom instruction

26. Intersection of Four Types of Data Demographics Processes Perceptions Student Learning Dr. Victoria L. Bernhardt

27. Activity: Card Sort • Work with a partner and take 7 minutes to do the following: • Take out the four title cards. • Sort the remaining cards by data type. • Once you are done sorting, refer to the answer sheet provided to check your card sort.

28. Activity: Turn and Talk • With your partner, discuss the following: • What data types do I have readily available and regularly use? • What data types might I want to consider collecting more of? • Be ready to share some of your thoughts with the large group.

29. Connection to Assessment Item Bank • Serves as a source of Student Learning (student performance or outcome) Data • Has the potential to incorporate demographic data with student learning data

30. “Analyses of demographics, perceptions, student learning, and school processes provide a powerful picture that will help us understand the. . .impact on student achievement. When used together, these measures give. . .the information they need to improve teaching and learning and to get positive results.” What Data Are Important Data Analysis for Continuous School Improvement, Victoria Bernhardt, Ph.D.

31. Making Use of the Data through an Analysis Process

32. Total Quality Management Plan-Do-Check-Act (PDCA) Image from http://gramconsulting.com/wp-content/uploads/2009/02/plan_do_check_act.png

33. Bernhardt’s Problem Solving Cycle

34. Wahlstrom’s Process 1. Collect 2. Organize 4. Use 3. Analyze Using Data to Improve Student Achievement, Deborah Walhstrom, Page xiii

35. Depka’s Improvement Cycle View, discuss, and interpret data Analyze Evaluate effectiveness Assess Plan Goal set, strategize, and plan Set plan into motion Implement The Data Guidebook for Teachers and Leaders, Eileen Depka, page 20

36. Reeves’s Decision Making for ResultsSix-Step Process Inquiry; Develop Questions Decision Making for Results, Data-Driven Decision Making, Leadership and Learning Center, page 19

37. Activity: Turn and Talk • With a partner, discuss the following: • Which of these analysis processes are you familiar with? • What analysis process do you currently use to analyze data?

38. Using Assessment Data to Inform Instruction Analyze Data Organize Data

39. Of the data that we have, the most powerful is classroom data. That’s the data that most effectively drives instruction. -Dennis Fox

40. Assessments can be. . . “Quick” & “On the Run” to “In-Depth” & “Deep” The breadth and depth of your assessment depends on your needs.

41. How Often Do We Need to Do Data Analysis? “Assessment occurs all day long. . . It’s continuous—second by second, minute by minute.” -Shea, Murray, and Harlin

42. Organize Data

43. Why Organize Data? Eases analysis because it… • Helps to answer questions in relation to the purposes of the assessment • Puts data in a format that allows you to see patterns and trends • Keeps users from jumping to unsubstantiated conclusions

44. How to Organize the Data • Data must be organized in a way that is meaningful to the user. • Organize the data in a manner that helps to determine if the purpose of an assessment has been met.

45. Organization can be. . . “Quick” & “On the Run” to “In-Depth” & “Deep” The breadth and depth of your organization depends on your needs.

46. Some Ways to Organize the Data Overall performance class average, score distribution By Student Individual, different types of groups Content knowledge, skills, processes By Question By Item Type By Passage Type

47. Format for Organizing Data:Data Displays • A data display in research and evaluation is defined as “an organized, compressed, assembly of information that permits conclusion drawing and action.” (Miles & Huberman, 1994, p. 11) • All data analysis does not require a formal organization of the data in a data display. • When using a data display it should be one that is appropriate for the intended use.

48. Common categories of data displays • Tables • Graphs

49. Activity: Patterns and Trends • Independently do the following: • Examine the assigned data • Identify any patterns and trends • In groups of three to five, do the following: • Each person shares their independent findings • Be ready to share with the large group one or two patterns and trends identified from the data

50. Some Ways to Organize the Data Overall performance class average, score distribution By Student Individual, different types of groups Content knowledge, skills, processes By Question By Item Type By Passage Type