Data collection & Analysis Practices

1. Data collection & Analysis Practices Caitlin Reash – University of Mount union

2. DEMOGRAPHICS Student Enrollment: 440 students 9.6% Black, 4.5% Multi-Racial, 85.2% White 62.7% Economically Disadvantaged 20.9% Students with Disabilities Faculty: 30 General Education teachers, 7 Special Education teachers, 4 Title One teachers (2 reading, 2 math), 1 Guidance Counselor, and 1 Principal. 100% hold a Bachelor’s Degree – 66.7% hold a Master’s Degree School: Designated as a Medium-High Poverty school Many students receive breakfast at the school daily and are on Free and Reduced lunch Parent involvement would be considered low by most teachers LRC rating: Continuous Improvement Value-Added was met Culture of Data Collection and Analysis: In this school, data collection and analysis is mainly done by classroom teachers, with a few exceptions. Teachers are required to administer, score, analyze, and make the data available to parents (if applicable). Teachers understand the purpose for most of the assessments, but still feel that assessments take up a significant amount of their teaching time. Data is typically received and reviewed in a timely manner, as the results are available online the same day the test is given for some of the tests. Teachers use Teacher Based Team meetings to review and analyze the data for various purposes. Assessments are used for student achievement, progress monitoring, and for intervention strategies. They are also used for grouping students based on skill sets.

3. Process Analysis Gathering the Data: To gather the data, I spoke with the principal first to ensure that I had her approval before continuing. This was done during a face to face conversation, and at that time, I also generated a list of the assessments that were administered in the building. The next day, I went to a teacher from each grade level and asked her if she would be willing to be informally interviewed about the assessments in her grade level. All teachers that I asked were willing to speak with me, so I let them know that I would follow up with them shortly, which I did through face to face interaction. When I met with the teachers I spoke with, we reviewed each of the assessments that they administer whether it is a standardized assessment, unit (reading series) component, or one that they created as a grade level. In a couple instances, there was more than one teacher present during the interview (which I had let the teacher I talked with know was alright). This was helpful because it provided me with a more comprehensive view of the assessments, along with their grade level feelings towards them. Stakeholders: Throughout this project, I spoke with the Principal in the building, along with the Special Education Director who also serves as a testing organizer (which I did not know until this process), and eight teachers from various grade levels and positions. Effectiveness: The way in which data was collected for this project took some time, but was effective in that I was able to talk with teachers at each grade level about the assessments they administer. Had I only talked with teachers from a grade level or two, I would not have the understanding that I do about the beliefs of each grade level team when it comes to their assessments. During this process, I found out that there was a testing organizer, which I feel was something that should have been communicated to all teachers. If teachers needed the testing information for each grade level quickly, there is not a document or way for them to get it without talking to individuals in the building. This project allowed me to create a chart that will make the information much more accessible if it is used.

4. Grade Level Assessments Kindergarten-4th Kindergarten First Third Second Fourth Weekly: Reading Checks – Aligned with Series Monthly: Pre and Post Tests-Math Unit Tests aligned with Treasures Curriculum Criterion Referenced (SS, SCI, and MATH) Beginning of the Year: DIBELS (September) STAR Math & Reading (September) IOWA (September) OAA Language Arts Practice (October) Middle of the Year: DIBELS (January) STAR Math & Reading (January) End of the Year: DIBELS (April) OAA (May) STAR Math & Reading (May) Beginning of the Year: DIBELS (September) STAR Math & Reading (September) OAA Language Arts Practice(October) Middle of the Year: DIBELS (January) STAR Math & Reading (January) End of the Year: DIBELS (April) OAA (May) STAR Math & Reading (May) CTBS: GIFTED (BASED ON REFERRAL) Monthly: Post Unit Assessments aligned with Treasures Beginning of the Year: DIBELS (September) CTBS (September) STAR Math & Reading (September) Middle of the Year: DIBELS (January) STAR Math & Reading (January) End of the Year: DIBELS (April) STAR Math & Reading (May) State Diagnostic (May) Monthly: Pre and Post Tests Math and Reading- aligned with curriculum Beginning of the Year: KRAL (August) DIBELS (September) Middle of the Year: DIBELS (January) End of the Year: DIBELS (March) STAR Early Literature (APR) Running Records (MAY) CTBS: GIFTED (BASED ON REFERRAL) Monthly: Pre and Post Tests Math and Reading- aligned with curriculum Beginning of the Year: DIBELS (September) STAR Math & Reading (September) Running Records (Sept) Middle of the Year: DIBELS (January) STAR Math & Reading (January) Running Records (Jan) End of the Year: DIBELS (April) STAR Math & Reading State Diagnostic (May) Running Records (May) CTBS: GIFTED (BASED ON REFERRAL)

11. KRA-L Kindergarten Readiness Assessment - Literacy Kindergarten August Reading levels when entering Kindergarten • The KRA-L is used by teachers to determine the levels of students entering Kindergarten. • Results are reported to the state. • Testing occurs during the first three days of school for the rest of the building (1-4). • Kindergarten begins three days later than grades 1-4 to allow time for testing/placement. • Once testing is complete, Kindergarten teachers come together to analyze the data. • Teacher have two to three hours to review scores, group children based on scores, and distribute into classes. • Many of the Kindergarten teachers felt rushed to group students and be prepared for parents to come the same afternoon that data is analyzed/students placed. • During a teacher based team meeting, Kindergarten teachers, along with the school principal and county representative, discussed changes that could be made regarding the KRA-L assessment. • Overall, teachers felt that the KRA-L was not comprehensive and representative of what students should be tested on. • Additional time was suggested in order to also administer a teacher created assessment for math and other reading skills (not yet created). • An increase in the time to analyze scores and the way in which children were grouped was also shared. • Teachers would like heterogeneous groupings instead of the current “high, medium high, medium low, low” groupings.

12. DIBELS Dynamic Indicators of Basic Early Literacy Skills Kindergarten-4th Sept, Jan, & Apr Literacy Skills/Title One Reading Intervention • DIBELS is administered by Title One reading teachers three times a year. • Results are made available to teachers the same day as the test is given to his/her class. • The information collected is analyzed by the Title One teachers at the completion of testing for grades K-4. • Based on the students’ scores, reading intervention groups are determined to provide supports to at-risk students. • Currently, a two day pull-out, two day push-in model is being used to provide necessary interventions. • In Kindergarten only, parents receive a notice as to whether their child has met benchmark or not. • Title One teachers, along with classroom teachers, are currently questioning the effectiveness of the current push-in model because of the success/student improvement that was achieved through the five day pull-out model. • The main reason for this is that in the current model, the student to teacher ratio is 13:1, whereas in the pull out model, the ratio is at most 4:1.

13. STAR Early Literacy, Reading, and Math End of K-4th Sept, Jan, & May Progress Monitoring/Intervention Groupings • The STAR test is administered by grades K-4. At the end of Kindergarten, students take the STAR Early Literacy to determine their reading level entering First Grade. • Results are made available to teacher immediately due to the electronic format of the tests. • The scores are then analyzed by teachers, including Title One math, in order to monitor progress and determine intervention groupings. • This also helps to determine the skills to be addressed during both math and reading interventions. • Based on math scores, the Title One teachers conduct mini lessons (pull-out) in addition to the push-in math intervention time. • Student scores and analysis are shared with parents after each time the test is administered. • Teachers interviewed are pleased with the types of questions used, the speed/ease of getting results, and the data/interventions suggested based on the student scores. • Teachers also find that this is an effective way to monitor progress. • Second grade teachers are considering using this test monthly because of the data provided and how its use in effective interventions.

14. State Diagnostic Required by Ohio in grades K-2 K-2 May Determine student achievement levels • The State Diagnostic test is required by Ohio to be administered in Kindergarten through 2nd grade at the conclusion on the academic year. • Teachers administer the test and results are made available immediately. • At the current time, these results are not analyzed, but a spreadsheet is made in order to show the state that tests were administered. • The school principal said that this can help to place students in appropriate classes the next year. • Teachers interviewed feel that the current way it is being used is not effective, and they do not understand the need for it other than it is required.

15. Running Records Reading End of K-1 Sept, Jan, & May Track reading ability/Show reading growth • Running records are completed by classroom teachers in order to determine students’ current reading levels. • Given three times a year, results are also used to show progress made by students. • Results are immediate due to the fact that teachers assess each student. • Information is used by teachers in order to select the appropriate texts for the students. • Running records used to be done in additional grade levels, but in talking to the principal, when reading coaches were dismissed, running records were not given except in the end of Kindergarten and during First Grade. • A first grade teacher felt that they are an accurate way to formatively assess how her students are reading and to ensure that she is providing reading instruction that will benefit all of her students at their various reading levels.

16. Pre/Post Tests Reading and Math K-4 Monthly Progress Monitoring / Drive Instruction • In some way, each grade level administers pre and post tests for math and reading. • Kindergarten: Teacher created math and reading assessments that align to the CCSS (Common Core State Standards) • The reading assessment is a hybrid of the Treasures’ Unit Tests and additional skills that the teachers want to monitor. • First Grade: Reading Series Unit Tests along with a CCSS aligned math assessment • Second Grade: Post Tests aligned to CCSS • Third Grade: Pre/Post aligned to CCSS • Fourth Grade: Pre/Post aligned to CCSS for math, Treasures’ Unit Tests for reading, Criterion-Referenced Tests for Science and Social Studies, and formative Weekly Reading Checks that are from Treasures as well. • The Pre/Post data is used to drive instructional choices, monitor progress, and to determine intervention groups for both reading and math instruction. • Some grade levels do not see the need for unit assessments in addition to the other tests that are given. • On the other hand, grade levels feel that it does help to make instructional decisions that will benefit all learners.

17. CTBS California Test of Basic Skills Grade 2 and also Based on Referral September / Can Vary Student Achievement / Gifted Identification • The CTBS is a diagnostic test that the school uses in order to determine student achievement in second grade, along with gifted identification. • The principal says that the school no longer uses the Terra Nova test, and that the CTBS has taken its place. • Based on teacher referral, students are given the CTBS in order to see if they qualify for gifted services. • Referrals can be made by teachers starting in Kindergarten • Second grade teachers feel that the information is not communicated well.

18. IOWA Test of Basic Skills 4th Grade September Student Achievement • The IOWA Basic Skills Test is administered to 4th grade students. • Results are compiled by an outside testing service and then provided to the school. • This year, the test was given in September, and results were received in December. As of right now, the principal was the only one who had seen the results. • She mentioned that she was not sure when the tests were submitted and that could be a reason for the long return time (Sept-Dec). • Student achievement is the purpose of these assessments.

19. OAA Ohio Achievement Assessment Grades 3-4 Practice: OCT Actual: MAY Student Achievement / Value Added • Starting in third grade, the OAA is administered to track student achievement. • In October, students are given the reading portion as a practice test. • The actual test is not given until May, and this is when all portions are administered. • Results from the practice test are used to determine areas of need, as well as to provide a pre test to use for student growth in the Spring. • After testing is complete, results are compiled by an outside testing service and returned to the school within a month typically. • Teachers are working to improve student achievement through intervention strategies and the implementation of a new reading series. • This is only type of Value-Added data currently provided. • The data is provided on the State Report Card

20. Conclusions This building is in the very early stages of effective data collection that is easily accessible and known to all teachers. Currently, there is not an electronic system in place to allow for easy access to assessment information or student data. Recommendation: To remedy this, for testing information, I feel that using a Google Doc or some other electronic editing tool would make the information easily accessible to everyone in the school. Not only could they see the information, but they could edit it when changes occur (for instance if second grade was going to begin using the STAR instead of Pre and Post Test to drive instruction). For student data, I would suggest that we utilize our Progress Book (electronic system) more effectively. We could input student scores of that system rather than in a “data folder” in their student file folder. The next year, teachers would be able to see all of their students’ information without having to pull each student’s file folder. From grade level to grade level, there is disconnect about the assessments that are given, and while teachers in the grade level understand the purpose of each assessment, they do not always feel that they are necessary. Recommendation: Have grade level meetings that are vertical to discuss the use of each assessment in their give grade levels. At these meetings they could also discuss the purpose of the assessment and if one could be use in its place for the same purpose. Currently, the State Diagnostic tests are not used for anything except to prove to the state that they were completed, so having a team of teachers brainstorm uses for those assessments would be beneficial. An article from Learning Point Associates said, “Working in a team, individuals can discover new ideas and views by collaborating with their teammates—discoveries they would never have made on their own” (2004).

21. Conclusions Tests that are found to be most effective by teachers are the STAR series and Pre and Post tests. The STAR is an efficient tool for monitoring progress, and Pre and Post tests help to drive instructional decisions and organize intervention groupings. The DIBELS is also an assessment that monitors progress, but because of the computerized nature of the STAR, it ranks higher among teachers than the DIBELS, especially with the Common Core State Standards coming down the pike and the computerized testing that will go along with them. Recommendation: Since the CCSS is going to be a sort of computerized test, I would encourage the use of the STAR since it is electronic. The STAR is currently being used as a progress monitoring tool,which is one of its designed uses (Renaissance Learning, 2011). In addition, I would explore using it in place of pre and post tests to drive instructional choices. Even if the questions are not directly related to the reading series (like the pre and post tests are), the skills that the students are lacking in could be addressed using those reading series components. In Delaware, they are using computer-adaptive testing for standardized and noticing that there a number of benefits including faster results, less time for students to take the tests, and better result data (Davis, 2012). The amount of testing in third grade is much less than the testing that takes place in fourth grade. On the state report card, grade four is performing higher than grade three when it comes to reading (Ohio Department of Education, 2011). Recommendation: Grade three should do reading checks and interventions based on students’ reading levels. Since literacy coaches were let go, Running Records became used much less infrequently. If teacher do not want to do running records for their students, then weekly reading checks (from the series) like those done in fourth grade could be beneficial for reading improvement. (Again, vertical teams could discuss items such as these if the school had them,)

22. Conclusions Teachers feel overwhelmed and that there is not enough teaching time. Recommendation: A first grade teacher suggested a substitute to come in and administer the running records or teach the class while the teacher does them in a quiet setting instead of at recess, which is when she currently gives them. When possible, teachers could use formative assessment instead of the typical paper and pencil pre and post tests. This “assessment” would then take place during teaching time instead of taking away from it.

23. References Davis, M. R. (2012, October 12).Adaptive testing evolves to assess common-core skills. Education Week. Retrieved from http://www.edweek.org/dd/articles/2012/10/17/01adaptive.h06.html Learning Point Associates. (2004). Guide to using data in school improvement efforts. Naperville, IL. Retrieved from http://www.learningpt.org/pdfs/datause/guidebook.pdf Ohio Department of Education. (2011). Building report card 2010-2011. Columbus, OH. Retrieved from http://www.ode.state.oh.us/ reportcardfiles/2010-2011/BUILD/019380.pdf Renaissance Learning. (2011). The foundation of the STAR assessments: The science of STAR. Wisconsin Rapids, WI. Retrieved from http:// doc.renlearn.com/KMNet/R003957507GG2170.pdf