IDEAS: Institute for Developing Excellence in Arkansas Schools

1. IDEAS: Institute for Developing Excellence in Arkansas Schools Denise Airola, Sean Mulvenon, Charles Stegman, Calli Holaway-Johnson National Office for Research on Measurement and Evaluation Systems (NORMES) University of Arkansas

2. What Do We All Really Want? • Professional reports • Analysis for data-driven decisions • Intuitive graphics • Easy-to-use statistical programs • Professional development • Perfect data Actually, this isn’t true! What we really want is …

3. The “BUTTON” Click Here! This button will automatically generate what you want just by thinking about it!

4. The “Button” • Identifies the page Mary needs to review to pass the test • Identifies the critical areas of the curriculum where a teacher needs to improve so you can plan individualized and group professional development • Generates perfect graphs and tables to present to the school board This button doesn’t exist and never will!

5. So what do you do without the “Button”? Develop your EIQ!

6. What is EIQ? • Educational Intelligence Quotient (EIQ) is the ability to collect, analyze, report, and effectively use educational data. • Many levels of the educational system today have an overload of data with not enough EIQ.

7. Levels? • Educational data systems have levels • State • District • School • Classroom • Data and analysis are different at different levels

8. Within levels, we have more levels • State, District, and School • Analysts and Programmers • People responsible for the databases • Programmers • Statisticians • Consumers • Administrators • Teachers • Parents • Educational Stakeholders

9. Analysts and Programmers • Recommended Skills Sets • Computer system architecture • Engineering degrees • Statisticians • Degrees in statistics • Programming expertise • Language of your system architecture and statistics software

10. Analysts and Programmers • Expectations • Ability to build and structure your systems • Ability to develop and understand statistical modeling • Ability to study the system and make innovative improvements • Ability to “break” data down to lowest unit collected and rebuild up to state levels • Ability and knowledge to transfer and transform information into more useable forms • Develop web pages, reports, etc.

11. Consumers People who need to use the information, but don’t necessarily need to generate new information. These people are focused on deliverables that are practical and easily understood. • Administrators • Teachers • Parents • Educational Stakeholders • Legislators • Business leaders

12. Consumers • Expectations • Educators, both administrators and teachers, must have basic skills • Understanding of algebra • Ability to interpret graphs and charts • Basic statistics • Solid understanding of testing and measurement • Parents and other Educational Stakeholders • All information provided in open web pages must be clear, concise, and provide brief overviews of interpretations

13. Why is there a lack of EIQ? • The myth of the “Button” and the proliferation of “point-and-click” analysis programs • Lack of accountability and training for administrators and teachers • Professional development needs accountability • Colleges of education need oversight • No College Left Behind (NCLB)

14. Results of Low EIQ • Excess time and money spent on: • quality assessments that are underutilized • poor assessments that are overutilized • Increased frustration from teachers and administrators due to lack of EIQ Translation: A real loss in educational instructional time

15. How NORMES is Increasing EIQ • NORMES is staffed with educational statistics faculty, as well as graduate assistants and research associates with degrees in computer engineering, statistics, and curriculum. • This unique staffing contributes to development of all reporting systems.

16. How NORMES is Increasing EIQ • University Level • Greater expectations in teacher and administrator course work • Generating graduate students in both computer engineering and statistics to help school, district and state systems • State Databases • Helping to design and implement systems that facilitate independence

17. How NORMES is Increasing EIQ • Public and private data reports generated by determining: • the most effective use of the assessment data • which analyses are statistically appropriate • what information is going to be valuable to educators, parents, and the general public

18. How NORMES is Increasing EIQ • For local districts: • Program evaluations • Performance Growth Index (PGI) analysis

19. How NORMES is Increasing EIQ • For state data: • User-friendly reporting formats • High-quality public reports • Pre-configured reports available to administrators • Advanced analysis of school and district level data • Interactive, online training for administrators, curriculum coordinators, assessment analysts, and teachers

20. Public Data in Arkansas • School Performance • NOGGINS • GAPS • ALERT

21. http://normes.uark.edu

22. You can view school, district, and state level reports using tabs

23. From a district page you can select any school in the district and view that school in a new page

24. Districts have a financial page that will include 3 years of data side-by-side

25. Select a school or district once and navigate to related reports: • School Improvement Reports • Achievement trends • Achievement gaps (ALERT)

27. Student Achievement Snapshots—the next generation: • Toggle between grade levels • NRT and CRT available • Scroll down for tables with 3 years of subgroup results

28. Three years of data by subgroup are provided.

29. What if I want to see how other schools like mine are performing? Find comparison schools by school size, % FRLP, attendance rate, graduation rate, expenditure per student. Coming in 2007: % LEP, % SPED, etc,

30. Comparison schools can be sorted to view other characteristics. 2007 upgrade will allow you to drill to the public report for any school.

31. Dig into the details: • Do subgroups in the school demonstrate differences in achievement? • Do these differences constitute achievement gaps? • Are the differences large? • Are the differences meaningful? • Is the school closing achievement gaps?

32. ALERT: Analyzing Learning Equity Research Trends • Direct comparison of mean differences between the groups does not provide a “meaningful” context of what is an important difference • ALERT includes effect sizes, providing useful information to assist interpretation

33. What is the value in identifying meaningful gaps? • For effective improvement efforts based on documented needs, not perceptions, you must identify: • where achievement gaps exist (schools, classrooms, students), • how large they are, and • whether gaps have been persistent over time.

35. To answer the achievement gap question: Select the subgroup you want to view. Select the subject you want to view.

41. Select another school in the same district or another district in the state. Click on “Details” to link to the School Performance page for this school.

46. Private Data in Arkansas • ED.STATS Portal • Pre-configured reports • Data downloads • EGuide • Custom reports and analysis • Data downloads

47. Updates, information, and resources to support the use of the portal

48. School reports for school logins District reports for district logins References to use with reports

49. NORMES produces a variety of documents according to state specifications. These documents are pre-populated with student data and can be copied into different web applications.

50. Longitudinal Reports • ITBS • 3 years of NRT data in a .rft document to edit, copy and paste • Strand Report • 3 years of strand information to help in reporting strengths and weaknesses for groups