Data-Driven Decision Making …and assessment …And evaluation

1. Data-Driven Decision Making…and assessment…And evaluation Week 11, Thursday

2. Today’s Flocks • Flock #1: Katie F., Jacob, and Jennifer • Flock #2: Andi, Kim, and Josh • Flock #3: Kristen, Natalie, and Brittany • Flock #4: Katie B, Jordan, and Tom • Flock #5: Katelin, Scott, and Grace

3. Data-Driven Decision Making (DDDM)Activities 1

4. Get a SMART “clicker” • Baker, Tom • Berman, Scott • Bollinger, Katie • Braun, Katelin • Caffarini, Andi • Cotton, Grace • Funk, Katie • Gerbsch, Jordan • Graves, Natalie • Haubold, Kristen • Koressel, Jacob • Lynch, Jennifer • Seeber, Kimberly • Van Hook, Brittany • Yoder, Josh Each of the remotes has a number on the back of it. Find the one that corresponds to your name

5. Activity 1: DDDM “quiz” • This is not a quiz – this resembles a quiz (tool, format, etc) • You can think, use readings, use prior knowledge, even guess • I want to do three things with this • Start a discussion about DDDM • Use a technology-based assessment and evaluation tool • Look at the results of our “quiz” from a DDDM perspective

6. SMART Response “Quiz”

7. Discussion of answers to “Quiz”

8. Exploration of SMART assessment tools and results screens

9. Flock Discussion

10. Elevator Pitch • If you had to describe in 30 seconds (+/- 5) what DDDM was, and its benefits, what would you say? Flock up, and as a flock, create this "elevator pitch." You will then have one member of your flock who will share this with us. • Take 3-5 minutes, then share it!

11. Data-Driven Decision Making: Strengths and Challenges

12. Let’s Recap • What is Data-Driven Decision Making? • What does it entail? • Why does it matter? • What could it look like for: • A teacher? • A principal? • A technology specialist/coordinator? • A superintendent or district/school corp leader?

13. What's right about it? • What do you think the strengths are of using a DDDM approach for: • Teachers? • Principals? • Technology specialists • School district/corp leaders? • Political/civic leaders? • Sources of income/funding (“investment” in education) • What are the other benefits/strengths

14. LC Sandbox - demonstration

15. In your flocks… • You will be visiting the Learning Connection • What is the Learning Connection • http://lcsandbox.doe.in.gov • You will need the MS Word document I have emailed to you with LC sandbox usernames/passwords • If you are a W200 ULA, you should already have this information • Get into your flocks, and you will take 5 minutes to look for the DDDM tools available to each of the usernames you were assigned (next slide) • Your flock will share this with the class as a whole

16. In your flocks… • Flock #1: Jeremy Bening & Brenda Cocran • Flock #2: Dr. Koslan Dewitt & Mrs. Whobrey • Flock #3: Bob Holms & Mr. Dorcent • Flock #4: Ms. Kawasaki & Blake Cirioni • Flock #5: Mrs. Whobrey & Brenda Cocran

17. EVALUATING DDDM

18. What’s not so right about it? • First, what do you think the #1 complaint about DDDM is? • What is it? • What is the argument? • Is it justified • What are the weaknesses of a DDDM approach (We are not discussing the merit of testing, but DDDM)?

19. Where the problem lies • “Data-Driven Decision Making” • There is NOTHING wrong and everything right about making informed (teaching) decisions based upon evidence (and not feeling) of whether or not a student has learned something • There is an issue here, but not worth lighting your hair on fire: • The issue here is bad decision making • It is easy to take a good thing and make bad decisions • If there is a problem with DDDM at this level, it is the same problem any tool has – you have to know how to use it! • The real issues are not with this phrase – let’s keep looking

20. Where the problem lies • “Data-Driven Decision Making” • On to the word “driven” • I cannot see an argument about letting data drive your decisions • This can protect teachers, and give a justifiable foundation to your assessments • Allowing data to drive the choices you make as a professional gives you direction and a place to turn as you evaluate your progress

21. Where the problem lies • “Data-Driven Decision Making” • You probably guessed this is where I was headed! • The issue really is not with “data” itself, but with what: • Constitutes data and evidence • What TPTB constitute as data and evidence • What is “empiricism?” • What is “rationalism?” • What is “evidence?” • What constitutes “data” that is acceptable in our world as evidence?

22. Where the problem lies • The “qualitative”/”quantitative” debate in 1 minute • What are “metrics” • There is a bias towards that which can be justified through numerical and statistically-supportable evidence as being the strongest evidence for learning and student growth • NCLB and numerical benchmarks for AYP (Adequate Yearly Progress) • The “What Works” database published by the US Dept of Education, and what constitutes evidence-based research of learning • Research biases in education • There is some merit in this, but it comes from a specific view of knowledge and evidence: post-positivism

23. What you need to know • There are other ways to provide evidence and documentation of what works locally (localized knowledge, not generalizable knowledge) • Narrative-driven case studies • Interviews and theme analyses of documents and what people have to say • Observations • More… • The real issue concerning DDDM – the one that needs to be considered by every teacher – is what constitutes real evidence. • Are tests evidence – yes! But not the full picture • What does the full picture look like

24. The last word… • The issue is not with the usefulness of numerical and evidence supported through statistical processes, but that these present a partial picture that needs to be supplemented • Never write off the value of test scores, student achievement data, etc. It was very difficult for me to access before NCLB, but will be much easier for you (and encouraged). I wish I had access to it! • But what this offers you is only part of the story, even if it is the story by law or by public/political perception.

25. The last word… • Because of NCLB, you will have a treasure-trove of student demographic and achievement data at your fingertips through new data networks, such as those provided through EMIS/SIS networks • Technology – such as the SMART response system, or online test and survey tools, or such – can provide real-time or near-real time information and results • What are some of these tools? • What would this look like in your units

26. Reverse Quiz/survey

27. What you will do • For this activity, you will focus on your unit, and take a concept, goal, or objective (desired), and you will create a quiz. • This quiz will give you (near) real-time data, feedback, and tracking information on student progress. • I understand that many of you have units that are heavy on skills, but it is still possible to look at concepts, underpinnings, and key procedures, skills, ideas, etc. • You will do this individually and post a link to the quiz on your blog • If possible, have two people take your quiz, then discuss what the taking of the quiz told you about the quality of the quiz (paragraph at most)

28. What’s Due, When

29. Duedates • You should be working on producing the quadrillion different parts of your units • Please ask me (email, office hours – Tues and Thurs) or your peers for feedback. • You really, really should have draft unit and lesson plans, so you know what you have to produce • My office hours are: • Tuesdays: 3:30-4:30pm in the TTL • Tuesdays: 4:30pm-classtime in my cubicle • Thursdays: 4pm-classtime in my cubicle • By appt Mon-Thurs (Fridays only if necessary – I cannot always come to campus on Fridays)