Simulation Based Inference for Learning

1. “Our curriculum is needlessly complicated because we put the normal distribution, as an approximate sampling distribution for the mean, at the center of the curriculum, instead of putting the core logic of inference at the center.” George Cobb (USCOTS 2005) Simulation Based Inference for Learning Bret Rickman MS, M.Ed. Portland Community College

2. What to Expect • Why Implement SBI as Core for Intro Stats? • Implementing SBI: Method • Tactile • Applet Simulation (Randomize, Repeat, Reject) • Theory and Excel • Implementing SBI: Successes & Challenges • Reflections and Open Discussion

3. Why do I Implement SBI as Core for Intro Stats? • The GAISE standards advise it. • Recommendation #5: Use Tech to Explore Concepts and Analyze Data. • George Cobb Article: The Introductory Statistics Course: A Ptolemaic Curriculum. • My personal experience with fundamental change.

4. 2016 GAISE Report

5. George Cobb Article: The Introductory Statistics Course: A Ptolemaic Curriculum. “We need a new curriculum, centered not on the normal distribution, but on the logic of inference. • When Copernicus threw away the old notion that the earth was at the center of the universe, and replaced it with a system that put the sun at the center, his revolution brought to power a much simpler intellectual regime. • We need to throw away the old notion that the normal approximation to a sampling distribution belongs at the center of our curriculum, and create a new curriculum whose center is the core logic of inference. • What is that core logic? I (Cobb) like to think of it as three Rs: Randomize, Repeat, Reject. “

6. George Cobb Article: The Introductory Statistics Course: A Ptolemaic Curriculum. “Randomize data production; Repeat by simulation to see what's typical and what's not; Reject any model that puts your data in its tail. The three Rs of inference: Randomize, Repeat, Reject 1. Randomize data production To protect against bias To provide a basis for inference Random samples let you generalize to populations Random assignment supports conclusions about cause and effect 2. Repeat by simulation to see what's typical Randomized data production lets you re-randomize, over and over, to see which outcomes are typical, which are not. 3. Reject any model that puts your data in its tail “

7. SBI Implementation

8. SBI Implementation: Step 1 - The ‘Story’, Tactile and Applet Sims • 1 Example – ‘The Woman Who Could “Smell” Parkinsons’ • Joy & Les Milne • 2 Tactile model to build distribution and check simple probability • 3 Transition to Applet simulation • Randomize (produce the data) • Repeat (by simulation to see what’s typical) • Reject (any model that puts your data in its tail) Rossman / Chance Sim Applets Washington Post Amazing Woman Story

9. SBI Implementation: Step 2 – Math Basis & Excel • 1 Present the “math” (as ‘how we did things in the old days’) with straightforward example & connection to the simulation • 2 Excel example & usage with immediate student involvement. Include connection to simulation.

10. SBI Implementation: Engagement Activity # 1 • Can Babies Tell Right from Wrong? The Yale Study

11. SBI Implementation: Engagement Activity # 1 Lesson Steps • Observational Units, Variable(s), Type(s), Research Question. • Tactile Model (coins & dot plot) • Simulation (Rossman / Chance Applet) • Theoretical (especially Excel or Google Sheets)

12. Setup Items Probability of event occurring Sample Size (n) Number of Trials (repetitions) Start the Simulation with this button! Analysis Section Will display Mean & Std Dev of the simulation run! Dot Plot Display of Simulation Run Theoretical Calculations

13. SBI Implementation: Sim Applet Results

14. Results • Pretty unlikely to obtain 14 or more heads in 16 tosses of a fair coin, so … • Pretty strong evidence that pre-verbal infants do have a genuine preference for helper toy and were not just choosing at random

15. SBI Implementation: Theoretical (BC)

16. SBI Implementation: Engagement Activity # 2 – Class Generated Data Facial prototyping • Who is on the left – Bob or Tim? Socrative Poll

17. SBI Implementation: Facial Prototyping • Does our sample result provide convincing evidence that people have a genuine tendency to assign the name Tim to the face on the left? • How can we use simulation to investigate this question? • What conclusion would you draw? • Explain reasoning process behind conclusion

18. SBI Implementation: Facial Prototyping • Observational Units, Variable(s), Type(s), Research Question • Tactile, Simulation, Theoretical

19. SBI Implementation: Engagement Activity #3 – Regression Inference 1969-70 Draft Lottery

20. SBI Implementation: 1969-70 Military Draft • How can we use simulation to investigate this question? • What conclusion would you draw? • Explain reasoning process behind conclusion

21. SBI Implementation: Engagement Activity #3 – Regression Inference 1969-70 Draft Lottery

22. SBI Implementation Successes & Challenges

23. SBI Implementation Successes • 1 Rossman/Chance Simulators Free & Easy to Use • 2 Much more straightforward to implement in our (PCC) Stats II class • 3 Assessments can focus on situational items – not “how to use the tool” • 4 Students were comfortable using the simulators

24. SBI Implementation Challenges • 1 SBI is great for conceptual understanding, but doesn’t include the historical math basis. • 2 SBI doesn’t as readily relate / apply to the teaching of our traditional descriptive statistics and probability rules (yet!). • 3 To a certain extent, requires a computer lab – until the switch to ‘wireless device usage’ approval & implementation. • Side Note: Student feedback seemed to focus on the benefits of learning / using Excel. But students were using the simulators during exams!

25. Reflections • Fun curriculum to teach using SBI with inference as the core of the curriculum – great for conceptual student understanding. • Not enough of my classroom assessment data to evaluate effectiveness of Inference core. Is it possible that the data may not even indicate true effectiveness? • Students are more interested and engaged with SBI curriculum than ‘traditional’ method.

26. Audience Comments & Questions • What are your thoughts on using Inference as the core element in Stats I & II curricula? • How will ‘switching’ to SBI & Inference core affect other academic departments? • What about technology use – more specifically mobile / wireless devices – in the statistics classroom? • Your thoughts and comments.

27. Acknowledgements and Helpful Links & Sources Special thanks to Dr. Allan Rossman(Cal Poly – San Luis Obispo) ! 2016 GAISE Report link: http://www.amstat.org/asa/files/pdfs/GAISE/GaiseCollege_Full.pdf George Cobb article: The Introductory Statistics Course: A Ptolemaic Rossman/ Chance Applets: Rossman / Chance Simulation Applets Website