Integrating A Problem-Based Learning Approach Into Large Sections of Graduate-Level Introductory Biostatistics Courses

1. Integrating A Problem-Based Learning Approach Into Large Sections of Graduate-Level Introductory Biostatistics Courses Patrick D. Kilgo Emory University Department of Biostatistics and Bioinformatics Southern Regional Council on Statistics June 6th, 2011

2. Core Course Setting • Graduate level biostatistics courses with associated lab components • All incoming Master’s degree candidates in public health are required to take BIOS 500: • Descriptive statistics • Probability • Common hypothesis tests • 5 large sections – approximately 85 students per section • Two classes per week – 80 minutes per class • Follow-up regression course (BIOS 501) is optional • Linear regression / ANOVA • Logistic regression • Survival analysis

3. What’s The Problem? RETENTION • It is common for our students to have forgotten almost everything in the intervening month between Fall and Spring semesters • Thesis season: By their second year, the average student has: • Forgotten most of the statistical concepts they once “knew” • Has forgotten how to apply concepts and statistical tests and also the programming necessary to accomplish their analysis • Resorted to roaming the halls of the third floor, beckoning any statistical-looking person for help

4. Problem-Based Learning (Duch, 2001) • We learn and retain when solving a problem ourselves • “Complex, real world problems are used to motivate students to identify and research the concepts and principles they need to know to work through those problems” • Small learning teams are used to collectively acquire, communicate and integrate information • “Instructor is no longer the sage on the stage but rather is the guide on the side.”

5. Problem-Based Learning Objectives (Duch, 2001) • Think critically and be able to analyze and solve real-world problems • Find, evaluate and use appropriate learning resources • Work cooperatively in teams and small groups • Demonstrate versatile and effective communication skills, both verbal and written • Use content knowledge and skills acquired at the university to become continual learners.

6. Previous PBL Biostatistics Courses • Carolyn Boyle, Mississippi State, Journal of Statistics Education v.7, n.1 (1999) • Applied in an animal science setting • 18 veterinarian students • 8 cases over two semesters • The only published account of PBL in biostatistics

7. Goals – Excellence In These Areas … • Generating descriptive statistics • Choosing the appropriate analysis approach when faced with a research problem • Interpreting findings from research studies • Writing reports and communicating results of research findings following a statistical analysis • Thinking through analytical problems and subsequently designing studies • Working in groups to solve research problems • Beginning the statistical thinking/planning for your Master’s thesis • Discussing statistical analysis with other faculty, students and employers

8. Extra Resources Required for PBL • Departmental Support - $$$$$$$$ • 3 additional experienced “co-instructors” • Though some disagree, I still believe that this task is beyond the capabilities of the average TA • 3 additional classrooms • Patience and flexibility on the part of the lab instructors • A ton of my time

9. General Framework of My PBL Class • No more tests or homework • No required textbook: I asked them to find any statistical text for reference • 4 “cases” (problems) over the course of the semester • Mondays: Lecture (Taught by Kilgo) • Wednesdays: 4 PBL “breakout” sections of size 20 where cases are worked on in groups of 4-5. (Taught by Kilgo and 3 co-instructors) • Teach them deeper, not wider

10. Deeper, Not Wider • Half as many lectures = must be efficient • Before I presented a topic I asked myself three questions: • How likely are the students to encounter this topic in practice? • How likely is the average student to remember this topic in three weeks? • Will they be taught this topic in their introductory epidemiology course? • Sample of topics omitted: • Many probability axioms and concepts (~1 lecture) • Bayes rule (~1 lecture) • Binomial and Poisson distributions (~2-3 lectures) • Nonparametric tests (~1-2 lectures) • Several statistical tests – McNemar’s Test, ANOVA (~2 lectures)

11. First Implementation • Fall 2009, a class of 72 first-year, first semester Global Health students • Non-majors • PBL co-instructors: • Lisa Elon – fellow faculty-level colleague • Laura Ward – staff senior biostatistician • Jeff Switchenko – 5th year doctoral student • Open-ended, real-life, interesting problems in public health and medicine • Individual deliverables, even though group work was encouraged • Data analysis report with emphasis on methods, results, conclusions, limitations

12. Cases • Case 1 – Designing a study to determine whether data collected from Automatic Crash Notifiers in cars can be used to determine the need for Level I trauma care • No data in this case • Thought experiment • Case 2 – Were players accused in the Mitchell Report of taking steroids better offensive performers? • Students had to make a descriptive case one way or the other. • Outliers, multiple observations per player, skew, etc.

13. Cases • Case 3 – Validation of an experimental testing device designed to diagnose pre-Alzheimer’s disease • Data management, t-tests, assumption violations, experimental design issues • Real-life problem – collaboration between Emory and GA Tech • Case 4 – Smallpox Vaccine Trial • Chance to compare modern methods to Jenner’s method • Students had to read Jenner’s original paper • Chi-square tests, odds ratios, Interactions (non-homogeneity)

14. Final Project • Students proposed a personalized final project in the middle of the semester • Could be anything from a research interest to a personal interest: • What is the effect of maternal iron supplements on neo-natal iron levels? • Do women think mustaches are more sexy when they are ovulating? • Students asked for specific variables, guessed at their distribution and hypothesized about group differences. • Instructors generated datasets for them so that they were studying something that is interesting to them in a context they are familiar with.

15. First Semester PBL Evaluation …How comfortable are you with the following …?

16. Feedback From Students • If I could go back in time to the beginning of the semester with a choice of class formats I would … • Choose the lecture-only format (4/57) • Choose the problem-based format (48/57) • Be indifferent towards the format (5/57)

17. First Semester Growing Pains • Timing of cases / lectures / labs • Should have taken a TA when one was offered • Workload distribution – most of the assignments came due later in the semester • Different approaches from different co-instructors

18. First Semester Pleasant Surprises • Students liked SAS • Very positive course feedback • Students having an easier time working with faculty on projects • Many requests for a third course offering • Was as much a class in research writing and organization as it was biostatistics – their scientific writing greatly improved over the semester

19. Second Implementation – BIOS 501 Spring 2010 • Only three cases – no final project • Case 1 – Predicting traffic deaths using 1964 NHTSA-type data • Linear regression, transformation, skew, outliers, missing data, validation, confounding. • Case 2 – The evaluation of off-pump CABG compared to on-pump CABG with respect to major adverse outcomes • Logistic regression, lots of covariates, confounding, fitting of associative models, graphics, independent risk factor identification, interactions • Case 3a - Survival Analysis –The Role of Race and Race Mismatch in Determining Survival in Pediatric Heart Transplant Patients • Case 3b – The Effect of ICU LOS on Long-Term Survival • KM curves, Cox proportional hazards regression, confounding, etc.

20. Conclusions - Feedback From Students • Very positive in general • Complaints include: • Workload distribution • Time-consuming • Learning material/working on cases concurrently • “I got an 800 on the math GRE and I’m struggling in your class … I felt like I would have done better in the traditional section” • BIOS 500 • Fall 2009: 4.6/5.0 • Fall 2010 4.2/5.0 • BIOS 501 • Spring 2010 4.7/5.0 • Spring 20114.7/5.0 Likert Scale Question: I learned a lot in this course …

21. THANK YOU!