Statistics in medicine: a risky business?

1. Margaret MacDougall(Email: Margaret.MacDougall@ed.ac.uk)PHS lunchtime seminar, University of Edinburgh, 20 November 2008 Statistics in medicine: a risky business?

2. A typical snapshot from a statistically exercised clinician’s brain(simplified version)

3. confidence intervals 4 p-values meta-analysis relative risks absolute risk reduction odds ratios random effects model multivariate analysis the immediate response … statisticalpower

4. 5

5. Background: 3 fundamental problems encountered by statisticians in the training of undergraduate medical students and by qualified doctors • poor retention of key statistical concepts such as confidence interval, absolute risk, relative risk, odds ratio, p-value, statistical power and significance level; • a lack of efficacy in communicating notions such as relative and absolute risk and more generally, the findings of systematic reviews to patients; • a lack of professional training in the use the statistical findings presented in medical journals to make informed decisions on the optimal choice for the patient.

6. Project aim With a view to confronting the above problems, the MSOR-funded project Statistics in medicine: a risky business was set up with the aim of addressing the following questions: MSOR: Maths, Stats and Operational Research Subject Centre of the Higher Education Academy (HEA) - one of 25 subject centres of the HEA defined by academic disciplines

7. Project aim (continued) • how can we effectively steer undergraduate medical students through the conceptual maze of statistical concepts on the road to evaluating risk? • how can we convince undergraduate medical students that the use and application of statistics is an integral part of good medical practice? • can we empower tomorrow’s doctors to identify and implement the correct statistical tools for evaluating patient risk so as to make the best choices for their patients?

8. Learning environment • Risk CAL created within EROS (Edinburgh Reusable Object Sequencer): an online sequencing engine owned by the University of Edinburgh • Delivery of CAL modules to individual cohorts to be controlled by means of web-based Virtual Learning Environment known as EEMeC (Edinburgh Electronic Medical Curriculum) CAL: sequentially arranged collection of Computer Assisted Learning objects

9. Recent innovations aimed at enhancing effective student engagement and autonomy • Student-student dialogues to highlight common ambiguities concerning the best choice of statistical methodology • Use of a variety of informed strategies for rendering learning materials inclusive for dyslexic learners, two of which illustrated here: - Integration of optional online ‘story books’ with CAL materials accessed by all students - Use of ‘Want to check the technical details?’ boxes

10. Student-student dialogues to highlight common ambiguities concerning the best choice of statistical methodology Start page from a student-student dialogue

11. Integration of optional online ‘story books’ with CAL materials accessed by all students option to continue option to access book Inclusion of a story book to suit different learning styles

12. Story books …. • Promote effective engagement through deep learning • Empower the learn to choose their preferred learning style • Hence promote self-directed learning

13. Use of ‘Want to check the technical details?’ boxes ‘Want to check the technical details?’ box to support optional learning

14. Risk CAL features uses a variety of types of learning objects to promote effective engagement

15. Recent related publications • MacDougall M (2008) Dyscalculia, dyslexia, and medical students’ needs for learning and using statistics. Medical Education Online (in press) • MacDougall M (2008) Statistics in medicine: a risky business? MSOR Connections (November 2008) • MacDougall M (2008) Ten tips for promoting autonomous learning and effective engagement in the teaching of statistics to undergraduate medical students involved in short-term research projects. Journal of Applied Quantitative Methods, 3(3): 223-240

16. Future plans for curriculum development and research activities • Focus groups with clinical staff with a view to integration with existing curriculum (early 2009)  publication of findings (please let me know if you are interested) • Extension of available examples and exercises and modularization of CAL materials • Research activities: student evaluation  publication of findings  further refinement of CAL materials • Parallel activities to be carried out with sister CALs on confidence intervals and related measures – The distribution of the sample mean and Testing for a difference between two populations

17. Acknowledgements • MSOR Network: providing the necessary funding to support the development of the risk CAL • University of Edinburgh Principal’s eLearning Fund: modularization of the CAL to facilitate its use in teaching • Jackie Aim (eLearning Developer): dedication to the eLearning development work and her constant willingness to embark on new adventures • Lesley Anne Lancastle and Stephanie Dunkeld Wills (undergraduate medical students), John M MacPherson and Michael Ross: very able volunteer actors who have helped to bring the CAL to life.