Assessing the Effect of Visualizations on Bayesian Reasoning through Crowdsourcing

1. Assessing the Effect of Visualizationson Bayesian Reasoning through Crowdsourcing LuanaMicallef Pierre Dragicevic Jean-Daniel Fekete

2. The probability that a woman at age 40 has breast cancer is 1%. The probability that the disease is detected by a mammography is 80%. The probability that the test misdetects the disease although the patient does not have it is 9.6%. If a woman at age 40 is tested as positive, what is the probability that she indeed has breast cancer? 0% - 30% 30% - 60% 60% - 100%

3. ATTENTION

4. The probability that a woman at age 40 has breast cancer is 1%. The probability that the disease is detected by a mammography is 80%. The probability that the test misdetects the disease although the patient does not have it is 9.6%. If a woman at age 40 is tested as positive, what is the probability that she indeed has breast cancer? 0% - 30% 30% - 60% 60% - 100%

5. The probability that a woman at age 40 has breast cancer is 1%. The probability that the disease is detected by a mammography is 80%. The probability that the test misdetects the disease although the patient does not have it is 9.6%. If a woman at age 40 is tested as positive, what is the probability that she indeed has breast cancer? 0% - 30% 30% - 60% 60% - 100%

6. P ( Cancer | Positive Mammography ) = 7.8%

8. 95 doctors out of 100 said the answer is between70% to 80%

9. Why the correct answer is so low

10. Bayes’ Theorem P ( cancer | +ve mammography ) = P ( +ve mammography | cancer) P (+ve mammography | cancer) + P (+ve mammography | cancer)

11. The probability that a woman at age 40 has breast cancer is 1%. women with cancer women without cancer

12. If a woman at age 40 is tested as positive, what is the probability that she indeed has breast cancer? The probability that the disease is detected by a mammography is 80%. The probability that the test misdetects the disease although the patient does not have it is 9.6%. 7.8% women with cancer women without cancer

13. Can such visualizations facilitate Bayesian reasoning

14. Proposed Visualizations

15. contingency table signal detection curves trees Bayesian boxes bar-grain boxes

16. + Euler diagram frequency grid

17. Euler diagram + glyphs

18. Previous Studies Mainly in Psychology Claim that Bayesian problem representation impacts comprehension

19. but … InconsistentfindingsMost effective Bayesian problem representation? UNCLEAR Inconsistent and sometimes inappropriate diagram designs Diagrams do not match textual information

20. (Slomanet al., 2003)

21. Area-Proportional Not Area-Proportional

22. and the subjects … Specific background usually highly-focused university students Specific age group Sometimes, specific department carried out as part of their course

23. so … • cannotgeneralize their findings toa more diverse population of laypeople

24. Our Work Assessing the Effect of Visualizations on Bayesian Reasoning through Crowdsourcing

26. to identify… • - the most effective visualization for the crowd • - whether hybrid visualizations are helpful • - the link between the visualizations and different spatial and numeracy abilities

28. but… • how appropriate is

29. Amazon MTurk Used and evaluated for research and InfoVis Demographics of workers are well-understood Captures aspects of real-world problem solving better - alarge diverse population with different backgrounds, education, occupations, age, gender - workers carry out tasks rapidly but accurately to improve their rating - reduces experimental biases, as demand characteristics

30. http://www.eulerdiagrams.org/eulerGlyphs

38. Experiment

39. 168 workers with MTurk approval rate ≥ 95%

40. Demographics

41. 25 min $1

42. 3 Bayesian problems classics in Psychology in natural frequencies format

50. followed by • objective and subjective numeracy tests • paper folding spatial abilities test • brief questionnaire