1. Predictive modeling competitions making data science a sport Anthony Goldbloom CEO, Kaggle e-mail anthony.goldbloom@kaggle.com twitter @antgoldbloom Photo by mikebaird, www.flickr.com/photos/mikebaird

2. Global competitions Predicting HIV viral load Competition closes 77% 1½ weeks 70.8% State of the art 70%

3. Diverse experts solving diverse problems Grant Application Forecasting Stock Price Prediction HIV Research Chess Ratings Travel Time Prediction Edmund & Adrian London & USA Dr. Derek Gatherer UK Felipe Maia Uppsala University Ivan Russian Federation Dr. Christopher Hefele, New York Philipp Emanuel Widmann Heidelberg, DE Chih-Li Sung & Roy Tseng Penghu & Taipei Robert Warsaw Gzegorz Swiszcz Gera Cole Harris Texas Jure Zbontar Ljubljana Giuseppe Ragusa Rome Chris DuBois Portland Claudio Perlich USA Edmund & Adrian London & USA Jason Trigg Pennsylvania John Blatz Baltimore Rajstennaj Barrabas USA Chris Raimondi Batimore Jason Trigg Pennsylvania Uri Blass Tel-Aviv Lee Baker Las Cruces, NM Nan Zhou Pittsburgh Jeremy Howard Australia Thomas Mahony Canberra Glen Maher Canberra Emir Delic Australia

4. Motivation • Why host a competition? • Why compete? • How it works • Heritage Health Prize • Questions

5. “I keep saying the sexy job in the next ten years will be statisticians.” Hal Varian Google Chief Economist 2009

6. Crowdsourcing Mismatch between those with data andthose with the skills to analyse it

7. Countless possible approaches to any data prediction problem. Which to choose? 7

8. 18 year old beating his professors 8

9. Motivation • Why host a competition? • Why compete? • How it works • Heritage Health Prize • Questions

10. Tourism Forecasting Competition Forecast Error(MASE) Existing model Aug 9 2 weeks later 1 month later Competition End

11. Chess Ratings Competition Existing model (ELO) Error Rate(RMSE) Aug 4 1 month later 2 months later Today

12. Our User Base

13. Users apply different techniques • neural networks • logistic regression • support vector machine • decision trees • ensemble methods • adaBoost • Bayesian networks • genetic algorithms • random forest • Monte Carlo methods • principal component analysis • Kalman filter • evolutionary fuzzy modeling

14. Benchmarking

16. ~25% Successful grant applications NASA tried, now it’s our turn

17. Ideal for complex problems

18. ~25% Outcomes of a competition to predict the success of grant applications: Successful grant applications • Better identify likely successes to avoid wasting resources on hopeless applications • Identify and communicate the characteristics of a successful application to future applicants

19. Motivation • Why host a competition? • Why compete? • How it works • Heritage Health Prize • Questions

20. Why Participants Compete 2 1 More fun than Sudoku Clean, Real world data Professional Reputation & Experience 4 3 Interactions with experts in related fields Prizes

21. User base

22. User base

23. Motivation • Why host a competition? • Why compete? • How it works • Heritage Health Prize • Questions

24. 2 3 1 Upload Submit Evaluate & Exchange

25. Use the wizard to post a competition

26. Participants make their entries

27. Competitions are judged based on predictive accuracy

28. Competition Mechanics Competitions are judged on objective criteria

29. Motivation • Why host a competition? • Why compete? • How it works • Heritage Health Prize • Questions

30. An upcoming competition, powered by Kaggle • De-identified dataset containing medical records of 100,000 Americans • $3 million prize http://www.heritagehealthprize.com

31. & Unfilled Prescriptions & Hypertension & High Cholesterol Diabetes Probability of going to hospital in the next year

32. NetFlix Prize 2006 – 2009 $1 million prize 50,000 registrations 2011 $3 million prize Projected 100,000 registrations

33. Motivation • Why host a competition? • Why compete? • How it works • Heritage Health Prize • Questions

34. Chess Ratings – Elo vs. the Rest of the World IJCNN Social Network Challenge Tourism Forecasting (Part 2) Predict Grant Applications

35. Jeff Moser Jeremy Howard Nicholas Gruen Anthony Goldbloom

36. What could the world’s bestanalysts find in your data? e-mail anthony.goldbloom@kaggle.com phone +61438400053 Photo by gidzy, www.flickr.com/photos/gidzy