Exploring Randomness: Delusions and Opportunities

1. Exploring Randomness: Delusions and Opportunities Larry Weldon SFURA November 18, 2008

2. Recent Criticisms of Statistics? • Taleb, Nassim Nicholas (2007) Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets, Second Edition, Random House, New York. • Taleb, Nassim Nicholas (2007). The Black Swan: The Impact of the Highly ImprobableRandom House, New York. • www.stat.sfu.ca/~weldon

3. Problems with Statistics Education • Textbook-based and Technique-based • Textbook content is circa 1960 • Inference Logic was always controversial • Computers & Software Change Everything • Inertia to Curriculum Change

4. Examples of Modern Statistics Featuring • Use of graphics, smoothing and simulation for exploration and summary • Exploratory use of parametric models Claim • Surprising Results (even though simple methods) • Useful for real life

5. Example 1 - When is Success just Good Luck? An example from the world of Professional Sport

7. His team: Geelong

8. Geelong

9. Recent News Report “A crowd of 97,302 has witnessed Geelong break its 44-year premiership drought by crushing a hapless Port Adelaide by a record 119 points in Saturday's grand final at the MCG.” (2007 Season)

10. Sports League - FootballSuccess = Quality or Luck?

11. Are there better teams? • How much variation in the league points table would you expect IFevery team had the same chance of winning every game? i.e. every game is 50-50. • Try the experiment with 5 teams. H=Win T=Loss (ignore Ties for now)

12. 5 Team Coin Toss Experiment • Win=4, Tie=2, Loss=0 but we ignore ties. P(W)=1/2 • H is Win, T is L • 5 teams (1,2,3,4,5) so 10 games • T T H T T H H H H T Typical Expt lg.points

13. Implications? • “Equal” teams can produce unequal points • Some point-spread due to chance • How much?

14. Sports League - FootballSuccess = Quality or Luck?

15. Simulation of 25 league outcomes with “equal teams” 16 teams, 22 games, like AFL lg.points.hilo

16. Sports League - FootballSuccess = Quality or Luck?

17. Does it Matter? Avoiding foolish predictions Managing competitors (of any kind) Understanding the business of sport Appreciating the impact of uncontrolled variation in everyday life (Intuition often inadequate)

18. Postscript! 2008 Results

19. Example 2 - Order from Apparent Chaos An example from some personal data collection

20. Gasoline Consumption Each Fill - record kms and litres of fuel used Smooth ---> Seasonal Pattern …. Why?

21. Pattern Explainable? Air temperature? Rain on roads? Seasonal Traffic Pattern? Tire Pressure? Info Extraction Useful for Exploration of Cause Smoothing was key technology in info extraction

22. Aside: Is Smoothing Objective? 1 2 3 4 5 4 3 2 1 2 3 4 5 Data plotted ->>

23. Optimal Smoothing Parameter? • Depends on Purpose of Display • Choice Ultimately Subjective • Subjectivity is a necessary part of good data analysis Note the difference: objectivity vs honesty!

24. Summary of this Example • Surprising? Order from Chaos … • Principle - Smoothing and Averaging reveal patterns encouraging investigation of cause

25. Example 3 - Utility of Averages Arithmetic Mean – Related to Investment? 0 .5 1 4 AVG = 5.5/4= 1.38

26. Stock Market Investment • Risky Company - example in a known context • Return in 1 year for 1 share costing $10.00 25% of the time0.50 25% of the time1.00 25% of the time4.00 25% of the time Good Investment? i.e. Lose Money 50% of the time Only Profit 25% of the time “Risky” because high chance of loss

27. Independent Outcomes • What if you have the chance to put $1 into each of 100 such companies, where the companies are all in very different markets? • What sort of outcomes then? Use coin-tossing (by computer) to explore …. • HH,HT,TH,TT each with probability .25

28. Stock Market Investment • Risky Company - example in a known context • Return in 1 year for 1 share costing $10.00 25% of the time 0.50 25% of the time1.00 25% of the time4.00 25% of the time HH HT TH TT

29. Diversification: Unrelated Companies Choose 100 unrelated companies, each one risky like the proposed one. Outcome is still uncertain but look at typical outcomes …. Break Even One-Year Returns to a $100 investment Average profit is 38% - Actual profit usually +ve risky

30. Gamblers like Averages and Sums! • The sum of 100 independent investments in risky companies can be low risk (>0)! • Average > 0 implies Sum > 0 • Averages are more stable than the things averaged. • Square root law for variability of averages Variability reduced by factor n

31. Summary of Example 3 • Diversification of investments allows tolerance of risky investments • Simulation and graphics allow study of this phenomenon

32. Example 7 - Survival Assessment • Personal Data is always hard to get. • Need to make careful use of minimal data • Here is an example ….

33. Traffic Accidents • Accident-Free Survival Time- can you get it from …. • Have you been involved in an accident?How many months have you had your drivers license?

34. Accident Free Survival Time Probability that

35. Accident Next Month Can show that, for my 2002 class of 100 students, chance of accident next month was about 1%.

36. Summary of Example 7 • Very Simple Survey produced useful information about driving risk • Survival Analysis, based on empirical risk rates and smoothing, is a general way to summarize duration information

37. Example 8 -Lotteries:Expectation and Hope • Cash flow • Ticket proceeds in (100%) • Prize money out (50%) • Good causes (35%) • Administration and Sales (15%) 50 % $1.00 ticket worth 50 cents, on average Typical lottery P(jackpot) = .0000007

38. How small is .0000007? • Buy 10 $1 tickets every week for 60 years • Cost is $31,200. • Lifetime chance of winning jackpot is = …. • 1/5 of 1 percent! lotto

39. Summary • Surprising that lottery tickets provide so little hope! • Key technology is exploratory use of a probability model

40. Example 9 - Peer Review: Is it fair? Analysis via simulation - assumptions are: • Average referees accept 20% of average quality papers • Referees vary in accepting 10%-50% of average papers • Two referees accepting a paper -> publish. • Two referees disagreeing -> third ref • Two referees rejecting -> do not publish

41. 6 13 6 Ultimately published: 6 + .20*13 (approx) =9 papers out of 25 16 others just as good! peer

42. Peer Review Fair? • Does select some of the best papers but • Does not select most of the best papers • Similar property of school admission systems, competition review boards, etc.

43. Summary of Example 9 • Surprising that peer review is so dependent on chance • Key procedure is to use simulationto explore effect of randomness inthis context

44. Example 10 - Investment:Back-the-winner fallacy • Mutual Funds - a way of diversifying a small investment • Which mutual fund? • Look at past performance? • Experience from symmetric random walk …

45. Trends that do not persist rwalk

46. Implication from Random Walk …? • Stock market trends may not persist • Past might not be a good guide to future • Some fund managers better than others? • A small difference can result in a big difference over a long time …

47. A simulation experiment to determine the value of past performance data • Simulate good and bad managers • Pick the best ones based on 5 years data • Simulate a future 5-yrs for these select managers

48. How to describe good and bad fund managers? • Use TSX Index over past 50 years as a guide ---> annualized return is 10% • Use a random walk with a slight upward trend to model each manager. • Daily change positive with probability p

49. fund.walk.test

50. Simulation to test “Back the Winner” • 100 managers assigned various p parameters in .54 to .56 range • Simulate for 5 years • Pick the top-performing mangers (top 15%) • Use the same 100 p-parameters to simulate a new 5 year experience • Compare new outcome for “top” and “bottom” managers