The Collective Intelligence of Diverse Agents: Micro Foundations of Uncertainty

1. The Collective Intelligence of Diverse Agents:Micro Foundations of Uncertainty Lu Hong & Scott E Page SFI 6 6 - 07

2. Outline • Aside on Theoretical Foundations • The Wisdom of Crowds • Standard Models • Interpretation Framework • Mathematical Results • Diversity, Democracy, and Markets SFI 6 6 - 07

3. Methodological Tradeoff Logical Informal |____________________________________________| Mathematical Appreciative Brittle Flexible |____________________________________________| Mathematical Appreciative SFI 6 6 - 07

4. Agent Based Models Logical Informal |_____ABM___________________________________| Mathematical Appreciative Brittle Flexible |____________________________________ABM____| Mathematical Appreciative SFI 6 6 - 07

5. Model Benchmarking Real World Math ABM SFI 6 6 - 07

6. Model Validation Real World Math ABM SFI 6 6 - 07

7. Methodological Translation Real World Math ABM SFI 6 6 - 07

8. Models of Collective Wisdom SFI 6 6 - 07

9. Von Hayek ...it is largely because civilization enables us constantly to profit from knowledge which we individually do not possess and because each individual's use of his particular knowledge may serve to assist others unknown to him in achieving their ends that men as members of civilized society can pursue their individual ends so much more successfully than they could alone. SFI 6 6 - 07

10. Aristotle “For each individual among the many has a share of excellence and practical wisdom, and when they meet together, just as they become in a manner one man, who has many feet, and hands, and senses, so too with regard to their character and thought.’’ SFI 6 6 - 07

11. Aristotle “Hence, the many are better judges than a single man of music and poetry, for some understand one part and some another, and among them they understand the whole.” Politics book 3 chapter 11 SFI 6 6 - 07

12. SFI 6 6 - 07

13. The Wisdom of Crowds:Galton’s Steer 1906 Fat Stock and Poultry Exhibition, 787 people guessed the weight of a steer. Their average guess: 1,197 lbs. SFI 6 6 - 07

14. The Wisdom of Crowds:Galton’s Steer 1906 Fat Stock and Poultry Exhibition, 787 people guessed the weight of a steer. Their average guess: 1,197 lbs. Actual Weight: 1,198 lbs. SFI 6 6 - 07

15. Who Wants to Be a Millionaire SFI 6 6 - 07

16. Experts or Crowds? Experts: Correct 2/3 of the time Audience: Correct over 90% of the time SFI 6 6 - 07

17. Three Mathematical Models SFI 6 6 - 07

18. Model 1: Known Information Best Selling Cereal of All Time • Corn Flakes • Rice Krispies • Cheerios • Frosted Flakes SFI 6 6 - 07

19. Answer: c) Cheerios SFI 6 6 - 07

20. How Errors Cancel Consider a crowd of 100 people 10% Know the correct answer 10% Narrowed down to two answers 36% Narrowed down to three answers 44% No clue SFI 6 6 - 07

21. # Votes for Correct Answer 10: 10% Know the correct answer 5: 10% Narrowed down to two answers: 12: 36% Narrowed down to three answers: 11: 44% No clue 38 TOTAL SFI 6 6 - 07

22. Why The Crowd’s Correct The correct answer gets 38 votes. Assume that the other 62 votes are spread across the other three. Each of those three receives around 20 votes. SFI 6 6 - 07

23. N.B. The crowd can be correct with very high probability even if no one in the crowd knows the correct answer. SFI 6 6 - 07

24. The Math 40: Cheerios or Corn Flakes 30: Cheerios or Frosted Flakes 30: Cheerios or Rice Krispies Cheerios gets 50 votes! SFI 6 6 - 07

25. Model 2: Correlated Signal Suppose that we’re trying to discover whether or not a truck full of sour cream has gone bad due to a faulty refrigerator. SFI 6 6 - 07

26. Model 2: Correlated Signal Suppose that we’re trying to discover whether or not a truck full of sour cream has gone bad due to a faulty refrigerator. True State: G (good) or B (bad) SFI 6 6 - 07

27. Signals Suppose that we can test pints of sour cream and get signals (g and b) and that with probability 3/4, these signals are correct. SFI 6 6 - 07

28. Signals Suppose that we can test pints of sour cream and get signals (g and b) and that with probability 3/4, these signals are correct. If the sour cream is bad, 3/4 of the time we’ll get the signal b. SFI 6 6 - 07

29. Three People True State: B Correct Outcomes P1P2P3Probability b b b (3/4)(3/4)(3/4) = 27/64 b b g (3/4)(3/4)(1/4) = 9/64 b g b (3/4)(1/4)(3/4) = 9/64 g b b (1/4)(3/4)(3/4) = 9/64 Total = 54/64 SFI 6 6 - 07

30. Three People True State: B Incorrect Outcomes P1P2P3Probability g g g (1/4)(1/4)(1/4) = 1/64 b g g (3/4)(1/4)(1/4) = 3/64 g b g (1/4)(3/4)(1/4) = 3/64 g g b (1/4)(1/4)(3/4) = 3/64 Total = 10/64 SFI 6 6 - 07

31. General Model With probability p > 0.5, people get the correct signal. Therefore, if N people get signals, pN get the correct signal. As N gets large, the expected probability of a correct vote goes to one. SFI 6 6 - 07

32. Model 3: Averaging of Noise Suppose that we want to predict the luminosity of a star. Each of 100 people stationed around the globe takes out a light meter and takes a reading. SFI 6 6 - 07

33. Model 3: Averaging of Noise Suppose that we want to predict the luminosity of a star. Each of 100 people stationed around the globe takes out a light meter and takes a reading. Call the reading for person k, r(k) SFI 6 6 - 07

34. Noise/Interference The signal that a person gets equals the true luminosity, L, plus or minus an error term, due to ambient light, humidity or who knows what. r(k) = L + e(k) e(k) is the error term SFI 6 6 - 07

35. Noises Off The average of the signals equals L plus the average of the error terms: [r(1) + r(2) + r(N)]/N = L + [e(1) + e(2) +..e(N)]/N If the error terms are, on average, zero, then they all cancel, and the prediction is accurate. SFI 6 6 - 07

36. Important Questions Why should we assume that these error terms are, on average, equal to zero? Why should we assume the signals are independent? Is this how an ABM would capture collective wisdom? SFI 6 6 - 07

37. Markets and Democracy Model 1: Some people know the answer Model 2: People get signals that are probabilistically correct Model 3: People see the true state plus an error SFI 6 6 - 07

38. NONE do. SFI 6 6 - 07

39. Signal noise Outcome Signal SFI 6 6 - 07

40. Generated Signals • True state of the world: x • Signal: s • Joint probability distribution: f(s,x) • Conditional probability distribution: f(s|x) SFI 6 6 - 07

41. Generated Signals True state generates something that is correlated with the state’s value - luminosity of stars S = L+e - quality of a product {good, bad} s = True quality with prob p SFI 6 6 - 07

42. A Generated Signal A chef of unknown quality produces batches of risotto. Each batch is a signal of the chef’s quality. Batches temporally separate enough to be considered independent revelations of quality. SFI 6 6 - 07

43. Predictive Model: Lu Hong model Attributes Prediction SFI 6 6 - 07

44. Interpretations Reality consists of many variables or attributes. People cannot include them all. Therefore, we consider only some attributes or lump things together into categories. (Reed 1972, Rosch 1978) SFI 6 6 - 07

45. “Lump to Live” If we did not lump various experiences, situations, and events into categories, we could not draw inferences, make generalities, or construct mental models. SFI 6 6 - 07

46. Predictive Models Edwards is a liberal; therefore he’ll raise taxes. The stock’s price earnings ratio is high; therefore, the stock is a bad investment. SFI 6 6 - 07

47. How Do We Predict? We parse the world into categories and make predictions based on those interpretations. SFI 6 6 - 07

48. Interpretations Victorian Novel Modern Architecture Price Earnings Ratio Modern Art SKA SFI 6 6 - 07

49. Predictive Models I love SKA music!! SFI 6 6 - 07

50. Model Interpreted Signals • Situations/objects in the world have many attributes (x1, x2, x3 …. xn) • Outcome function maps situations to outcomes/states F:X S • Agents have predictive models based on subsets of attributes. SFI 6 6 - 07