SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Wisdom of Crowds in Human Memory: Reconstructing Events by Aggregating Memories across Individuals

blue avatar
blue
84 diapositives · 989 Views · Aug 04, 2014

This research explores the "wisdom of crowds" phenomenon, demonstrating how group estimates can surpass individual judgments by aggregating collective memories. Key tasks studied include ordering events (like US presidents), ranking cities, and matching items. Using Bayesian models, the project addresses challenges in memory reconstruction while incorporating individual differences and cognitive processes. By employing hierarchical models, the study aims to improve event recall and estimation accuracy, offering insights into cognitive sciences and collaborative memory.

Wisdom of Crowds in Human Memory: Reconstructing Events by Aggregating Memories across Individuals - slide 1
01 / 84Wisdom of Crowds in Human Memory: Reconstructing Events by Aggregating Memories across Individuals
1 / 17

Télécharger la présentation

Wisdom of Crowds in Human Memory: Reconstructing Events by Aggregating Memories across Individuals
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Wisdom of Crowds in Human Memory: Reconstructing Events by Aggregating Memories across Individuals Mark Steyvers Department of Cognitive Sciences University of California, Irvine Joint work with: Brent Miller, Pernille Hemmer, Mike Yi Michael Lee, Bill Batchelder, Paolo Napoletano

  2. Wisdom of crowds phenomenon • Group estimate often performs as well as or better than best individual in the group

  3. Examples of wisdom of crowds phenomenon Galton’s Ox (1907): Median of individual estimates comes close to true answer Who wants to be a millionaire?

  4. Tasks studied in our research • Ordering/ranking problems • declarative memory: order of US presidents, ranking cities by size • episodic memory: order of events (i.e., serial recall) • predictive rankings: fantasy football • Matching problems • assign N items to N responses • e.g., match paintings to artists, or flags to countries • Traveling Salesman problems • find shortest route between cities  problems involving permutations

  5. Recollecting order from Declarative Memory Abraham Lincoln Ulysses S. Grant time Ulysses S. Grant Rutherford B. Hayes Rutherford B. Hayes James Garfield Abraham Lincoln Andrew Johnson James Garfield Andrew Johnson Place these presidents in the correct order

  6. Recollecting order from episodic memory http://www.youtube.com/watch?v=a6tSyDHXViM&feature=related

  7. Place scenes in correct order (serial recall) A B C D time

  8. Goal: aggregating responses ground truth group answer ? A B C D = A B C D Aggregation Algorithm A D B C D A B C B A D C A C B D A B D C

  9. Bayesian Approach group answer = latent random variable A B C D Generative Model A D B C D A B C B A D C A C B D A B D C

  10. Task constraints • No communication between individuals • There is always a true answer (ground truth) • Aggregation algorithm never has access to ground truth • unsupervised methods • ground truth only used for evaluation

  11. Research Goals • Aggregation of permutation data • going beyond numerical estimates or multiple choice questions • combinatorially complex • Incorporate individual differences • going beyond models that treat every vote equally • assume some individuals might be “experts” • Take cognitive processes into account • going beyond mere statistical aggregation  Hierarchical Bayesian models

  12. Part IOrdering Problems

  13. Experiment 1 • Task: order all 44 US presidents • Methods • 26 participants (college undergraduates) • Names of presidents written on cards • Cards could be shuffled on large table

  14. Measuring performance Kendall’s Tau: The number of adjacent pair-wise swaps = 1 = 1+1 = 2 Ordering by Individual A B E C D A B E CD E C D A B C D E A B True Order A B C D E

  15. Empirical Results (random guessing) t

  16. Many methods for analyzing rank data… • Probabilistic models • Thurstone (1927), Mallows (1957), Plackett-Luce (1975) • Lebanon-Mao (2008) • Spectral methods • Diaconis (1989) • Heuristic methods from voting theory • Borda count … however, many of these approaches were developed for preference rankings

  17. Bayesian models constrained by human cognition • Extension of Thurstone’s (1927) model • Extension of Estes (1972) perturbation model

  18. Bayesian Thurstonian Approach C B A Each item has a true coordinate on some dimension

  19. Bayesian Thurstonian Approach Person 1 B A C … but there is noise because of encoding and/or retrieval error

  20. Bayesian Thurstonian Approach Person 1 B A C B C A Each person’s mental representation is based on (latent) samples of these distributions

  21. Bayesian Thurstonian Approach Person 1 B A C Observed Ordering: A < B < C B C A The observed ordering is based on the ordering of the samples

  22. Bayesian Thurstonian Approach Person 1 B A C Observed Ordering: A < B < C B C A Person 2 B C A Observed Ordering: A < C < B C B A People draw from distributions with common means but different variances

  23. Graphical Model Notation j=1..3 shaded = observed not shaded = latent

  24. Graphical Model of Bayesian Thurstonian Model Latent ground truth Individual noise level Mental representation Observed ordering j individuals

  25. (weak) wisdom of crowds effect t model’s ordering is as good as best individual (but not better)

  26. Inferred Distributions for 44 US Presidents George Washington (1) John Adams (2) Thomas Jefferson (3) James Madison (4) median and minimumsigma James Monroe (6) John Quincy Adams (5) Andrew Jackson (7) Martin Van Buren (8) William Henry Harrison (21) John Tyler (10) James Knox Polk (18) Zachary Taylor (16) Millard Fillmore (11) Franklin Pierce (19) James Buchanan (13) Abraham Lincoln (9) Andrew Johnson (12) Ulysses S. Grant (17) Rutherford B. Hayes (20) James Garfield (22) Chester Arthur (15) Grover Cleveland 1 (23) Benjamin Harrison (14) Grover Cleveland 2 (25) William McKinley (24) Theodore Roosevelt (29) William Howard Taft (27) Woodrow Wilson (30) Warren Harding (26) Calvin Coolidge (28) Herbert Hoover (31) Franklin D. Roosevelt (32) Harry S. Truman (33) Dwight Eisenhower (34) John F. Kennedy (37) Lyndon B. Johnson (36) Richard Nixon (39) Gerald Ford (35) James Carter (38) Ronald Reagan (40) George H.W. Bush (41) William Clinton (42) George W. Bush (43) Barack Obama (44)

  27. Model can predict individual performance t individual t distance to ground truth s inferred noise level for each individual

  28. Extension of Estes (1972) Perturbation Model • Main idea: • item order is perturbed locally • Our extension: • perturbation noise varies between individuals and items True order A B C D E A C B D E Recalled order

  29. Strong wisdom of crowds effect t Perturbation Perturbation model’s ordering is better than best individual

  30. Inferred Perturbation Matrix and Item Accuracy Abraham Lincoln Richard Nixon James Carter

  31. Alternative Heuristic Models • Many heuristic methods from voting theory • E.g., Borda count method • Suppose we have 10 items • assign a count of 10 to first item, 9 for second item, etc • add counts over individuals • order items by the Borda count • i.e., rank by average rank across people

  32. Model Comparison t Borda

  33. Experiment 2 • 78 participants • 17 problems each with 10 items • Chronological Events • Physical Measures • Purely ordinal problems, e.g. • Ten Amendments • Ten commandments

  34. Example results Perturbation Model Thurstonian Model 1. Oregon (1) 2. Utah (2) 3. Nebraska (3) 4. Iowa (4) 5. Alabama (6) 6. Ohio (5) 7. Virginia (7) 8. Delaware (8) 9. Connecticut (9) 10. Maine (10) 1. Freedom of speech & relig... (1) 2. Right to bear arms (2) 3. No quartering of soldiers... (3) 4. No unreasonable searches (4) 5. Due process (5) 6. Trial by Jury (6) 7. Civil Trial by Jury (7) 8. No cruel punishment (8) 9. Right to non-specified ri... (10) 10. Power for the States & Pe... (9)

  35. Average results over 17 Problems t Strong wisdom of crowds effect across problems Mean Individuals

  36. Predicting problem difficulty city size rankings t t distance of group answer to ground truth ordering states geographically std( s ) dispersion of noise levels across individual

  37. Effect of Group Composition • How many individuals do we need to average over?

  38. Effect of Group Size: random groups t

  39. Experts vs. Crowds • Can we find experts in the crowd? Can we form small groups of experts? • Approach • Form a group for some particular task • Select individuals with the smallest sigma (“experts”) based on previous tasks • Vary the number of previous tasks

  40. Group Composition based on prior performance # previous tasks T = 0 T = 2 T = 8 t Group size (best individuals first)

  41. Methods for Selecting Experts Endogenous: no feedback required Exogenous: selecting people based on actual performance t t

  42. Aggregating Episodic Memories Study this sequence of images

  43. Place the images in correct sequence (serial recall) A B C D E F G H I J

  44. Average results across 6 problems t Mean

  45. Example calibration result for individuals t individual distance to ground truth s inferred noise level (pizza sequence; perturbation model)

  46. Predictive Rankings: fantasy football Australian Football League (29 people rank 16 teams) South Australian Football League (32 people rank 9 teams)

  47. Part IIMatching Problems

  48. Study these combinations

  49. Find all matching pairs C A B D E 1 2 3 4 5

  50. Experiment • 15 subjects • 8 problems • 4 problems with 5 items • 4 problems with 10 items