Summarization and Personal Information Management

1. Summarization and Personal Information Management Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute

2. Announcements • Questions? • Poster session next Tuesday in class! • Plan for Today • Overview Discussions • Impact of Moderation on Group Behavior (Marty McGuire) • Focus on Group Deliberation Processes (Gahgene Gweon) • Close up look at specific group processes • Discussion of summarization work related to enabling facilitators to manage group processes

3. Homework 2: Due Feb 19 in class

4. Homework 2: Due Feb 19 in class

5. Models of Information Overload for Individuals

6. Continued focus on the individual

7. What about groups?

8. Sunstein Book Quote • Why are individual cognitive errors so often amplified at the group level? Informational pressures and social influences are unquestionably at work.

9. Putting it into perspective… Information Literacy Information Overload Groups making good decisions through processing information effectively Groups making poor decisions through processing information ineffectively Moderators support group decision making and group behavior.

10. Results from Behavioral Studies Inform Design New focus on groups as opposed to individuals Problem Human Behavior Solution Design Technology Today’s focus Problem?

11. Impact of Moderation on Group Behavior

12. Marty Robert McGuire

13. Cosley, et al. 2005 • GroupLens Research • Background and Motivation • Contribution on MovieLens • Experimental design • Results • Discussion

14. GroupLens Research • http://grouplens.org/ • Built MovieLens to test collaborative filtering algorithms • Rate movies • Receive recommendations • http://movielens.org/ • Excellent platform for studying user contribution online

15. Collective Effort Model(Karau and Williams)‏ • Why contribute? • Public goods • Why not contribute? • Free-riding • Social loafing • “Tragedy of the commons”

16. Oversight • Member contribution • “Free” content • Many concerns • Quality control • Fraud • Vandalism • Oversight can: • Reduce low-quality contributions • Encourage high-quality contributors

17. Contribution on MovieLens • Chad manages the movie database • Members can make suggestions • Problems • Chad is too busy • Chad is a tyrant? • Solution • Allow members to add movies to the database

18. 5 Hypotheses More oversight leads to... ...less anti-social behavior ...higher quality initial contributions ...higher final quality information in the database ...more contributions overall Knowing that oversight is present or absent will increase the effect of oversight

19. Experimental Design • 3x2 Design • 3 different levels of oversight • None • Peer • Expert • 2 different levels of oversight visibility • Visible • Non-visible Any Issues? Movies can only be added if taken from the suggestion queue Creates a social dilemma... “Do I add this movie, or hijack the entry for a movie that I care about?”

20. Results • Lots of interesting stuff here! • Unfortunately, not all statistically significant • Your thoughts?

21. What about the 5 Hypotheses? More oversight leads to... ...less anti-social behavior ...higher quality initial contributions ...higher final quality information in the database ...more contributions overall Knowing that oversight is present or absent will increase the effect of oversight

22. Additional Discussion Did the 20-movie addition cap have an effect? “I also didn’t like how you capped the number of additions I was able to make. In all honesty I would have added several hundred more had I been allowed.” Unintended consequences “I am proud of the work I have done...this is like an architect having to watch his building being torn down. I fear that any more than a few days of this and the cleanup will become incomprehensible...you have to remember how IMDb keeps their stuff clean!! It’s certainly not by way of the users!!” “Using social science theory to drive and critique design is promising, but not foolproof. For instance, the collective effort model has several weaknesses.” Anything else?

23. Questions • Do you believe the results support the collective effort model? • What might be an alternative hypothesis of what is going on? • Do you agree that the hypothesis that knowledge of oversight increases the effect of moderation? • What does the collective effort model predict about automatic moderation?

24. Group Deliberation Processes

25. Four Big Problems • Sunstein, C. (2006). Four Big Problems. In Infotopia: How Many Minds Produce Knowledge (Chapter 3), Oxford University Press, pp75-102.

26. Student Quote • I have to wonder what governmental groups and other "deliberating bodies" know about these types of phenomena, and how that knowledge can be applied to swaying group decisions!

27. Need for support • Group projects are necessary, yet difficult to accomplish • Benefit: access to broader skill sets, potential to learn from one another (Pea 1993; Salomon 1993) • Challenges: dealing with conflict (Faidley 2000), social loafing (Karau & Williams 1993), coordinating individual contributions (Strijbos 2004) • Managers or instructors oversee group process and provide feedback • Support from expert facilitators improves group work (Meloth & Deering 1999, Hmelo-Silver 2004)

28. My research • Agenda: How can we aid instructors in providing better guidance to student groups? • Goal: provide a lens for instructors to make the group processes more visible so they can intervene • Method: Observe student group meetings’ group process -> detect conversational patterns -> apply Machine learning technology to predict outcome

29. Feasibility Test: Predicting Productivity * Using instructor assigned productivity ratings as gold standard * Best performance without text features: .16 correlation coefficient * Best performance with text features: .63 correlation coefficient

30. Supporting Project Course Instructors * Design based on interview data discussed last week.

31. Steiner, Hackman & McGrath Deliberation

32. My research & current chapter • Collect different types of problems that occur during group processes and try to automatically detect such problem instances • Current chapter also talks about problems that occur during group processes, specifically conversation itself INPUT PROCESS OUTPUT

33. Problems that occur during “process” • Story of 2 students who never communicated • 4 problems from the chapter • Amplifying Errors • Hidden Profiles • Cascades • Polarization • Can we share examples of each problems based on your experience?

34. Amplifying Errors • “some garbage in, much garbage out” • Example: if individual jurors are biased and think that the defendant committed a crime, the jury’s opinion on the defendant will amplify. • Why? • Your example? Availability heuristic, framing effects, representative heuristics, conjunction errors

35. Hidden Profiles • Accurate understating that groups could obtain but do not. • Example: US decision on Bay of Pigs. People did not reveal evidence against invasion although they had the information. • Why? • Your example? Common knowledge effect, informal influence, social pressure

36. Cascades • Individuals do not reveal what they know based on prior opinion formed • Example: One person chimes in with an example of something. Several other people echo that example with similar examples • Why? • Your example? Informational cascade, reputational cascade

37. Polarization • Ending up in a more extreme position in line with one’s tendencies after the deliberation • On scale of 1~8, how likely is a terrorist attack? If an individual originally thinks 5, after deliberation he will think 6. • Why? • Your example? Information influence, social influence, confidence & extremism, conform to group

38. Brain Storming • Think about the four problems: • Amplifying Errors, Hidden Profiles, Cascades, Polarization • What would you look for to detect that these were occurring? • What could a human moderator do to regulate these processes? • What could be done through automatic moderation if the detection was accurate enough?

39. Questions?