Integrated Collaborative Learning Environments with Dynamic Support

1. Integrated Collaborative Learning Environments with Dynamic Support Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute

2. Design PrincipleKnow what problem you are trying to solve!!

3. Passivity in Class Discussions

4. Accountable Talk A codified set of moves that Ts and Ss can be taught that build effective language in the classroom Press individuals to explain and elaborate Require individuals to listen and respond to each other Reveal misunderstandings, incomplete ideas.. Let Ss know how others are “hearing them” Spread participation through the group

5. Transactivity • Students explicitly display their reasoning • Students orient their contributions towards previous contributions • May be competitive or non-competitive • May be representational operational • May be oriented towards self or other Student 1: Alright, we put that the negative and positive whole numbers. Teacher: Alright, other ideas you want to add on, yes Mario. Student 2: I put a um numbers that doesn't include decimal points or fractions but they include, they can include negative signs or positive signs.

6. Accountable Talk elicits Transactivityin group discussion

7. Accountable Talk elicits Transactivityin group discussion Eddie: Well, i don't think it matters what order the numbers are in. You still get the same answer. But three times four and four times three seem like they could be talking about different things. Teacher: Rebecca, do you agree or disagreewith what Eddie is saying? Rebecca: Well, I agree that it doesn't matter which number is first, because they both give you twelve. But I don't get what Eddie means about them saying different things. Teacher: Eddie, would you explain what you mean? Eddie: Well, I just think that like three times four can mean three groups of four things, like three bags of four apples. And four times three means four bags of three apples, and those don't seem like the same thing. Tiffany: But you still have the same number of apples, so they are the same! Teacher: OK, so we have two different ideashere to talk about. Eddie says the order does matter, because the two orders can be used to describe different situations. So Tiffany, are you sayingthat three times four and four times three can't be used to describe two different situations? Justification Request Reasoning Critique Clarification Revoicing Reasoning Application Request Request for Elaboration

8. Unequal Participation

9. Wizard-of-Oz Setup Tutor Student Student

10. Problem Solving Interface Interface allows students to submit Incorrect solutions

11. Reflection Interface

12. Scaffolding for Collaboration • Prompts to encourage the instructionally beneficial behavior of students • Explanations • Equal contributions

13. Explanation Prompt (Student 1 corrects the answer for the confederate learner without offering any explanation) Student1: c is 1 Auto-Prompt (to student 1): Help the student understand your correction. Student2 : what does c mean? Student1 : constant Student2 : ok Student1 : the constant in front of the variable which in our case is variable t Student2 : ok Student1 : now for the second part we use the constant function

14. Equal Contribution Prompt (After the participant makes 5 correct problem actions without confederate student’s contribution) Auto-Prompt: It seems like the other student has not contributed lately. Why don’tyou see if they need help? Student1: you want to do the other half? Student2: ok

15. Learning effect • Significant benefit for prompting on student learning with pretest as a covariate • F(1,39) = 4.47, p < .05, effect size .58

16. Process Losses

17. Another Domain: Earth Sciences • Collaborative idea generation in the Earth Sciences domain • Chinese TagHelper learns hand-coded topic analysis • Human agreement .84 Kappa • TagHelper performance .7 Kappa • Trained models used in follow-up study to trigger interventions and facilitate data analysis

18. Individuals+ Feedback Individuals+ NoFeedback Pairs+ Feedback Pairs+ NoFeedback Example Dialogue * Feedback during idea generation increases learning and idea generation productivity (except during the first 5 minutes) (Wang et al., 2007)

19. Individuals+ Feedback Individuals+ NoFeedback Pairs+ Feedback Pairs+ NoFeedback Unique Ideas 12 Nom+N Nom+F Real+N 10 Real+F 8 #Unique Ideas 6 4 2 0 0 5 10 15 20 25 30 Time Stamp Process Analysis Process loss Pairs vs Individuals: F(1,24)=12.22, p<.005, 1 sigma Individuals+Feedback Individuals+NoFeedback Pairs+Feedback Pairs+NoFeedback Process loss Pairs vs Individuals: F(1,24)=4.61, p<.05, .61 sigma Negative effect of Feedback: F(1,24)= 7.23, p<.05, -1.03 sigma Positive effect of feedback: F(1,24)=16.43, p<.0005, 1.37 sigma

20. Deficient Help Exchange

21. Why study help? Offering of deep help and elaborated explanations predicts post test performance (e.g., Webb et al., 2002) Help behavior mediates learning (e.g., Gweon et al., 2006; Gweon et al., 2007) Exchanging help in the context of collaborative learning increases identification with a learning community and motivation, and improves race relations (Sharan, 1980) Providing help leads to feelings empowerment for “low status” students (Elbers & Hann, 2004)

22. How can we prompt helping behavior? • DIRECT: Explicit prompts (Gweon et al., 2006) • LESS DIRECT: Manipulating availability of help from problem solving environment (Gweon et al., 2007) • Girls offered more help with delayed feedback from environment • Boys offered more help with immediate help from the environment • SUBTLE: Conversation agents inject humor (Kumar et al., 2007) • Positive effects on perceived help exchange (p<.05), effective help exchange (p<.1), and Learning (p=.06)

23. Jan packed several books to amuse herself on along car ride to visit her grandma. After 1/5 of the trip she had already finished 6/8 of the books she brought. How many times more books should she have brought than what she packed? Collaborative Problem Solving Environment

24. Extraneous Entertainment?

25. Coding Scheme (Gweon et al., 2007) • Help provision mediates learning (Gweon et al., 2006; Gweon et al., 2007) • (R) Help Requests: “Help me”, “I’m stuck”, “I don’t get it.” • (P) Help Provisions: “Find the common denominator”, “Try the flip and multiply strategy” • (C) Can’t help: “I don’t know”, “I’m stuck too” • (D) Deny help: “ask the teacher”, “you’re an idiot”, “press the help button” • (N) Nothing

26. Examples [R] Student 1: What do we put on top of the fraction? [P] Student 2: Did you find a common denominator? <student 1 correctly finds the common denominator> [R] Student 1: I don’t get it [D] Student 2: hold on <then student 1 tried something and got negative feedback from the problem solving environment.> <finally student 1 tried something else, which was correct, and got positive feedback from the problem solving environment> [R] Student 1: Why 16? [C] Student 2: I don’t know. [R] Student 1: I don’t get it <student 2 tries something and gets negative feedback from the problem solving environment> <student 2 tries something else and gets negative feedback from the problem solving environment> <student 2 clicks on the help button> <student 1 tries something that is correct and gets positive feedback from the problem solving environment>

27. Big findings:More help related episodes in experimental conditionMore episodes where people got help, and then managed to solve the problem themselves in the experimental condition

28. Vision

29. Vision • Support for collaborative learning is like training wheels • Effective support allows learners to achieve better collaboration • Unnecessary support can be demotivating • Fading support is ideal • But too little support can be detrimental as well • Trained human facilitators are able to achieve the right balance

30. Vision • Support for collaborative learning is like training wheels • Effective support allows learners to achieve better collaboration • Unnecessary support can be demotivating • Fading support is ideal • But too little support can be detrimental as well • Trained human facilitators are able to achieve the right balance

31. Vision • Support for collaborative learning is like training wheels • Effective support allows learners to achieve better collaboration • Unnecessary support can be demotivating • Fading support is ideal • But too little support can be detrimental as well • Trained human facilitators are able to achieve the right balance

32. Vision • Support for collaborative learning is like training wheels • Effective support allows learners to achieve better collaboration • Unnecessary support can be demotivating • Fading support is ideal • But too little support can be detrimental as well • Trained human facilitators are able to achieve the right balance

33. Vision • Support for collaborative learning is like training wheels • Effective support allows learners to achieve better collaboration • Unnecessary support can be demotivating • Fading support is ideal • But too little support can be detrimental as well • Trained human facilitators are able to achieve the right balance

34. Vision • Support for collaborative learning is like training wheels • Effective support allows learners to achieve better collaboration • Unnecessary support can be demotivating • Fading support is ideal • But too little support can be detrimental as well • Trained human facilitators are able to achieve the right balance

35. Architecture for Dynamic Collaboration Support

36. VMT-Basilica ** Students learn up to 1.25 standard deviations more when interactive support is provided in the environment.

37. Basilica Architecture A channel filter collects all of the events that occur at the interface from all students involved in the conversation

38. Basilica Architecture Events are then channeled to special purpose filters such as a text processing filter Other filters may keep track of time or other factors not related to student behavior

39. Basilica Architecture Filters that are related to decision making for specific types of interventions receive notifications from the analysis filters and pass notifications on to the Actors, which are responsible to launching interventions

40. Basilica Architecture Actors of different types produce events that are sent to the Outgoing Message Spooling Filter From there, events are sent through the Presence Actor to the interface.

41. Issues to consider • What problem are you trying to solve? • Formulate analysis scheme • When should you intervene? • Use technologies like TagHelper and SIDE to track interaction and trigger support • What should the intervention be? • Technologies like TuTalk can be used to offer support

42. Questions?