Markku S. Hannula Faculty of Education , University of Helsinki, Finland with

1. mathtrack:Mobile gaze tracking for the study of attention and emotion in collaborative mathematical problem solving Markku S. Hannula Faculty of Education, University of Helsinki, Finland with Anu Laine, Eeva Haataja, Enrique GarcíaMoreno-Esteva, Miika Toivanen, Visajaani Salonen & Jessica Salminen-Saari, Phuoc Tran (University of Helsinki), Nora McIntyre (Scheffied) David Clarke, Man Ching Esther Chan & Gaye Williams (University of Melbourne)

2. Structure of thepresentation • Affect as a research domain • Gazetracking as a method • MathTrackproject Käyttäytymistieteellinen tiedekunta / Henkilön nimi / Esityksen nimi

3. mathematics and affect • Emotions in the regulation of problem solving behaviour • Beliefs about mathematics explaining learning behaviour • Identity and choices (e.g. career decisions) • Anxiety and the mechanisms how it hinders learning • Enjoyment and flow as optimal learning mode • Boredom and its consequences The mechanisms of affective regulation are largely non-conscious and inaccessible to introspection or observation  Affectiveprocesses in schooltakeplace in a socialcontextwheredifferentsocialprocessesinteract How to dig deeper?

4. History pre 1990s (McLeod, 1992) • Behaviorism neglected affect as ”imaginary constructs” • Social psychology measured attitudes • Large number of quantitative studies • Differences between countries • Decline in affect over school years • Gender differences • Lack of a theoretical framework • Confusing , ambiguous terminology • Local theories (e.g. Self-efficacy, mathematics anxiety, attribution theories, aesthetics) Markku Hannula

5. New trends in 1990’s (McLeod, 1992) • Cognitive science • avoided affect to avoid complexity • ”affective and cognitive domains are intimately linked.” • E.g. Metacognition and affect (interest, confidence) • Cognitive psychology emphasizes theoretical issues, qualitative methods, beliefs and emotion Markku Hannula

6. Elements of affect

7. Main criticisms • Ambiguous terminology / Definitions • Conceptual framework • Social turn Markku Hannula ,Joensuu

8. Social turn in ME (Lerman 2000) • Theories on affect focussed on the individual • Cobb & Yakel: beliefs – norms (extension) • Identity (ICME 2004, Sfard & Prusak, etc.) Markku Hannula

9. A new (meta)theoreticalfoundation(Hannula 2011, 2012) Markku Hannula

10. State and trait aspects of the elements

11. How about the social turn and the embodied cognition? Markku Hannula

12. Is affectIndividual or Social? • Generated by individual experiences • Characteristics of the individual • Psychological theories • In-depth interviews • Constructed socially • Emerging in shared social scenarios • Social theories • Observing situated behaviour

13. Affect isIndividual and Social ! • Generated by individual experiences • Characteristics of the individual • Psychological theories • In-depth interviews • Constructed socially • Emerging in shared social scenarios • Social theories • Observing situated behaviour

14. Three readouts of emotion (Buck, 1999) Markku Hannula

15. Metatheoryfor affect(Hannula, 2012) State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

16. State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

17. State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

18. State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

19. State Trait Cognition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula

20. Social Psychological Embodied State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

21. Social Psychological Embodied State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

22. Social Psychological Embodied State Trait CognitionMotivation Emotion Faculty of Educational Sciences / Markku S. Hannula

23. The other angle to affect

24. Socialdynamics of affect Markku Hannula

25. BUILDING THINKING CLASSROOM Liljedahl, P. (2016). Building thinking classrooms: Conditions for problem solving. In P. Felmer, J. Kilpatrick, & E. Pehkonen(eds.), Posing and Solving Mathematical Problems: Advances and New Perspectives. (pp. 361-386). New York, NY: Springer. Liljedahl, P. (2014). The affordances of using visibly random groups in a mathematics classroom. In Y. Li, E. Silver, & S. Li (eds.), Transforming Mathematics Instruction: Multiple Approaches and Practices. (pp. 127-144). New York, NY: Springer.

26. Collaborativeproblemsolving • … is central in moderneducation • … canbestudiedusing mobile gazetrackers • … is interestingbeyondtheimmediateeducationalcontext MAVI 24 Markku S. Hannula

27. MathTrack • Four-yearproject 2016-2020 • Fundedby Academy of Finland

28. Why visual attention • Affective communication consists of more than just words; facial expressions, gestures, glances, body movement and prosody are also important aspects of it. • This multimodality of the communication raises a question regarding the student’s attention to the different channels used.

29. Eye tracking • Eye tracking (or gaze tracking) has been used to study visual attention mostly • in laboratory settings • loneindividuals • with a focus on cognition • BUT: Learning is contextual, affective, and social!

30. mobile gaze tracker (Hardware) • The teacher and four students wear a mobile gaze tracking device • Frame equipped with mini cameras which produce a video scene and keep track of the gaze direction. • The gaze tracking glasses are connected with three cords to a laptop carried in a backpack. • The glass frame is 3D-printed, all components are standard off-the-shelf, and the instructions and software are published • Do it yourself! Toivanen, M., Lukander, K., & Puolamäki, K. (2017). Probabilistic Approach to Robust Wearable Gaze Tracking. Journal of Eye Movement Research, 10(4), [2]. DOI: 10.16910/jemr.10.4.2 MAVI 24 Markku S. Hannula

31. Mobile GT software • The direction of gaze is software-calculated, producing a mark on the video scene. • The gaze location is computed as the recording takes place. MAVI 24 Markku S. Hannula

32. Methods – Main study:research design • Teacher and four students wear gaze-tracking glasses • Students work collaboratively on a mathematical problem • Same task across groups • Same structure (alone – with a friend – group of four – whole class) • Scripted teacher scaffolding (no hints, just encouragement and clarifying questions) • Stimulated recall interviews after the lesson

33. Data: synchronizedvideos+ stimulatedrecallinterviews + questionnaire

34. Gaze as a measure of collaboration (and engagement)Garcia Moreno-Esteva et al. (In preparation)

35. Potentialapproaches to studythedynamicgrouplevelaffect • Gazesynchrony • Indication of groupcohesion • Eyecontact (Eeva’s MAVI paper)orotherattentionto peers • Communication, cohesion • Reactions to facialexpressions and otherexpressions of affect • Reactivity to other’s, emotionalcontagion

36. Hannula, M. S. (2004a). Affect in mathematical thinking and learning. Turku, Finland: AnnalesuniversitatisTurkuensis B 273. • Hannula, M. S. (2004b. Regulating motivation in mathematics. A paper presented at the Topic Study Group 24 of ICME-10 conference. Retrieved September 15th 2005 at http://www.icme-organisers.dk/tsg24/Documents/Hannula.doc

37. Chapman, J. W., Tunmer, W. E. & Prochnow, J. E. (2000). Early reading-related skills and performance, reading self-concept and the development of academic self-concept: A longitudinal study. Journal of educational psychology 67, 145-152. • Damasio, A. R. (1994). Descartes Error: Emotion, Reason and the Human Brain, New York: G.P. Putnam’s Sons • Hannula, M.S., Maijala, H., & Pehkonen, E. (2004). Development of understanding and self-confidence in mathematics; grades 5-8. In M. J. Høines & A. B. Fuglestad (Eds.) Proceedings of the 28th Conference of the International Group for the Psychology of Mathematics Education. Vol 3, pp. 17 – 24. Bergen University College. • LeDoux, J. (1998). The Emotional Brain. Phoenix, Orion Books Ltd. • Linnanmäki, K. (2002). Matematikprestationerochsjälvuppfattning, en uppföljninggsstudie I relation till skolspråkochkön. [Mathematical proficiency and self-concept; a longitudinal study with an attention to language and gender]. Turku: ÅboAkademi University Press. • Hannula, M. S., Maijala, H., Pehkonen, E. & Nurmi, A. 2005. Gender comparisons of pupils’ self-confidence in mathematics learning.Nordic Studies in Mathematics Education 10 (3-4), 29 – 42

38. McLeod, D.B. (1992). Research on affect in mathematics education: a reconceptualization. In D.A.Grouws (Ed.) Handbook of Research on Mathematics Learning and Teaching, pp. 575-596. New York: MacMillan. • Furinghetti, F., & Pehkonen, E. (2002). Rethinking characterizations of beliefs. In. G. C. Leder, E. Pehkonen & G. Törner (Eds.) Beliefs: A Hidden Variable in Mathematics Education, pp. 39 – 58. Dordrecht, The Netherlands: Kluwer. • Boaler, J. (1997). Reclaiming school mathematics: The girls fight back. Gender and Education 9, 285-305. • Yakel, E. & Cobb, P. (1996). Sociomathematical norms, argumentation, and autonomy in mathematics. Journal for Research in Mathematics Education 27 (4), 458-477.

39. Buck, R. (1999). The Biological Affects: A Typology. Psychological Review 106 (2), 301-336. • Fennema, E. & Sherman, J.A. 1976. Fennema-Sherman mathematics attitudes scales. • JSAS Catalog of Selected Documents in Psychology, 6, 31 (Ms. No. 1225). Hart, L. (1989). Describing the affective domain: saying what we mean. In McLeod, D. & Adams, V. (eds.) Affect and Mathematical Problem Solving, 37-45. • McLeod, D.B. (1992). Research on affect in mathematics education: a reconceptualization. In D.A.Grouws (Ed.) Handbook of Research on Mathematics Learning and Teaching, pp. 575-596. New York: MacMillan.

40. Lehman, B., D'Mello, S. K., and Person, N., (2008). All Alone with your Emotions: An Analysis of Student Emotions during Effortful Problem Solving Activities. Supplementary Proceedings of the Workshop on Emotional and Cognitive issues in ITS (WECITS). • Lerman, S. (2000). The social turn in mathematics education research. In J. Boaler (Ed.), Multiple perspectives on mathematics teaching and learning (pp. 19–44). Westport, CN: Ablex. • Martin, A.J. (2001) The Student Motivation Scale: a tool for measuring and enhancing motivation, Australian Journal of Guidance and Counselling11 (2001), pp. 1–20 • Meyer, D. K. & Turner, J. C. (2002). Discovering emotion in classroom motivation research. Educational Psychologist 37 (2), 107 – 114.

41. Rösken, • Schoenfeld, A. H. (1985). Mathematical Problem Solving. San Diego: Academic Press. • Goldin, G. A. (2000). Affective pathways and representation in mathematical problem solving. Mathematical Thinking and Learning 2(3), 209-219. • Liljedahl, P. (2005). Sustained engagement: preservice teachers' experience with a chain of discovery. In Proceedings of the Fourth Congress of the European Society for Research in Mathematics Education 17. – 21.2. 2005 in SantFeliu de Guíxols, Spain. Retrieved 3.11.2005 at http://cerme4.crm.es/Papers%20definitius/2/wg2listofpapers.htm