Building Mental Models with Visuals for e-Learning

1. Building Mental Models with Visuals for e-Learning ShalinHai-Jew Aug. 7 – 10, 2008 Minneapolis, MN MERLOT: Still Blazing the Trail and Meeting New Challenges in the Digital Age

2. Mental Models Definitions: A learner’s internal conceptualization of a system / paradigm, situation, personage, phenomena or equipment; a learner’s sense making (vs. an instructor’s “conceptual model” on which the designed learning is based); implicit and explicit knowledge, internalized and externalized knowledge; subconscious and conscious beliefs Synonyms: An underlying substructure; an analogue of the world; an operationalized mental template for “meaning and form” (Riggins & Slaughter, 2006, p. 4); worldview Building Mental Models with Visuals for E-Learning

3. Mental Models (cont.) Ideas People Act On: Theories in use vs. espoused theories (the implied theories based on people’s actions vs. what they say they believe), to paraphrase Argyris & Schon Structured Knowledge: A hypothetical knowledge structure “that integrates the ideas, assumptions, relationships, insights, facts, and misconceptions that together shape the way an individual views and interacts with reality” (Steiger & Steiger, 2007, p. 1); an embodiment of domain knowledge in order to abstractly reason about the domain (objects, grouping, interrelationships, sequences, processes, and behaviors) Building Mental Models with Visuals for E-Learning

4. Theoretical Underpinnings Constructivism, which asserts that learners make meanings in their own minds Cognitivism, the study of the human mind, awareness and mental functions, especially the Dual Channel Model (auditory-verbal and visual-pictorial), with strategies to maintain cognitive load (Mayer & Moreno, 1998) Building Mental Models with Visuals for E-Learning

5. Metaphoric Mapping Building Mental Models with Visuals for E-Learning

6. Some Typesof Mental Models Representational: describes, articulates, renders coherent, illustrates, and defines Predictive: anticipates, proposes trend lines, predicts, and projects / forecasts Proscriptive: defines how something should be ideally Speculative: proposes an un-testable thesis, purely theoretical (may be mental models at extreme scales, beyond sight, sound, and human perception) Building Mental Models with Visuals for E-Learning

7. Some Typesof Mental Models (cont.) Live Info-Heavy Modeling: collates live, dynamic, and multi-variate information (from remote sensors, cameras, people, and other sources) into a semi-coherent larger view / visualization; may be user-interactive and user-manipulatable for analysis and decision-making Building Mental Models with Visuals for E-Learning

8. Pedagogical Considerations Identification of threshold concepts—difficultcore concepts that once understood provide a broad base for comprehension of more advanced concepts The fomenting of cognitivedissonance related to beliefs, perception, attitudes, behaviors to help learners adjust internal mental models Building on naïve mental models which tend to be elusive, poorly formed, incomplete, poorly structured, difficult to articulate, illogical, overly generalized, and invalid (from Nardi & Zarmer, 1991, p. 487) Building Mental Models with Visuals for E-Learning

9. Pedagogical Considerations (cont.) Surfacing learner mental models; offering methods to test assumptions; avoiding negative transfer; avoiding undesirable dependencies to the learning tools, and reinforcing accurate conclusions and observations (consistently, with augmenting cues) Identifying and addressing misconceptions (of objects, of relationships, fragmented knowledge, comparison, contrast, over-generalization, over-simplification, and others) Building Mental Models with Visuals for E-Learning

10. Pedagogical Considerations (cont.) Must have a defined learning approach: clear context, defined terminology, defined and reasonable learning objectives, defined objects, clear relationships, interactivity, processes, and a systems view in the design; offer opportunities for “discovery methods” to build mental models (Moreno, 2004, p. 99) Should have some consideration for self-regulated learning (SRL) and self-discovery learning (SDL), particularly with automated learning constructs and self discovery learning virtual spaces Building Mental Models with Visuals for E-Learning

11. Pedagogical Considerations (cont.) Should consider expert approaches (establishing a context for the task; classifying problems based on underlying principles and concepts) (Alexander & Judy, Winter 1988, p. 382) Building Mental Models with Visuals for E-Learning

12. Designing a Mental Model Identify a learning domain. Select a portion (or the whole) to model. Define the foundational realities. Define the learning objectives and outcomes. Define the relevant terminology and nomenclature. Define the range of possible variables and measures. Define relevant processes within the model. Prototype and build the mental model while considering and adhering to mental modeling standards. Build learning scenarios. Build test scenarios, and test with novices and experts. Building Mental Models with Visuals for E-Learning

13. Setupsfor Mental Model Learning Pre-learning Human facilitation / automated facilitation Decision supports Learner tracking and measures Debriefing Pre- and post-testing Takeaways and downloadables Building Mental Models with Visuals for E-Learning

14. Digital Mental Model Viability The accuracy and comprehensiveness of the depicted information The fidelity and logical alignment of the model to the real world The appliedpredictability and utility particularly with multiple interrelated factors in play (with clear lines of reasoning) The efficacy of the model used in conjunction with other tested models in a complex environment The expressibilityor communicability (vs. implicit representational forms like metaphors or symbols) Building Mental Models with Visuals for E-Learning

15. Digital Mental Model Viability(cont.) The timeliness (non-obsolescence / real-time value) and updatability of the model (with new information) The soundness of the assumptions and premises (without inherent biases or propensities) The portability of the mental model between technological systems / of information from the model The real-time, time-varying, information-rich, multi-stream, multi-representational and real-time feedback Building Mental Models with Visuals for E-Learning

16. Digital Mental Model Viability(cont.) The aesthetic presentation The originality and uniqueness of the digital mental model The malleability of the model to incorporate new, foundational design elements The legality of the materials (accessibility, intellectual property rights, avoidance of libel and slander, and others) Building Mental Models with Visuals for E-Learning

17. Mental Models Enablement with High-Tech Visualizations and Graphics in E-Learning Building Mental Models with Visuals for E-Learning

18. Seeing Illusion-making and the hard-wiring of the human brain and visual perception Flow and movement; 1D, 2D, 3D and 4D; transparency and overlap; texture; luminance, brightness, saturation, and reflectance; contrast; shape, size, boundaries and edges; axes, planes, and others “Gestalt Laws” of Pattern Perception: proximity, similarity, connectedness, continuity, symmetry, closure, relative size, figure and ground (Westheimer, Koffka, & Kohler, 1912, 1935, as cited in Ware, 2004, pp. 189 – 197) Labels and text, sounds, and voice Building Mental Models with Visuals for E-Learning

19. High Tech Affordances Structure mapping with computer coding (ontologies, taxonomies) and spatial layouts (bubble graphs, node-link diagrams) of mental maps Uses of multimedia multi-channel modes for coherence, transparency, and clarity (some with real-time elements) Simulation for similarity, experiential learning, and full-sensory immersion (with experiential continuity) Situated cognition for field-dependent learning; situated action for field-dependent analysis, decision-making and behaviors Building Mental Models with Visuals for E-Learning

20. High Tech Affordances (cont.) Informational visualizations (2D, 3D and 4D…) to capture ever greater informational complexity without clutter or visual confusion (and without cognitive overload) Environmental visualizations via animation and movement; sequential experiences; branched experiences Interactivity and hypothesis testing Immersive spaces with automation, intelligent agents and / or live human interactions (and possible mediation) Information gathering in the “digital enclosure” [swarm behavior, “urban probes,” stalked trashcans, “lost postcard” with URLs behaviors (Paulos & Jenkins, 2005, pp. 341 – 350) Building Mental Models with Visuals for E-Learning

21. Conveying a Mental Model Realiaand real-world artifacts (digital stills and video feeds, live sensor feeds, mobile sensors), introduction of serendipity and “apparent chance” Simulations, depictions (sensory details: sight, sound, smell, taste, and touch) Behaviors (agent and user) and interactivity (with feedback loops) User informed choices Avoidance of unintentional negative learning Building Mental Models with Visuals for E-Learning

22. VisualizationforPresentation Representational, descriptive and realistic or theoretical, conceptual or imaginary (alternate conceptual universes) Holistic or partial, decomposition of images, pullouts Process dynamism or change vs. static Stylized or non-stylized, natural High, medium or low fidelity; selective or non-selective fidelity Conveyance of emotions through emo ‘bots and agents, building trust and relations with robots Macro or micro perspectives Discrete or continuous (Tory & Möller, 2002, as cited in Tory & Möller, Jan. – Feb. 2004, p. 72) Building Mental Models with Visuals for E-Learning

23. VisualizationforPrototyping Modeling production designs and blueprints Representing different phases of a build Revising plans Effective for virtual teaming shared mental models (but avoiding “groupthink”) (Thomas & Bostrom, 2007, pp. 1 – 8) Building Mental Models with Visuals for E-Learning

24. VisualizationforCulling Data May extract data from visual captures (such as road information from a satellite image, traffic imagery; gas dispersion flows from a live site; quantifying the number of people or objects at a scene; facial recognition software; checking architectural compliances in terms of distances in a blueprint; 3D imaging or cross-sections of a tumor; identification of correlations between forms / images; simulated flows and transitions; weathering and aging; forensically analyzing satellite images; disaster response; natural resources management; agricultural planning, and others) Building Mental Models with Visuals for E-Learning

25. VisualizationforCulling Data(cont.) Digital re-constructions of events Digital cartography / map-making Simmed projections of potential events (with or without human inputs / interactions) Deformation and animation of soft objects (from video captures); de-noising image captures (for clearer info); feature enhancements Analyzing hyperspectral imagery Building Mental Models with Visuals for E-Learning

26. VisualizationforOrganizing Data Defining relationships between informational objects in a domain-specific database as compared to an expert-based domain competency model (Ahmad, et al., June 2007, pp. 452 – 461); checking mental models (naïve and expert) Building Mental Models with Visuals for E-Learning

27. Some Mental Models inE-Learning Simulations Immersive learning spaces Role playing in case studies, team or group simulations, simmed decision-making (e.g. game theory) Knowledge systems, ontologies, and taxonomies, with user interfaces that map with learning realities Traditional e-learning Building Mental Models with Visuals for E-Learning

28. 2DVisuals • Sketches, drawings, blueprints, diagrams, charts, tables, timelines, icons, symbols and designs • Slideshows (static and dynamic) • Screenshots • Interactive maps, screencasts, and games • Photo montage, photorealistic images • Non-photorealistic images and depictions • Video • Animated agents, avatars, maquettes / models for intended work • Satellite imagery, live data-fed images, and acoustical imaging Building Mental Models with Visuals for E-Learning

29. 3Dand4DVisuals Fractals Videos, field recordings Animated agents, avatars, maquettes (models of intended works like a sculpture), and scenes Satellite imagery, live data-fed images, and acoustical imaging Immersive spaces, microworlds, and metaworlds Augmented reality, ambient space Holography Haptic visual interface 4D + 3D with time (temporal changes and motions) Building Mental Models with Visuals for E-Learning

30. Combined 2D and 3D Visualizations Orientation indicators (icons, separate windows) In-place methods (clip and cutting planes) Orthographic 2D overlays around a 3D object Medical imaging Flow visualization Oceanographic visualization Computer aided design (Tory, Oct. 2003, p. 371) Building Mental Models with Visuals for E-Learning

31. Capturing Images in Digital Form • Digital cameras, mobile devices, mini-cams • Scanners • Microscopes • Telescopes • 3D devices / multiple synced cameras • Sonar image devices, acoustic image devices • Remote sensors, mobile robot sensors, unmanned aerial vehicles (UAVs) • 3D game engines • Database-stored information and statistics • Radar • Satellite • Telephone call registries • Remote labs • Digital pens and tablets • CAD / CATIA • Desktop screen captures • 2D to 3D with minimal image sets (as in ubiquitous video for immersive “flythroughs” for situational awareness) Building Mental Models with Visuals for E-Learning

32. The Role of Digital Visuals … digital storytelling…science-based digital wetlabs…medical diagnosis…deep sea exploration…outer space exploration…aerial image analysis…human facial identification…museum and art gallery captures…manga illustrations…information extraction…machine art…telemedicine…immersive simulations…visual information access…video tooning…architectural designs…landscape architecture…performance art…mobile visualization…geographically mapped spaces via GPS…thermal imaging…time-lapse /time monitoring of flows / stock portfolios … …in Mental Modeling Building Mental Models with Visuals for E-Learning

33. Image Maps DESCRIPTION: Spatial information, interactive, integration of text and images, conveyance of forms and distances, spatial relationships Tends to be informationally pre-determined and static, with designed interactive effects Building Mental Models with Visuals for E-Learning

34. Glyphs or Iconic Visualizations DESCRIPTION:A sculptured figure or relief carving; a font type as in an element of writing; a visual object that contains one or more data variables (coded in the shape, color, transparency, orientation, or other aspects of an icon) Often used in cartography (map-making), logic, semiotics (signs and symbols), and pictoral information systems (Ebert, Shaw, Zwa & Starr, 1996, p. 205) Building Mental Models with Visuals for E-Learning

35. Photomosaics DESCRIPTION: An arrangement of aerial or seabed photos that form a composite image; a visual effect in which an image is created of many smaller images Used for forensic analysis Building Mental Models with Visuals for E-Learning

36. Screen Captures / Screenshots DESCRIPTION: Realistic to the computer screen, annotatable; static (non-motion) and non-dynamic; dynamic (with motion); may have voice overlays Examples of interfaces Authentic at the moment of capture, usually not refreshed (as in websites) Building Mental Models with Visuals for E-Learning

37. Screencasts DESCRIPTION: Process-oriented, sequential, annotated, realia, voice narrated, multi-sensory Used to teach about how to use software programs or interfaces via desktop computers Captures of live synchronous interactive experiences, including voice, video, text, live annotation, and other features Used for virtual teaming meetings, classes, and live interactions Building Mental Models with Visuals for E-Learning

38. Fractals DESCRIPTION: 3D and 4D, geometric, elegant, relational, a kind of machine art based on mathematical formulas Shows relationships, trends Self-similarity in design (at least stochastically) Tends towards irregularity Is meaningful at both macro and micro levels Tends towards recursiveness Building Mental Models with Visuals for E-Learning

39. Photo-realistic Images DESCRIPTION: Digital photo captures and imagery May be microscope-enhanced, may be telescope-enhanced May originate from satellite, acoustical image gathering , sonograms, x-rays, CAT scans May be editable and enhanced, and digitally augmented Requires a sense of objective size and measure; requires a correct white balance May be mixed with overlays of annotation, drawing or other information, annotatable May be informational, illustrative , decorative, and others Building Mental Models with Visuals for E-Learning

40. Non-Photorealistic Images • Image morphing • Photo-mosaicing • Cartoon rendering from images • Computerized drawing and imaging; fictional avatars • Photogravure effects / intaglio printmaking; etching simulation • Machine art • Acoustic-created synced imagery • Digital sculpting • Theoretical modeling and visualizations (particularly in the sciences and arts) • Synthesized image overlays for information-rich experiences (usually with photo-realistic images or real spaces) Building Mental Models with Visuals for E-Learning

41. DigitalVideo DESCRIPTION: Involves color, movement and sound; realistic or fantastical; sequential or non-sequential; may be stylized; may include sound May be interactive if interspersed with Flash and other objects May be segmented for easier deployment Building Mental Models with Visuals for E-Learning

42. Avatars DESCRIPTION: Human or animal or symbolic shapes; playable characters May communicate in voice / sound and / or text May make decisions and actions in digital spaces Represent their animating players from the real-world Building Mental Models with Visuals for E-Learning

43. (Semi-)Intelligent Agents DESCRIPTIONS: Non-playable, automated characters; may be static or dynamic Programmed abilities, roles, emotions, beliefs, actions, intelligence and decision-making tendencies May play a direct pedagogical or instructional role May be a tutor May infuse a sense of telepresence into automated learning spaces Building Mental Models with Visuals for E-Learning

44. Flocking Group Behaviors DESCRIPTION: The automation of autonomous digital entity behaviors in coordinated motion, with or without individual agent guides, with agent attraction / repulsion; also swarming, schooling, herding, autonomous pedestrians and crowd behaviors; character or object motion simulation Members of a crowd as α-agents (alpha agents); may be inertial and pre-determined Basic C. Reynolds’ “boids” approach (1986): cohesion or flock centering (staying close with fellow agents); alignment (matching velocity or speed with α-agents), and separation (avoid collisions with nearby agents) Building Mental Models with Visuals for E-Learning

45. Live Data-feed Images DESCRIPTION: Remote sensor-fed, database-fed, representations often in spatial layouts, satellite feeds, and other types of multi-spectral / multi-source / multivariate integrated data Evolving and changing Real-time Potential suggested trend lines Macro and micro perspectives Building Mental Models with Visuals for E-Learning

46. Digital “WetLabs” DESCRIPTION: Process-based actions, causes-and-effects, human-mediated or remote labs or simulations, 3D computer simswith game-engine physics Narration of processes Building of context with facts Explanations of measures Clear definition of materials used Explanations of the processes and effects Explanations of negation—what the process is not showing Building Mental Models with Visuals for E-Learning

47. Machinima DESCRIPTION: Machine + cinema, captures of avatar interactions and 3D immersive digital environments and game spaces; pre-recorded; includes sound In-world digital effects May be performance-based or unscripted and unpracticed Building Mental Models with Visuals for E-Learning

48. Machine-Generated Art DESCRIPTION: Based on math formulas, evolutionary art, chance and other factors; tends towards fractals Synthetic art with unique vector imprints and “style” “Chaos tools,” “morphogenesis,” “cellular machines,” “neuronal co-evolution,” and non-photo-realistic techniques Visualization algorithms Perpetual Art Machine Building Mental Models with Visuals for E-Learning

49. 3D Immersive Spaces DESCRIPTION: Live, unpredictable, human-populated, automated and true serendipity The capturing of visual complexity (with multi-channel sensory information without cognitive overload) The highlighting of particular isosurfacesfor analysis Scene updates Building Mental Models with Visuals for E-Learning

50. High-Tech Image Editing Video ‘tooning (Wang, Xu, Shum & Cohen, 2004, pp. 574 – 583) or turning video into a “spatio-temporally coherent cartoon animation” Photo-realistic image to manga illustration; personalized image-to-cartoon stills Image relighting, event relighting Building Mental Models with Visuals for E-Learning