Understanding Visual Media

1. Prof. Marc Davis, Prof. Peter Lyman, and danah boyd UC Berkeley SIMS Tuesday and Thursday 2:00 pm – 3:30 pm Spring 2005 http://www.sims.berkeley.edu/academics/courses/is146/s05/ Understanding Visual Media IS146: Foundations of New Media

2. Lecture Overview • Review of Last Time • History and Technology of Digital Imaging • Today • Understanding Visual Media • Preview of Next Time • Case Study: Cameraphone

3. Lecture Overview • Review of Last Time • History and Technology of Digital Imaging • Today • Understanding Visual Media • Preview of Next Time • Case Study: Cameraphone

4. Image reading • Visual communication is always coded • “It seems transparent only because we know the code already, at least passively — but without knowing what it is we know, without having the means for talking about what it is we do when we read an image.” • Our culture is moving from textual to visual • “Until now, language, especially written language, was the most highly valued, the most frequently analyzed, the most prescriptively taught and the most meticulously policed code in our society.” • Visual “literacy” is not taught and needs to be • “If schools are to equip students adequately for the new semiotic order, if they are not to produce people unable to use the 'new writing' actively and effectively, then the old boundaries between 'writing' on the one hand, traditionally the form of literacy without which people cannot adequately function as citizens, and, on the other hand, the 'visual arts', a marginal subject for the specially gifted, and 'technical drawing', a technical subject with limited and specialized application, should be redrawn.”

5. Image technology • development of technology (e.g. camera, film stock, etc.) • development of presentation techniques (e.g. illumination) • development of technology (e.g. camera, compression, manipulation) • development of presentation techniques (e.g. automatic collage) • development of technology (e.g. colour, brushes, canvas, etc.) • development of presentation techniques (e.g. central perspective) Digital image (1950) Painting VERMEER VAN DELFT, c. 1665,Oil on canvas, 46,5 x 40 cm, Mauritshuis, The Hague Photo (1830)

6. The camera Analog camera produces a continuous image, simulating the capture of the eye. Negative (reverse reality) Actual image (various manipulation processes) Analog technology is expensive and complicated to handle.

7. The digital image The image is a matrix of values, representing the colour, texture, dimension, etc.

8. The digital image - size File size for True Colour 640 x 480 x 3 = 0,87 MB 800 x 600 x 3 = 1,37 MB 1024 x 768 x 3 = 2,25 MB 1280 x 1024 x 3 = 3,75 MB 1 Bit -> black and white 8 Bit -> greyscale 16 Bit -> 64.000 colours (High Colour) 24 Bit -> 16 Mil. colours (True Colour)

9. The digital image - manipulation Pixel or vector • Image format • Resolution • Mode • File type • Image enhancement • resize • colour enhancement • artistic filters • establishing planes • any other sort of manipulation ALWAYS ask: what do I intend to do with the image now AND in the future?

10. Lecture Overview • Review of Last Time • History and Technology of Digital Imaging • Today • Understanding Visual Media • Preview of Next Time • Case Study: Cameraphone

11. Understanding Comics • Scott McCloud - comic artist • Explains visual communication through comics • Explains comics through visual communication • "If you've ever felt bad about wasting your life reading comics, then check out Scott McCloud's classic book immediately. You might still feel you've wasted your life, but you'll know why, and you'll be proud.” - Matt Groening, creator of The Simpsons.

12. What Are Comics? • “Juxtaposed pictorial and other images in deliberate sequence, intended to convey information and/or to produce an aesthetic response in the viewer.” (p. 9) • How do comics differ from • Photographs? • Movies? • Hieroglyphics? • Emoticons?

13. Why Understand Comics?

14. Old Comics: Trajan’s Column

15. Old Comics: Mayan Codex Nuttall

16. Old Comics: Tortures of St. Erasmus

17. Scott McCloud’s “Big Triangle” Picture Plane Reality Language

18. Scott McCloud’s “Big Triangle” Picture Plane Reality Language The lower left corner is visual resemblance (e.g., photography and realistic painting).

19. Scott McCloud’s “Big Triangle” Picture Plane Reality Language The lower right includes the products of what he calls iconic abstraction (e.g., cartooning).

20. Scott McCloud’s “Big Triangle” Picture Plane Reality Language And at the top are the denizens of the picture plane (“pure” abstraction) which cease to make reference to any visual phenomena other than themselves.

21. Scott McCloud’s “Big Triangle” Picture Plane Reality Language The move from realism to cartoons along the bottom edge is a move away from resemblance that still retains “meaning,” so words, the next logical step in the progression, are included at far right, thereby enclosing anything in comics’ visual vocabulary between the three points.

22. Scott McCloud’s “Big Triangle” Picture Plane Reality Language McCloud found that “The Big Triangle” as it came to be known, was an interesting tool for thinking about comics art...

23. Scott McCloud’s “Big Triangle” Picture Plane Reality Language ...and for visual art and language in general!

24. Cartoons and Viewer Identification

25. Closure: From Parts To The Whole

26. Closure: Bridging Time and Space

27. Closure in Comics

28. Closure: The “Gutter” in Comics

29. Closure: Moment-To-Moment

30. Closure: Action-To-Action

31. Closure: Subject-To-Subject

32. Closure: Scene-To-Scene

33. Closure: Aspect-To-Aspect

34. Closure: Non-Sequitur

35. Questions for Today • How do we interpret images and sequences of images? • How do we read different visual representations of the world (especially different levels of realism and abstraction) differently? • How does what is left out affect how we understand images and sequences of images?

36. Questions for Today • What are some of the differences between how text and images function in comics? • What would be lost/gained in moving between images and text?

37. Questions for Today • How could we represent images and sequences of images in order to make them programmable? • What could computation do to affect how we produce, manipulate, reuse, and understand images and sequences of images?

38. Lecture Overview • Review of Last Time • History and Technology of Digital Imaging • Today • Understanding Visual Media • Preview of Next Time • Case Study: Cameraphone

39. Readings for Next Time • Marc Davis, Simon King, Nathan Good, and Risto Sarvas. From Context to Content: Leveraging Context to Infer Media Metadata. in: Proceedings of 12th Annual ACM International Conference on Multimedia (MM2004). New York: ACM Press, p. 188-195, 2004. • Discussion Questions • Nancy A. Van House, Marc Davis, Morgan Ames, Megan Finn, and Vijay Viswanathan. The Uses of Personal Networked Digital Imaging: An Empirical Study of Cameraphone Photos and Sharing. in: Extended Abstracts of the ACM Conference on Computer-Human Interaction (CHI2005). New York: ACM Press, Forthcoming 2005. • Discussion Questions