Abstract Media Spaces

1. Abstract Media Spaces Rob Diaz-Marino CPSC 781 University of Calgary 2005

2. Outline • What is Abstraction? • Simple Media Spaces • Abstract Media Spaces (AMS) • Benefits and Drawbacks • Methods for Abstracting Media • Designing an AMS • Summary

3. What is Abstraction? “Guernica” by Pablo Picasso Reaction to the bombing of Guernica in World War II.

4. What is Abstraction? (2) • Throwing away information [1] Abstract Realistic Photo Cartoon Art Piece • Female • Blond Hair • Yelling • Emotion? • Female • Brown Hair • Yelling / Angry • Female • Yelling / Distressed • In a barn/cellar Fact • Image cropped • Unclear context • Part not drawn • No context • Lips, eyebrows, • fingers, mouth, • hair… • Eyes on same side • of head • -Eyes in wrong place • Distorted body shape • Hair on back of head • Blue-gray color Abstraction

5. Transmission Video-In Video-Out Transmission Picture-In Picture-Out Transmission Audio-In Audio-Out Simple Media Spaces …and so on.

6. Simple Media Spaces (2) • Literal Transmission • Input = Output • Low degree of abstraction • Some loss from original (reality) • Capture limitations • Compression – bandwidth limitations • Still perceptually equivalent

7. Why use Abstraction in a MS? Bob is at his computer Bob is not wearing clothes Bob is has a fruit-basket hat on his head Bob is yelling at his girlfriend on the phone Bob looks angry • To control information • Preserve Privacy • Shield sensitive details • Reduce Distraction • Eliminate unnecessary details • Re-map awareness cues • Reduce bandwidth needs Abstraction [1] Someone is at the computer The person is mostly flesh-colored The person has something large on their head The person is speaking loudly No details can be seen on their face

8. Drawbacks of Abstraction Bob is at his computer Bob is not wearing clothes Bob is has a fruit-basket hat on his head Bob is yelling at his girlfriend on the phone Bob looks angry • Loss of information • Useful: Identity, Actions, Availability, etc. • Incidental: Details, Emotional state, etc. • Loss of understanding • Unclear meaning • Unclear context Abstraction Someone is at the computer The person is mostly flesh-colored The person has something large on their head The person is speaking loudly No details can be seen on their face

9. Methods of Abstraction • Simple Degradation • Feature Extraction (Silhouetting)

10. Simple Degradation • Video • Still resembles original • Ex. Mike Boyle’s Video Filters [4] Blur Filter Video-Out Pixelation Video-In Video-Out

11. Simple Degradation (2) • Audio • Still resembles original Echo Audio Out Audio-In Muffling Audio Out

12. Feature Extraction • Video • Vision Techniques • Motion detection • Presence detection • Eye tracking • Face tracking • …

13. Feature Extraction (2) • Audio • Ex. Smith et al’s Low Disturbance Audio [3] • Speech  Non-Speech • Can still recognize voice • Cannot understand words • Similar to Blur Filter but for Audio! Extract Synthesize Audio-In Audio Out

14. Feature Extraction (3) • Text • Ex. Syllable replacement Jim: Hi! How are you doing? Bob: Doing okay… Jim: Are you busy? Bob: I’m on the friggin’ phone!! Jim: Oh, sorry! Extract Jim: Bla! Bla bla bla blabla? Bob: Blabla blabla… Jim: Bla bla blabla? Bob: Bla bla bla blabla bla!! Jim: Bla, blabla!

15. Media Translation • Convert one media form to another • No direct translation • Feature Extraction • Synthesizer – Visualization, Sonification, etc. Extract Synthesize Audio-Out Video-In

16. Media Translation (2) • Ex. AROMA [1] • Peripheral awareness

17. Media Translation (3) • Ex. Cambience (my thesis project) • Inputs • Web Cam (video) • Feature extraction • Motion detection, partitioning, thresholding, etc. • Outputs • Audio – volume, pan, etc.

18. Translation Pitfalls • Extreme abstraction • No longer understandable • Usable only as art piece • Learning Curve • Arbitrary mappings • Users may need to see literal data [1]

19. Designing an AMS • Processing • Must be done in REAL TIME • Can lower sampling rate to compensate • Peripheral vs. Foreground • Draw Inspiration • Ambient Displays • Visualizations

20. Designing an AMS (2) • 3 Architectures • Client-side processing • Server-side processing • Distributed processing

21. Server Client Output Extract & Synth Input Output Extract & Synth Extract & Synth Output AMS Architectures (1) • Client-Side Processing • Transmit raw data – privacy risk! • High Bandwidth usage • Low CPU for Server, high for Clients

22. Server Client Output Input Output Extract & Synth Output AMS Architectures (2) • Server-Side Processing • Transmit synthesized media • High Bandwidth usage • High CPU for Server, low for Clients

23. Server Client Output Synthesize Input Output Extract Synthesize Synthesize Output AMS Architectures (3) • Distributed Processing • Transmit extracted features • Lower Bandwidth usage • Lower CPU for Clients and Server

24. Designing an AMS (3) • Ex. Cambience • Video Input • Feature extraction on Server • Transmission • Scalar values • Audio Output • Sound synthesis on Client

25. Summary • Abstract Media Spaces • Throw away information • Simple Degradation • Feature Extraction • Can provide a privacy shield • Can provide better peripheral awareness • Allow media re-mapping • Can lower bandwidth usage

26. Questions?

27. References • Pedersen, E. R., Sokoler, T. (1997) AROMA: Abstract Representation of presence supporting Mutual Awareness. Proceedings of CHI’97, 51-58. • Wikipedia: The Free Encyclopedia. (n.d.) Retrieved October 2005 from http://en.wikipedia.org/wiki/Abstract_art, http://en.wikipedia.org/wiki/Pablo_Picasso,http://en.wikipedia.org/wiki/Cubism • Smith, I., Hudson, S. (1995) Low Disturbance Audio For Awareness and Privacy in Media Space Applications. In proceedings of ACM Multimedia ’95, ACM Press, p. 91-97 • Boyle, M. (2005) Privacy in Media Spaces. PhD Thesis, Department of Computer Science, University of Calgary, Calgary, Alberta CANADA T2N 1N4. April.