Virtual Vaudeville: An Interactive Historical Performance Simulation System

1. VIRTUALVAUDEVILLE David Z. SaltzPrincipal Investigator University of Georgia Coalition for Networked Information Task Force Meeting April 16, 2004 NSF 0121764 2002-2004

2. The Problem: • How to represent historical performances? • For teaching • For documentation of research • For hypothesis testing

3. Our Solution: • Live Performance Simulation System • Model theatrical spaces • Animate performances • Interact with community of spectators • Prototype Application: Virtual Vaudeville • Performance in New York City, 1895 • Generalizable System • Applicable to almost any performance event • Configurable software engine • Develop re-useable strategies

4. Interdisciplinary Collaboration • Computer Animation and Motion Capture (UGA) • Programming and Interface (Georgia Tech) • Agent-based Engine for Spectator Interactions(Naval Postgraduate School) • Historians • Cultural history (U. Pittsburgh), Vaudeville acts (Charleston College, UGA), Musical history (Metropolitan College), Architecture/Design (Appalachian College) • Theatre production (UGA) • Playwriting (Studio Z), directing, chorography, costumes, sets, lighting, acting, musical director, musicians

5. Why 19th Century Vaudeville? • Enormous force in popular culture; the television of late 19th and early 20th century America. • Window into 19th century American society. • Abundant but under-researched archival material. • Consisted of many short, self-contained segments.

6. Time and Venue: • B.F. Keith’s Union Square Theatre, New York City, 1895

7. Theatre Details

8. Acts: • Sandow the Magnificent, strongman Frank Bush,ethnic comic

9. Researching Acts To create the models and reconstruct scripts for the acts, our historians have searched archives around the country for programs, newspaper descriptions, photographs, old films and early wax cylinder recordings of the performers.

10. Frank Bush as Jewish character Frank Bush as Irish character The 3D model of Sandow is based on measurements taken of Sandow in 1894 by Dr. Dudley Allen Sargent of Harvard University. Modelling Historical Performers

11. David Spearman, one of three motion capture performers for Sandow. George Contini recording audio and facial movements for Frank Bush. Motion and Facial Capture

12. Animating the Acts

13. Animating the Acts

14. Placing Acts in the Theatre • 800 spectators in real-time environment • Audience demographics • Gender • Class • Ethnicity

15. Animating the Audience • 32 Spectator Groups • Groups react differently • 3-5 different faces and costumes per group • Additional variations in hats and facial hair

16. Conceptual Challenges • Presenting speculations as reality • Imperative to “fill in” all details both the strength and weakness of simulation as method. • Reinforcing positivist assumptions • Reducing history to objectively accessible artifacts • Disneyification • Presenting idealized view of past (simulacra vs. simulation) • Perpetuating Historical Stereotypes and Prejudices • Controversy about including blackface

17. Multiple-Mode Approach Balance immersion & critical perspective. At any time viewer can switch between: • Invisible Camera Mode • Hypermedia Notes • Avatar Mode

18. Invisible Camera Mode • Users "read" the simulated performance • Freeze action, rewind, fast-forward • User-driven camera • Jump to landmarks (spatial bookmarks) • Click on a spectator to jump into any seat • Pre-animated cameras • Close, medium and far cameras following performers • Performer perspective • Spectator “inner monologues” • More powerful than being in a real theatre… … but risks reinforcing positivism

19. Avatar Mode • Pre-defined avatar groups • Mother/Daughter, “Sport”/Showgirl, etc. • User controls avatar’s behavior • Applaud, laugh, frown, “heckle,” “ask,” etc. • Surrounding spectators are autonomous agents

20. Hypermedia Notes • Context-sensitive Annotations • Attached to objects • Attached to time-line • Introductory essays • Critical analyses • Archival evidence • Photographs • Scripts • Programs, reviews • Recordings, film

21. Multiple Platforms • Problem: Performance • Need to navigate highly detailed environment with hundreds of animated models in real time. • But need to be widely accessible • Solution: Two Versions • Gamebryo • High performance game engine • Requires high-end gaming PC • Shockwave/Quicktime Version • Cross Platform • Minimal system requirements

22. Thinking Ahead • Problem: Obsolescence & Sustainability • Solution: All content is external to Game Engine • Models, animations, hypermedia notes all in standard formats stored as external files. • Spectator interaction engine is an object that functions independently of game engine and animations (“black box” that sends and receives simple text messages) • Textures for models are created in multiple resolutions; the system uses the highest resolution the hardware allows.

23. Adaptability • Problem • Goal is to develop a Live Performance Simulation System, not a one-off simulation of Vaudeville. • Solution: Modular design • Game Engine Licensed for “Live Performance Simulation System,” not Virtual Vaudeville per se. • No programming required to change models or define spectator interactions. • Hypermedia engine is standard html browser.

24. Conclusion • Technical significance • Scholarly significance • Pedagogical significance • Artistic significance