WLCD: A Web Lecture Capture System with Robotic Speaker Tracking

1. WLCD: A Web Lecture Capture System with Robotic Speaker Tracking Steven Goldfarb for Jeremy Herr CHEP Mumbai, India - 15 Feb 2006

2. Web Lecture Capture Device The Web Lecture Archive Project • Partnership • University of Michigan • UM ATLAS Collaboratory Project • Department of Physics • Media Union • CERN • Technical & Acedemic Training • Summer Student Program • IT • American Physical Society • Primary Goals • implement an electronic archival system for slide-based presentations on the Internet; • assess the ability of the system to improve upon and complement existing archival methods; • determine the resources required to install and maintain the system as a standard archival service. J. Herr - University of Michigan

3. Current Recording Activities • Web Lecture Recording • ATLAS & LHC In General • Physics & Software Tutorials • ATLAS Week plenary sessions • CERN Training Seminars, Tutorials • American Physical Society • Opportunities for Physics in Biology 2002, 2004 • April Meeting 2004 • Fermilab • Software Tutorials • University of Michigan • Saturday Morning Physics • Medical School Symposium on Clinical Research 2004 • Harvard • International Conference for Systems Biology 2005 J. Herr - University of Michigan

4. Development of Automated Recording Technology • Motivations Driving Automation • Simultaneous Recording of Multiple Lectures • Limited Manpower • Need for Instant Publication • Components of Lecture Archiving Requiring Automation • Video Capture and Compression • Audio Recording • Slide Capture and Synchronization • Construction, Publication of Electronic Lecture J. Herr - University of Michigan

5. Development of Automated Recording Technology (2) • Brief History of Lecture Recording Automation • 1999: Used Sync-O-Matic 3000 Software to Record CERN Summer School Lectures • Sync-O-Matic completely automatic if • Installed on presentation computer • Slides in Powerpoint • But • Everybody wrote plastic transparencies back then! • And access to presentation computers not always possible • So • Had to record timing by hand, process lectures after the talk. • 2000: Extra video camera was pointed at the screen • timing information was collected manually afterward • 2001: Giosue Vitaglione wrote CarpePPT • Same idea as Sync-O-Matic, but producing Lecture Objects (non-proorietary) • 2002: Jake Bourjaily wrote CarpeLecture • Extension, allowing PDF and integrated with CDS Agenda • Now: Web Lecture Capture Device • VGA splitter connected to signal coming from presentation computer • high-resolution capture card is used with change-detection software • captures every image displayed on the speaker’s computer • No limit to format (PowerPoint, PDF, Web pages, command line prompts,…) J. Herr - University of Michigan

6. Recall the Lecture Object (Talk before Coffee) • Lecture Object • Standardize Storage of Web Lecture Ingredients, Recipe • Archive media in standard formats (mpeg-4, jpeg, etc.) • XML extension to SMIL (Synchronized Multimedia Integration Language) • Describes slide timing, media location, other information • SMIL = W3C Standard • Add metadata with RDF (Resource Description Framework) • RDF = W3C Standard • Standard Procedure for WLAP Transformations meta data XML Description mpeg-4 jpeg Lecture Object J. Herr - University of Michigan

7. Lecture Object - Definition • Organized Directory Structure and Naming System • Contents • Audio/Video media: MPEG-4 video file • Slides: JPEG images of everything displayed on lecturer’s screen • Supporting Documents: original PowerPoint file, animations, etc. • Log Files: describing technical history of and changes to media files • XML file: • a simple format similar to SMIL, following commonly used international standards and customized to effectively describe a typical lecture. • contains metadata fields using subsets of Dublin Core and IEEE Learning Object Metadata, describing media data • timing and synchronization information • Future Directions • Allow more media streams (animations, data displays, chalkboards) • Allow annotations, notes, links to other resources • Transformations to more viewing objects: Quicktime, Media Player, iPod, etc. J. Herr - University of Michigan

8. Lecture Object - Example • Example of Lecture Object using current standard J. Herr - University of Michigan

9. Robotic Camera Tracking • System Requirements • Portable • Robust • Affordable • Runs without expert intervention • Little setup or calibration • Survey of Existing Technologies J. Herr - University of Michigan

10. Robotic Camera Tracking – Current System • Current solution: “active” IR system using dual-cameras and infrared necklace • Target wears necklace made of IR LEDs • Invisible to human eye, but very bright in IR • Two cameras mounted together • IR camera tracks target • visible-light camera provides color video • This system satisfies our criteria • Portable: sits on a cart • Robust: simple design makes it very robust • Affordable: currently under 4,000 USD • No expert intervention: start it and it just works • Little setup: almost no calibration required • Accurate to within centimeters J. Herr - University of Michigan

11. Robotic Camera Tracking – Future Development • Remaining Challenges • Rooms with windows: on bright days, excessive sunlight can increase background noise and illuminate reflective objects • Other IR sources • Tablet PC’s use bright IR pens • incandescent light fixtures give off a lot of IR • transparency projectors give off a lot of IR • Improvements - Next system: machine-vision camera • Stationary, wide-angle camera • Frame rate, exposure time, timing are all controllable • The necklace strobes • identifies necklace uniquely to the machine vision camera • saves battery life • We are building and testing this system now • This should greatly improve reliability in adverse conditions J. Herr - University of Michigan

12. Technology Transfer • March 2005 • Provisional patent filed, describing the current dual-camera with active IR necklace. • October 2005 • University of Michigan researchers filed 287 invention disclosures in 2005 • Eight of these were selected to be featured at the 2005 Tech Transfer Inventor’s Reception, as the most commercially promising and world-changing. • Our tracking system was one of these eight. • March 2006 • Permanent patent will be filed. J. Herr - University of Michigan

13. Applications • Plans for Using the WLCD • Enable automated large-scale recording, streaming and archiving of university courses • Record and disseminate multiple parallel sessions at large professional society meetings • Facilitate the preparation of materials for distance learning • Generate supplemental instructional content for outside-of-class review • Provide services for recording of sample lectures given by faculty candidates • Recording of ATLAS Plenary Sessions • Many more ... J. Herr - University of Michigan

14. Hot Item: U-M Campus-Wide Classroom Recording • Now - Physics 140 test • Big impact: over 500 students each semester, including pre-medical, engineers, science majors. • Recording with current tracking system will begin March 2005 • Two 1-hour lectures per week, each available through U-M CourseTools within hours, and also podcast to student’s iPods. • Chalkboard work will be captured. • Entire system fits on one cart. • Fall 2006 – Pilot Project begins • Project has support of the Provost and many academic units • Three different classes will be recorded • Student usage will be analyzed through surveys and log monitoring by education experts in the Center for Research on Learning and Teaching (CRLT). • Project will continue through Summer 2006 J. Herr - University of Michigan

15. Example Videos • Demo video made with tracking system in May 2005 • Tracking camera video of Jen McCormick made in December 2005 • Web Lecture for iPod - David Gerdes, UM Physics Professor J. Herr - University of Michigan