1 / 11

Streaming Architectures for Very Large Collections

Streaming Architectures for Very Large Collections. Jim DeRoest Director Streaming Media Technologies, ResearchChannel Assistant Director, University Computing Services University of Washington deroest@cac.washington.edu 206 543-6343. Who We Are And Support.

deepak
Télécharger la présentation

Streaming Architectures for Very Large Collections

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Streaming Architectures for Very Large Collections Jim DeRoest Director Streaming Media Technologies, ResearchChannel Assistant Director, University Computing Services University of Washington deroest@cac.washington.edu 206 543-6343

  2. Who We Are And Support • ResearchChannel/UWTV - Exploring digital convergence and educational outreach. • National Echostar and CATV to over 5 million homes • International VoD and live streams – I2 Abilene • Experiments - First cinema quality HD over IP • KEXP – UW and EMP collaboration exploring digital radio • Uncompressed 1.4Mb live stream • Real time web playlist generation • World music format – DJ choice • Faculty, Staff, Student, Clinical, K20 • Other Affiliate/Collaborative Projects

  3. Managing Content • Large Collection Problem • Disparate Asset Types • HD, Broadcast TV, audio libraries, pictures, rich text • Instantiation: capture, encoding, metadata, storage • Multiple sources, formats, schema • Interfaces • TV post production, radio DJ, viewer/listener • Discovery and Administration • Naming, categorization, relations, internationalization • Distributed management, sharing, searching • Access Controls • Authentication, authorization, rights, privacy, security

  4. MediaWeb Project • 18 month technology/requirements review • Requirements • Interviewed engineers, DJ’s, librarians and consumers • Reviewed vendor products • Proprietary, 70% solution, consumer specific • Created prototype during review process • Pluggable, scalable, standards based architecture • Buy, build, both, call it quits? • Decision to go with prototype

  5. Transcoding Metadata Creation, Admin TV Broadcast Services Radio DJ Services UW MediaWeb– Digital Media Asset Management System Asset Management Services Remote Stream Manager Media Store Disk Cache Request Director Robotic Tape Asset Metadata Delivery Services Image, Quicktime, Windows, Real, HD, Webcasting …

  6. Core Services • Media Store • HSM online/nearline multi-vendor High Performance Storage System (HPSS) • Gigabit interconnects. • Distributed, redundant, recoverable • Block I/O stream to servers (smoke and mirrors • Metadata Service • SQL, directory, imbedded objects, access control, rights • Stream Manager • Maintains asset locality state and aging • Request Re-Director • Re-direct requests to target stream or application service

  7. Feeds • Video • Airtime/Dubtime to 50Mb MPEG2 i-frame base • Louth/Mav triggers/APIs • Metadata generation • Looking at Virage plugins • Audio • CD ripping raw audio base • Metadata map CDDB sites (Muse, AMG, …) • DJ metadata support Transcode from base formats as needed

  8. Metadata • Schema – How rich can we afford • Consumer specific • Standards – Dublin Core, MPEG7 • Vendor (Muse, AMG, Louth playlists, …) • Draft submitted for comment I2-VidMid VoD and ViDe • Access/Rights controls • Vendor specific, MPEG21 • Auth*2 – NetID Pubcookie, Shibboleth, … • Imbedded – Honored by clients • Trust • Metadata Objects • SQL (base), LDAP, Imbedded objects/methods

  9. Delivery • Streaming Servers • Standard file system or IO API required Stream initiated as first blocks arrive to local cache Tape, disk, OS and network speed vs. hybrid file systems • Selectable stream stub support First +30 seconds of clip hide mount time content on tape • Low bit rate (no MPEG1) Quicktime, Real, Windows Media • High bit rate Windows Media – KEXP 1.4Mb live stream Optivision – RC 5.6Mb live stream to remote sites MPEG2 – Concurrent Computer Xstreme 5.6Mb VoD MPEG4 – Looking at IBM Videocharger – interface issues HD – In-house (>270Mb), other vendor lower bit rates

  10. MediaWeb Project Status • Prototype operational with simple search • Geographically separated/mirrored server environment • Video: RC/UWTV content (~700 hrs MPEG2, lower bit rates) • Audio: Sample audio • Refining schema • Object experiments • Imbedded controls (MPEG7) • Reviewing large vendor schemas • Deploying admin tools • Building feed systems (RC, +35,000 disc KEXP library) • Incorporating faculty/student mgt (expiration, quotas, …) • Evaluating other large content requests • Ongoing evaluation, scaling, benchmarking architecture • Looking for opportunities to collaborate!

  11. Experiments • UW CSci animation teaching tool “How Networks Change Teaching: A Case Study” Louis Fox, Kris Martin, Barbara Mones-Hattal I2 VIMM Friday, October 5th 1:30EDT • KEXP live stream archives • KEXP automated playlist • EMP live webcasting and archival • UW Catalyst collection interface • HD IP Videoconferencing

More Related