Streaming technologies: Under the hood

1. Streaming technologies:Under the hood Egon Verharen Innovatie management SURFnet bv Egon.Verharen@SURFnet.NL

2. Contents • Digital Video family • Similarities en Differences • What is streaming and why streaming media? • Technology • what do you need • formats, standards, server, clients, network • demo • (forseable) Future • Lessons learned Content Management

3. Conferencing • tools Collaborative tools webbased virtual classroom live event streaming • Multimedia • archives Pieces of the puzzle • Why digital video in education

4. Digital Video: Similarities and Differences -Network video Video Broadcasting Streaming/Webcasting Like watching Television One - Way Tune In or enter URL typical bitrates from 28 kb/s - 8 mb/s Video format (H.261, H.263, MPEG) transport protocol (RTP) fast network connection QoS (jitter, synchronisation) Video Conferencing Video chat Like a telephone call Two - Way Call up or Answer typical bitrates from 56 kb/s - 1,5 mb/s Video format (H.261, H.263, MPEG) transport protocol (RTP) fast network connection QoS (min. delay, min. jitter, synchronisation

5. What & Why Streaming Media content • “Streaming” is buzz-word. • an adjective • Many definitions/definotions • Push vs. Pull • Client-oriented vs. server-oriented • on-demand vs. live • Streaming media: the ‘real-time’ playing of a video-, audio- and/or datastream on a machine from the moment the first bytes come in. IT TelCo

6. Streaming: live vs on-demand archive realtime Event-driven unicast VoD Event-driven multicast scheduled

7. Why streaming? (download times) Bandwidth transport time (650 MB) 2400 bit/s 631 hrs (26 days and 7 hrs) 9600 bit/s 158 hrs (6 days and 14 hrs) 2,5 Mbit/s 34 min 10 Mbit/s 8 min and 40 sec 34 Mbit/s 2 min and 33 sec 155 Mbit/s 32 sec 1 Gbit/s 5 sec 1 Gbit/s = 1.000 Mbit/s = 1.000.000 kbit/s = 1.000.000.000 bit/s

8. Streaming media advantages • data does not need to be downloaded; • high bitrate make high (video)quality possible; • you can jump ahead (or back) in stream (‘vcr principle’); • you can broadcast/netcast live; • suitable for tele-educatie, • but also information delivery.

9. Digitale Video QoS myth “DV cannot be implemented without QoS technology (which is new, complex, expensive and over the horizon)” • NO, you can start now ! • QoS for end-users = bigger picture, better audio. • QoS for network = delay, packet loss, jitter • Requirements differ per application: • interactivity vs streaming, minimal delay/jitter ? • Quality is function of many parameters • client, LAN, campus, access links to PoP, within PoP, natl & int.l. networks

10. Quotes & Provocations by Dennis Glenn, NWU • “Tape is dead” • “Delivery is king” • “Quality is not important” • quality of video • QoS • “Exposure is everything” • Content is worth nothing if you don’t show it

11. Digital Content management: driving forces Analog Digital LARGE archives Everywhere and always access Create once, distribute/use many Save and secure for (re)use Costs Value

12. Developments in the content chain Program maker Website owner Author Internet host Broadcast organisation Publisher Internet provider Channel Telecom operator Cable- operator Distributer Users/Viewer

13. Merging of the chains A/V/D-material Format conversion classification logical ordering Contentmanager Distribution media End-users

14. Video-over-IP value chain Media production Content use Content management $$ $ 1011011 Content deployment (content distribution/ content delivery) Encoding

15. Workflow TV “broadcast” via internet • “Dynamic” services example: DelayTV Staging NED1 Online Encoding Export Installatie Videoserver Online Validatie Batching Spotting Enhancing Installatie DB/Webserver NOS A DatabaseA disk2tape Mass Archive 30 hrs encoded video plus metadata each day! 5 min to 3 hrs Content management cycle By Ton van Mil, director NOB-interactive

16. Workflow TV “broadcast” via internet • “Dynamic” services example: News Staging NED1 Online Encoding Export Installatie Videoserver Online Validatie Batching Spotting Enhancing Installatie DB/Webserver NOS Programma Database disk2tape Mass Archive NOS Newsroom disk2CD Archief NOB graphics 3 hrs Content management cycle By Ton van Mil, director NOB-interactive

17. VIP platform components Reversed content provision User metadata provision Basic Content production Basic deployment Accounting server Advanced deployment Content management Advanced Content production Metadata MPEG7 End user

18. Streaming media service • content creation • producers, editors, encoders • content mgt • db mgt, META-DATA, indexing&searching • content delivery • dynamic streaming (more than one bandwidth), license control, synchr. Multimedia, • scalability (caching & replication) • content viewing • softw based players, MPEG2 hw based players, browser plugin and external helper apps.

19. Technology • standards, formats • still new codecs (e.g. MPEG4, IBMs LBR) • streaming servers and clients • network

20. Formats • Aspects: • Bitrates (constant, variable) • Framesize • QoS • Bandwidth (at end-user) • dynamic use: Real SureStream, simulcast • Standards • ISO/IEC (int.l. org for standardization/int.l. electrotechnical commission) • Moving Pictures Experts Group (ISO/IEC JTC1/SC29/WG11) • MPEG1,2,4,7 • ITU (int.l. telecommunication union) • Series H (video), Series G (audio) (ITU-T SG16)

21. Formats (cont.: MPEG) • MPEG1 1,5 mb/s; VHS; part3=audio=MP3 • MPEG2 6-8 mb/s; digitale TV • MPEG4 lower bitrate (better compr.); multimedia appl. layers (transparent info), content protection, QoS • MPEG7 metadata model ‘on top of’ MPEG1/2/4; searching, filtering, processes

22. Formats (cont.: Video en Audio) • Video • ITU-T Series H - Audiovisual and multimedia systems • H.261 H.263 • DV (Digital Video) • Audio • MP3 = MPEG-1 audio layer 3 • Series G - Transmission systems and media, digital systems and networks • G711, 722, 723.1, 726, 728, 729

23. Protocols • Streaming Media Protocols are protocols that are being used to support the real-time delivery of multimedia data • TCP vs. UDP • HTTP • progressive download • RTP • real time protocol • UDP • payload

24. Protocols (cont.: RTSP) • RTSP • Real Time Streaming Protocol • designed to be an open standard aims to provide an extensible framework to enable controlled delivery of real-time data, such as audio and video. • Denk: videocontrole (play, pauze, stop, back, forward) • http://www.ietf.org/rfc/rfc2326.txt

25. Distribution • Scenario’s • Broadcast vs. Groupcast vs. Personcast (Video-on-demand) • Technology • Unicast • Multicast • cookbook

26. What do you need CONTENT • Streaming server • on-demand/live; program manager; asset mgt • data pump • encoders (can be separate) • Streaming client • Fast PC • decoder (up to MPEG1: software, MPEG2 prefer. hardware) • monitor • Network connection • MPEG-1 (VHS-qual/CD):  2 Mbit/s guaranteed end-to-end ! • UDP/IP • multicast • Connectivity end-user determintes format (bitrate)

27. Streaming servers & Clients • Architectures • Real • Windows Media • Quicktime • “High-end” • Kasenna Mediabase, Oracle Videoserver • (not yet) Broadcast servers ! • Others • FVC I-studio, IBM Videocharger, Cisco IP/TV, … • Clients • almost all dedicated ! • Not (yet) interoperable

28. Architectures • RealMedia • codecs: RealVideo, RealAudio • streaming formaat: RealVideo, RealAudio, met plugins (bijv. BitCasting) MPEG1, MP3 • SureStream: multiple bitrate encodings, bandwidth negotiation • RealText, RealPix • SMIL (Synchronized Multimedia Integration Language) • Standards: RTSP, RTP (RDP)

29. Architectures (cont.) • Apple Quicktime • supports many codecs • streaming RTP payloads; Sorensen: VBR, CBR • streaming format: Quicktime (hinted movies) • plug-in architecture • tracks & effects & interactivity • standards: RTSP, RTP

30. Architectures (cont.) • Microsoft Windows Media Technologies • NetShow, DirectShow/ActiveMovie, Video for Windows • codecs: Windows Media Audio (WMA), MPEG-4 v3, ACELP (voice) • streaming format: ASF (advanced streaming format) • Multi-Datarate Encoding, Intelligent Transmission, Media rights • standards: ??; WMM • office integration !! (e.g. streaming powerpoint) • Most ‘compatible’ client (player)

31. Codecs • Codecs • MPEG1, MPEG2, MPEG4 • MP3 = MPEG-1 audio layer 3 • Qualcomm PureVoice • Qdesign Music (stereo audio for dialup data rates en fast processor) • a2b (AAC, high fidelity, faster data rates, encryption) • liquid audio (arch., AAC, hifi, encryption, licensing, watermarking) • java based: emblaze, GTS • Sorensen (video, both dialup and cable modem data rates) • Vivo (VIV), Vxtreme (VXF), VDOnet • http://www.terran.com/CodecCentral/

32. Servers • Aspects • storage • media pump • network (bandwidth) • on-demand vs. live • asset management ! • Examples (not complete !!) • Architectures • Apple Quicktime, Real G2, Microsoft Windows Media • “high end” IP streamers (live/on-demand) • Cisco IP/TV, IBM Videocharger, Kasenne Mediabase, Oracle Videoserver • broadcast servers • Sony,Philips, SUN, ..., AXCENT INSERT • frameworks • SUN & IBM Java Media Framework

33. Servers (cont.: Examples) • “high end” • FVC.COM I-Studio • Cisco IP/TV • standards support • IBM Videocharger • Digital Library; architectuer; interactivity • standards support • Kasenna (formerly SGI) Mediabase • Ncube/Oracle MediaCUBE/Videoserver • ... • Frameworks • SUN & IBM Java media framework • unified architecture for the playback, synchronization, capture, transmission and transcode of media -- including streaming audio and video -- across most major operating systems

34. Clients • Aspects • Buffer • Synchronisation • Multicast • Examples • FVC.COM I-viewer • Cisco IP/TV viewer • IBM Videocharger client • Apple Quicktime client • Microsoft Windows Media Player • Mbone tools • SDR, Vic, Rat, …

35. What does streaming mean for your network ?

36. All problems solved ? • NO ! • Open issues • standaards • protocol implementations • RTP, RTSP • formats • MPEG program/system/transport streams • Metadata • filesystems • network • multicast • Quality of Service (QoS) • content delivery

37. Rights issues • Rights organisations see “On demand” as the highest service level possible and therefore demand high compensation • Rights organisations (in NL) are nog (yet) qualified for • TV rights • Film • News, Sport • Models for slice payments are missing • [YMMV]

38. Lessons learned • Audio and data often more important than Video • scenario dependent • Image quality and refresh-rate still no problem with Internet users • compare tv viewers • Connectivity end-user determines format (bitrate) • Multicast • Commercialisation needed • Content ! • Refresh services

39. Streaming services at your screen • Do not underestimate technology involved • recording, encoding, network, storage, distribution • Navigation for the end-user is important • Automate the workflow • Know in advance what to do with the material afterwards • does it stay online, storage, re-distribute • Don’t forget the rights ! • Warning: it is expensive (still) • Always subcontract large applications/services

40. Future • Quality • MPEG2 (both streaming & conferencing); DV format -> HDTV • QoS/Classes of Service • Contents • 3D(e.g.. VRML, QuicktimeVR) • Metadata (MPEG7) • Codecs • fractal; VBR • Standards • RTP/RTSP, incl. Authenticatie; MPEG4 & MPEG7 • Efficiency • multicast: data too; content delivery (caching & replication) • New applications • gnutella • More streaming servers/clients • better access networks • media & asset management

41. Come forward: TF-STREAM • Who has experience ? • which tools, what have you done • plans for content creation, mgt, deployment • Ideas: how can this (new) technology be put to work in projects • at your institute • between institutes, organisations, NRNs, ... • TF-STREAM • http://www.terena.nl/task-forces/tf-stream/ • streaming@terena.nl (majordomo@terena.nl subscribe)

42. Demo streaming • SURFnet-TV • 1. FVC I-studio • 2. IBM Videocharger • 3. Cisco IP/TV • Real • Quicktime • WindowsMedia • SNOB/Snelnet • SGI Mediabase • Compcore player • FUNET-TV • US: • ICAIR • ResearchChannel • Vanderbilt Univ.

43. URLs (background info) • http://stream.surfnet.nl/ • http://contact.surfnet.nl/ • http://www.surfnet.nl/diensten/seminars/surfnet-sec/streaming/ • http://skin.surfnet.nl/video-audio/index.html • http://www.surfnet.nl/innovatie/surfworks/showcase/ • http://www.surfnet.nl/innovatie/surfworks/streaming/surfnettven.html • http://www.surfnet-tv.nl/ • http://dv.internet2.edu/ • http://dv.internet2.edu/videospace/index.html • http://www.researchchannel.com/ • http://www.vide.net/