Multimedia Streaming:

CSE228 Multimedia Systems Multimedia Streaming: Challenges and Protocols Jun Lu Xinran (Ryan) Wu Multimedia Streaming Protocols

Presentation Outline • Architecture for Multimedia Streaming • Summary of Challenges and Techniques • Protocol Stacks • Quick Introduction to RTSP, RTP/RTCP and RSVP • Reference Multimedia Streaming Protocols

What is Multimedia Streaming? • User’s perspective: • Quick start without waiting for full download. • Coming continuously without interruption. • VCR operation (pause, resume, fast forward, rewind, etc.) • Multimedia Streaming: Clients request audio/video files from servers and pipeline reception over the network and display Streaming Server Client/Receiver Storage Device Video Decoder Audio Decoder Multimedia Streaming Protocols Raw Video Compressed Video Multimedia Streaming Protocols Raw Audio Compressed Audio Transport Protocols Transport Protocols Internet Multimedia Streaming Protocols

Challenges in Media Streaming Protocols Clients/Receivers 1. Rate Control: Determine the sending rate based on the available bandwidth in the network. Ethernet Streaming Server Broadband Modem 3. Continuous Distribution: TCP/UDP/IP suite provides best-effort, no guarantees on expectation or variance of packet delay 2. Error Control: Improve video presentation quality in the presence of packet loss. Multimedia Streaming Protocols

Techniques in Multimedia Streaming Protocols (1) • Rate Control • Scalable compression • Base substream and enhancement substreams. • SNR scalability / spatial scalability / temporal scalability • Rate filter • Frequency filter • Frame-dropping filter • Re-quantization filter • QoS Feedback, e.g. RTCP. Multimedia Streaming Protocols

Techniques in Multimedia Streaming Protocols (2) • Error Control • Add redundant data in coding • MDC, (Multiple Description Coding) • FEC (Forward Error Coding) • Receiver End Error Concealment • Receiver conceal data loss. • Spatial interpolation, used in intra-coded frame. • Temporal interpolation, used in inter-coded frame. Multimedia Streaming Protocols

Techniques in Multimedia Streaming Protocols (3) • Continuous Distribution • Network Filter, e.g. frame-dropping filter. • Put filters at nodes connected to network bottleneck. • Content Replication, caching and mirroring. • Resource Reservation: RSVP Multimedia Streaming Protocols

A General View of the Multimedia Streaming Protocols • Stream description SDP, SMIL... Describe the session and content • Stream control RTSP Remote control the session • Media transport RTP Error control and flow control • Resource reservation (if any!): RSVP, DiffServ provide QoS for media streaming packets Multimedia Streaming Protocols

Protocol stacks for media streaming Control Plane Data Plane Compressed Video/Audio RTP Layer RTCP Layer RTSP Layer UDP TCP IPv4, IPv6 Internet Multimedia Streaming Protocols

Real Time Streaming Protocol (RTSP) SETUP rtsp://xxxxxxxxx:554/rtsp.wav/streamid=0 RTSP/1.0 CSeq: 3 Transport: rtp/avp;unicast;client_port=6970-6971 RTSP/1.0 200 OK CSeq: 3 Date: Thu, 29 May 2003 19:30:44 GMT Session: 9978-3 Transport: rtp/avp;unicast;client_port=6970 6971;server_port=10620-10621 Multimedia Streaming Protocols

Real Time Streaming Protocol (RTSP) (Con) • Supports the following operations: • Supports VCR-like control operations • Chooses delivery channels, e.g. UDP, TCP. • Supports any session description. • Establish and control stream of continuous audio and video media. • Retrieves requested media. • Add Media to an existing session. Multimedia Streaming Protocols

Real-Time Protocol (RTP/RTCP) • RTP is a data transfer protocol and RTCP is a control protocol. • RTP provides • Payload type identification: Identify which kind of information is being transmitted, RTP provides 128 possible different types of encoding; eg MPEG2 video, etc. • Sequencing: Reassemble the stream and detect packet loss. • Timestamping: Assure synchronization. • Source identification: Provide a means for the receiver to distinguish different sources. • RTP DO NOT provide • Quality of service • Reliability in packet delivery. • Security. Source Identifier Payload Type Sequence Number Timestamp Misc. Multimedia Streaming Protocols

Real-Time Protocol (RTP/RTCP) (Con) • Exchange information about the connection to assure quality of data distribution. • Control connection is held over a different channel than the RTP one. • RTCP provides • QoS Feedback: In form of sender reports/receiver reports. Senders adjust transmission rate based on reports. • Participant Identification: Human-friendly source identification. • Control Packets Scaling: Typically, limit the RTCP bandwidth to 5% of the session bandwidth, divided between the sender reports (25%) and the receivers reports (75%) • Minimal Session Control Information: Advanced control functions must be implemented in a higher level protocol. Multimedia Streaming Protocols

Quality of Service – Use of RSVP • RSVP is used for signaling end to end (admission control based on bandwidth, QOS requirements) Per flow policing DSCP marking Classify & schedule based on DSCP Diffserv Region Backbone Access Access RSVP signalling Trust Boundary Multimedia Streaming Protocols

Selected bibliography • General • Schulzrinne, IRT (Internet Real-Time) lab, http://www.cs.columbia.edu/~hgs/research/IRT • RTP • Schulzrinne, Casner, Frederick, Jacobson, « RTP, a tranport protocol for real-time applications », IETF, work in progress, <draft-ietf-avt-rtp-new-09.txt>, March 2001. • Schulzrinne, RTP home page, http://www.cs.columbia.edu/~hgs/rtp/ • RTSP • Schulzrinne, Rao, Lanphier, « Real-Time Streaming Protocol (RTSP) », IETF, Request For Comments 2327, April 1998. • Schulzrinne, RTSP home page, http://www.cs.columbia.edu/ ~hgs/rtsp/ Multimedia Streaming Protocols

Multimedia Streaming: