Multimedia Communications over the Internet

1. Multimedia Communications over the Internet

2. IP Packet-Switching Networks • Packet-switching protocols based on the Internet Protocol (IP) generally consist of a variety of different subnetworks of various technologies – The datalink and physical layer are largely outside the scope of the Internet efforts • IP is a connectionless (datagram) network layer – Different packets may traverse various routes between source and destination (they may arrive out of order or be duplicated) – Packets are transmitted on a best effort basis (no guarantee that a packet will arrive at destination

3. IP Networks (cont’d) • IP provides for the transmission of blocks of data called datagrams from source to destination, where source and destination are hosts identified by fixed-length addresses • Datagrams can be as large as 64 KB, but usually the are ~1500 bytes long. • IP also provides for fragmentation and reassembly of long datagrams, if necessary, for transmission through small packet networks. • IP basically runs over any Media Access Control (MAC) protocol

4. Internet Protocol Relationships (Non Real-Time) • At the highest level, the users invoke application programs that access services available across the Internet. An application interacts with the transport-level protocol to send or receive data. • Each application program chooses the size of transport needed, which can be either a sequence of individual messages or a continuous stream of bytes.

5. IP Relationships (cont’d) • Above IP, there are two transport layer options: TCP and UDP. – TCP provides end-to-end communication. It takes care of reliable, error-free transfer of data, and in- sequence delivery. UDP has less overhead compared to TCP, but does not guarantee transfers. – Both protocols support multiplexing, i.e. they allow several distinct streams of data between two hosts • Streams are labeled by source and destination port numbers

6. UDP • The User Datagram Protocol (UDP) offers only minimal services beyond those provided by the network layer – Multiplexing via port number – Checksum for the payload in the packet • IP provides only a checksum for the IP header

7. TCP • The Transmission Control Protocol (TCP) offers a reliable, sequenced byte stream service between two Internet hosts • Reliability is achieved by retransmission of lost or erroneous packets – Requires acknowledgment of received data • Flow control is achieved by means of a sliding window mechanism – Also used for congestion control: the sender reduces the window size in the case of traffic congestion • The sender probes for the currently available bandwidth by gradually increasing its window size until it senses packet loss - at which point it quickly reduces the window size

8. TCP (cont’d) • TCP is less suited to multimedia data than UDP – It trades reliability for delay jitter – Since it enforces in-order delivery, a single lost packet can hold up packets arrived after it – To improve delay behavior, an implementation is free to accept out-of-order packets and to acknowledge packets that have not been received but are excessively delayed • However, this may interfere with the TCP congestion control – The TCP congestion control mechanism imposes throughput limitations that may change on very short time scales

9. Connectivity Requirements • Connectivity requirements examples – Accommodating a listening audience of 25,000 streams daily with an average listening time of 6 minutes per stream requires, on average, only 100 concurrent streams – Accommodating a daily listening audience of 250,000 streams, with an average listening time of 20 minutes per stream, requires an average of 6,000 concurrent streams

10. Connectivity Requirements (cont’d) • Due to variable traffic patterns, load is not constant throughout the day, therefore the peak number of streams required may be twice or three times the average • When implemented, broadcast protocols (to an entire subnetwork) or multicast protocols (to members of a multicast group in various scattered subnetworks) allow to reduce the load on the network by allowing many user to share a single stream from the server

11. IP Multicast • IP Multicast allows a sender to transmit an IP packet to multiple receivers. Three possible ways for multicast: – Setting up multiple virtual circuits (not possible in Internet) – Including list of addresses in packet header (what happens if 1,000,000 addresses?) – Radio-like approach (used by IP Multicast) • Sender chooses an IP address from the class-D set (244.0.0.0 to 239.255.255.255). Receivers subscribe to multicast finding out the multicast address. • An IP multicast group can have any number of senders and receivers. Membership is dynamic. • IP Multicast is independent of transport mechanism (but TCP cannot be used)

12. HTTP ‧ HTTP (Hyper-Text Transfer Protocol) is built on top of TCP/IP ‧ For transmission through firewalls, often multimedia must be encapsulated in a HTTP envelope, lacking most of the required real-time features ‧ Multimedia data is downloaded on the client computer. Fast-start techniques can be used to begin playback as soon as enough of the content has been downloaded to the client • UDP and TCP protocols are otherwise used – UDP is sometimes blocked by firewalls

13. Requirements for Real-Time Multimedia • Sequencing. Packets must be reordered in real time at the receiver. If a packet is lost, must be detected and compensated without retransmission • Intramedia synchronization. Need some form of “time stamping” to know when to playback packets – Very important for VBR traffic • Intermedia synchronization. E.g., audio must be synchronized with video (lip-sync). • Payload identification. E.g., for media filtering • Frame indication. For synchronized delivery, it is useful to indicate when a video frame (or audio segment) begins or end.

14. RTP ‧ RTP (Real-Time Protocol) is a transport protocol for audio and videoconferences and other multiparticipant real-time applications. ‧ Designed to run over multicast IP ‧ Light-weight protocol, without error correction, flow control, or guaranteed time delivery functionality. ‧ Offers services such as playout synchronization, demultiplexing, media identification, and active- party identification.

15. Functionalities of RTP • Re-sequencing and loss detection • Multicast-friendly • Media-independent (voice, video,…) • Explicit support for mixers and translator – Mixers: take media from several users and mix them into one media stream (e.g., conference bridge) – Translators: take a single media stream, convert it to another format, and send it out (e.g. media filtering)

16. Functionalities of RTP (cont’d) • QoS feedback (via RTCP) – RTP sources can use this information to adjust their data rate (media scaling) • Loose session control – Using RTCP, participants can periodically distribute identification information (name, e-mail address,…) – Provides awareness of who is participating in a session without maintaining a centralized conference participant registry • Encryption

17. RTP (cont’d) • RTP specifies a packet structure for packets carrying audio and video data – Payload type identification – Packet sequence numbering – Time-stamping • RTP runs on top of UDP (can be viewed as a sub-layer of the transport layer) • RTP encapsulation is only seen at the end systems (not at the intermediate routers)

18. RTP - Example • Consider sending 64 Kb/s PCM-encoded voice over RTP • Application collects the encoded data in chunks, e.g. every 20 ms = 160 bytes/chunk • The audio chunk, along with the RTP header, forms the RTP packet, which is encapsulated into a UDP packet • The RTP header indicates the type of audio encoding in each packet. Sender can change encoding during a conference • RTP header also contains sequence number and timestamps

19. RTP Streams • RTP allows each source (e.g., a camera or a microphone) to be assigned its own independent stream of packets – E.g., for a videoconference between two participants, 4 streams could be opened: two streams for audio (one in each direction) and two for video • However, MPEG-1 and MPEG-2 bundle audio and video into a single stream during the encoding process - then only one RTP stream is generated in each direction.

20. Real-Time Control Protocol (RTCP) • Works in conjunction with RTP • Each participant in a RTP session periodically transmits RTCP control packets to all other participants. – Each RTCP packet contains sender and/or receiver reports with statistics useful to the application – E.g.: number of packets sent, number of packets lost, inter-arrival jitter, etc. • This feedback information can be used to control performance and diagnostic purposes – E.g. the sender may modify its transmission based on feedback

21. RTCP (cont’d) • RTCP can be used to synchronize different media stream within a RTP session – E.g.: videoconference where each sender generates one RTP stream for video and one for audio – Timestamps in RTP packets are tied to video/audio sampling clocks (not wall-clock time, ie real time) – Each RTCP sender-report packet contains, for the most recently generated packet in the associated stream, the timestamp of the RTP packet and the wall-clock time of when the packet was created – Receivers can use this association to synchronize the playout of audio and video

22. Protocols for Real-Time (cont’d) • RTSP (Real-Time Streaming Protocol) provides methods to realize commands (play, fast-forward, fast-rewind, pause, stop) similar to the functionality provided by CD players or VCRs. – Can control either a single or several time- synchronized streams of continuous media. – Can act as a network remote control for multimedia servers and can run over TCP or UDP

23. Live vs. Video-On-Demand • Live streaming is a live broadcast which allows users to join a session in which real time media is being sent over a network – Because these streams are live, users are not allowed to jump around to any point in time – Live streaming can exploit multicast or broadcast protocols • Video-on-demand represents content stored on a streaming server which can be viewed at any time – User is allowed to jump to any point in time in the media – Only unicast protocols are allowed