RTP/RTCP

1. RTP/RTCP • Background; • Overview; • Basic concepts; • RTP • RTCP • Applications of RTCP; • Applications of RTP;

2. Background • December 1992, Henning Schulzrinne, GMD Berlin, published RTP version 1 • Proposed (version 2) as standard November,1995 • Widely used in multimedia communications;

3. Background (cont’d) ALF & ILP --- Properties required for multimedia communication system • ALF (Application Layer Framing): It is the responsibility of application to split its data stream into a sequence of segments (so-called ADUs, Application Data Units) which are respected by the underlying transportation services. The segmentation is based on the source characteristics and the requirements of application level QoS control. • Simply speaking, it is the application (not the transport protocol) who determines the contents of packets (application-oriented packetization scheme); • Elimination of the improper packetization scheme and corresponding transmission controls at lower layers; • Generalization of ALF:look for elegant ways to allow application visibility/participation in lower-level activities, e.g. QoS – carry application intelligence to the point of QoS enforcement;

4. Background (cont’d) ALF & ILP --- Properties required for multimedia communication system (cont’d) ILP (Integrated Layer Processing): To simplify the unnecessary packet handlings internal to the network and let application do what it needs to do as much as possible; ALF  ILP The typical multimedia transport protocol that adheres to the above requirements  RTP

5. Real-time Transport Protocol (RTP) • RFC1889 • RTP provides end-to-end delivery services for real-time data interactive audio, video • source and payload type identification • sequence numbering (reseq, loss detection) • time-stamping (for intra and inter-media sync) • delivery monitoring and feedback • Runs over UDP (multiplexing, checksum) • Designed primarily for multicast of real-time data (also unicast, simplex, duplex)

6. RTP • Does not provide: • quality of service guarantees • synchronization among media streams • Lower layers or application can provide such guarantees • sequence numbers and timestamps can be used by application to provide synchronization

7. Session control • RTP does NOT provide session control • access control • floor control • etc. • need different protocols for those

8. Components • RTP; • RTCP; • Profile --- application-oriented configuration; • Format --- media-oriented configuration;

9. RTP components • RTP: The real-time transport protocol that carries data with real-time properties • Packet format; • Functional entity; • RTCP: RTP control protocol: monitors QoS, conveys info about participants • “loosely controlled” sessions, e.g., no membership control • more tightly controlled sessions require separate session control protocol • uses one port higher than RTP

10. RTP Architecture • Very thin transport layer to avoid restricting the application designer • RTP may often be integrated into the application rather than a separate layer • Definitions of several header fields deferred to A/V profile - e.g., marker bit

11. Profile specification • “Defines payload type codes and their mapping to payload formats (e.g., media encodings). A profile may also define extensions or modifications to RTP that are specific to to a particular class of applications. Typically, an application will operate under only one profile.”

12. Payload specification • which defines how a particular kind of payload data, such as H.261 encoded video, should be carried in RTP. These documents are typically titled "RTP payload Format for XYZ Audio/Video Encoding". Payload formats may be useful under multiple profiles and may therefore be defined independently of any particular profile. The profile documents are then responsible for assigning a default mapping of that format to a payload type value if needed.

13. RTP ... A/V profile H.261 video MPEG video Layered video PCM audio LPC audio ... PVH video LDCT video HVQ video RTP protocol architecture Core RTP specification Payload format Format decomposition Interdependent specifications arranged in a class hierarchy to support the Application-Level framing architecture

14. Basic concepts • RTP session The association among a set of participants communicating with RTP. For each participant, the session is defined by a particular pair of destination transport addresses (one network address plus a port pair for RTP and RTCP). The destination transport address pair may be common for all participants, as in the case of IP multicast, or may be different for each, as in the case of individual unicast network addresses plus a common port pair. In a multimedia session, each medium is carried in a separate RTP session with its own RTCP packets. The multiple RTP sessions are distinguished by different port number pairs and/or different multicast addresses.

15. Basic concepts • RTP packet A data packet consisting of the fixed RTP header, a possibly empty list of contributing sources (who contribute the payload), and the payload data. • RTCP packet • A control packet consisting of a fixed header part similar to that of RTP data packets, followed by structured elements that vary depending upon the RTCP packet type. Typically, multiple RTCP packets are sent together as a compound RTCP packet in a single packet of the underlying protocol.

16. Basic concepts Synchronization source (SSRC): The source of a stream of RTP packets, identified by a 32-bit numeric SSRC identifier carried in the RTP header so as not to be dependent upon the network address. All packets from a synchronization source form part of the same timing and sequence number space, so a receiver groups packets by synchronization source for playback. A synchronization source may change its data format, e.g., audio encoding, over time. The SSRC identifier is a randomly chosen value meant to be globally unique within a particular RTP session . A participant need not use the same SSRC identifier for all the RTP sessions in a multimedia session; the binding of the SSRC identifiers is provided through RTCP . If a participant generates multiple streams in one RTP session, for example from separate video cameras, each must be identified as a different SSRC.

17. Basic concepts Contributing source (CSRC): A source of a stream of RTP packets that has contributed to the combined stream produced by an RTP mixer . The mixer inserts a list of the SSRC identifiers of the sources that contributed to the generation of a particular packet into the RTP header of that packet. This list is called the CSRC list.

18. end system 1 from ES1: SSRC=6 end system 3 mixer from M: SSRC=52 CSRC list={6, 23} from ES2: SSRC=23 end system 2 Mixer • a mixer may change the data format (coding) and combine the streams in any manner • example: video mixer (~MCU)

19. end system 1 from ES1: SSRC=6 from ES2: SSRC=23 from ES1: SSRC=6 transl.1 transl.2 authorized tunnel from ES2: SSRC=23 end system 2 firewall from ES1: SSRC=6 from ES2: SSRC=23 Translator • data may pass through the translator intact or may be encoded differently • the identity of individual flows remains intact! • example: going through a firewall

20. Real Time Protocol

21. Header fields • V: version (currently 2) • P: padding (may be needed for encryption) • X: extension header • CC: contributing source count • M: Marker bit (e.g., end of talkspurt) • PT: Payload type (audio, video, encoding)

22. Header fields • Sequence number (16 bits): • increase by one for each RTP data packet sent; • used by the receiver to detect packet loss and to restore packet sequence; • Timestamp (32 bits): •  Reflect the sampling instant of the first octet in the RTP data packet; • Used in synchronization and jitter calculation; • Several consecutive RTP packets may have equal timestamps if they are (logically) generated at once, e.g., belong to the same video frame. • Consecutive RTP packets may contain timestamps that are not monotonic if the data is not transmitted in the order it was sampled, as in the case of MPEG interpolated video frames. (The sequence numbers of the packets as transmitted will still be monotonic.)

23. Header fields • SSRC (32 bits): • identifies the synchronization source; CSRC list: 0 to 15 items, 32 bits each The minimum size of RTP header is 12 bytes.

24. Modifications & Extension • Profile-Specific Modifications to the RTP Header: • In keeping with the ALF design principle, the header may be tailored through modifications or additions defined in a profile specification.  M/PT;  Payload headers; • Additional fixed fields that follow immediately after the SSRC field of the existing fixed header. Fixed fields  present in each packet; Extensions  if present is denoted by X bit; • Extension mechanism: • If the X bit in the RTP header is one, a variable-length header extension is appended to the RTP header, following the CSRC list if present.

25. How does RTP works • Timestamping - most important information for real-time applications. • The sender timestamp according to the instant the first octet in the packet was sampled. • The receiver uses timestamp to reconstruct the original timing • Also used for synchronize different streams; audio an video in MPEG. ( Application level responsible for the actual synchronization)

26. How does RTP work • Payload type identifier • specifies the payload format as well as encoding/compression schemes • The application then knows how to interpret the payload • Source identification • Audio conference

27. Where is RTP reside • RTP is typically run on top of UDP • Uses UDP’s multiplexing and checksum functions • RTP is usually implemented within the application (Lost packets and congestion control have to be implemented in the application level)

28. RTCP - Real Time Control Protocol • Designed to work together with RTP • In an RTP session the participants periodically send RTCP packet to give feedback on the quailty of the data. • Comparable to flow and congestion control of other transport protocols. • RTCP produces sender and receivers reports; statistics and packet counts

29. RTCT packet types • Recevier reports: feedback of data delivery • Packet lost, jitter, timestamps • Sender reports: • Intermedia synchronization, number of bytes sent, packet counters • SDES, BYE, APP

30. RTCP provides the following services • QoS monitoring and congestion control • Primary function: QoS feedback to the application • The sender can adjust its transmission • The receiver can determine if the congestion is local, regional, or global • Network managers can evaluate the network performance for multicast distribution

31. RTCP provides the following services (Cont) • Source identification • inter-media synchronization • control information scaling • Limit control traffic (most 5 % of the overall session traffic)

32. RTCP --- SR • From participants that are active senders; • Transmission and reception statistics;

33. RTCP --- SR header V(2 bits): version; P(1 bit): padding indicator; RC(4 bits): report block count; PT (8 bits): payload type; Length (16 bits); SSRC of the sender;

34. RTCP --- SR Sender information • NTP: 64bits • Indicates the wallclock time when this report was sent (seconds); • The integer part in the first 32 bits and the fractional part in the last 32 bits; • Compact representation: only the middle 32 bits are used; • Used to measure round-trip propagation delay; • RTP timestamp: 32bits • Corresponds to the NTP timestamp, but in the same units and with the same random offset as the RTP timestamps in data packets; • Used for intra- and inter-media synchronization;

35. RTCP --- SR • sender's packet count: 32 bits • The total number of RTP data packets transmitted by the sender since starting transmission up until the time this SR packet was generated. • The count is reset if the sender changes its SSRC identifier. • sender's octet count: 32 bits • The total number of payload octets (i.e., not including header or padding) transmitted in RTP data packets by the sender since starting transmission up until the time this SR packet was generated. • The count is reset if the sender changes its SSRC identifier. • This field can be used to estimate the average payload data rate.

36. RTCP --- SR Report block SSRC_n (source identifier): 32 bits The SSRC identifier of the source to which the information in this reception report block pertains. fraction lost: 8 bits The fraction of RTP data packets from source SSRC_n lost since the previous SR or RR packet was sent.

37. RTCP --- SR cumulative number of packets lost: 24 bits The total number of RTP data packets from source SSRC_n that have been lost since the beginning of reception. extended highest sequence number received: 32 bits The low 16 bits contain the highest sequence number received in an RTP data packet from source SSRC_n, and the most significant 16 bits extend that sequence number with the corresponding count of sequence number cycles. interarrival jitter: 32 bits An estimate of the statistical variance of the RTP data packet interarrival time, measured in timestamp units and expressed as an unsigned integer. D(i,j)= (Rj-Sj)-(Ri-Si) J=J+(|D(i-1,i)|-J)/16

38. RTCP --- SR last SR timestamp (LSR): 32 bits  The middle 32 bits out of 64 in the NTP timestamp received as part of the most recent RTCP sender report (SR) packet from source SSRC_n. delay since last SR (DLSR): 32 bits  The delay, expressed in units of 1/65536 seconds, between receiving the last SR packet from source SSRC_n and sending this reception report block. The round-trip propagation delay: (A- LSR - DLSR).

39. RTCP --- RR • From passive receivers; • Reception statistics; • The format of the receiver report (RR) packet : • same as that of the SR packet except that… • the packet type field contains the constant 201; • the five words of sender information are omitted (these are the NTP and RTP timestamps and sender's packet and octet counts). (2) The remaining fields have the same meaning as for the SR packet.

40. RTCP --- RR

41. RTCP --- SDES  Information on participants;  CNAME (canonical name) included to: (1) keep track of each participant. (2) associate multiple data streams from a given participant in a set of related RTP sessions, for example to synchronize audio and video. An SDES packet containing a CNAME item must be included in each compound RTCP packet. Other source description items may optionally be included if required by a particular application, subject to bandwidth constraints

42. RTCP --- SDES • The SDES packet is a three-level structure composed of • a header • zero or more chunks • each chunk is composed of items describing the source identified in that chunk. source count (SC): 5 bits The number of SSRC/CSRC chunks contained in this SDES packet.

43. RTCP --- SDES --- CNAME <1> Because SSRC identifier may change, the CNAME item is required to provide the binding from the SSRC identifier to an identifier for the source that remains constant. <2> Like the SSRC identifier, the CNAME identifier should also be unique among all participants within one RTP session. <3> To provide a binding across multiple media tools used by one participant in a set of related RTP sessions, the CNAME should be fixed for that participant. <4> To facilitate third-party monitoring, the CNAME should be suitable for either a program or a person to locate the source.

44. Compound RTCP packet Multiple RTCP packets may be concatenated to form a compound RTCP packet that is sent in a single packet of the lower layer protocol, for example UDP.

45. Compound RTCP packet <1> Reception statistics (in SR or RR) should be sent as often as bandwidth constraints will allow to maximize the resolution of the statistics, therefore each periodically transmitted compound RTCP packet should include a report packet. <2> New receivers need to receive the CNAME for a source as soon as possible to identify the source and to begin associating media for purposes such as lip-sync, so each compound RTCP packet should also include the SDES CNAME. • <3> The control traffic should be limited to a small and known fraction of the session bandwidth: • Small so that the primary function of RTP to carry data is not impaired; • Known so that the control traffic can be included in the bandwidth specification given to a resource reservation protocol, and so that each participant can independently calculate its share. • It is suggested that the fraction of the session bandwidth allocated to RTCP be fixed at 5%. • All participants in the session must use the same values so the same interval will be calculated. Therefore, these constants should be fixed for a particular profile.

46. Session Bandwidth & RTCP Interval double rtcp_interval (int members, int senders, double rtcp_bw, int we_sent, int packet_size, int *avg_rtcp_size, int initial) rtcp_bw = 5% session_bw; How long the transmission of RTCP(n) will last by using the participant’s share of the rtcp_bw;

47. Session Bandwidth & RTCP Interval <1> If (initial)  the interval = 0.5 * RTCP_MIN_TIME; //(2.5s) <2> To estimate the size of RTCP packets; <3> Calculation of the participant’s share of session_bw If senders > 0 .and. senders < members * 0.25  if we_sent  share = rtcp_bw*0.25/senders else  share = rtcp_bw*0.75/(members-senders) Otherwise  share = rtcp_bw/members; <4> Get the interval interval = avg_rtcp_size/share; if interval < RTCP_MIN_TIME  interval = RTCP_MIN_TIME; interval = interval * (drand48()+0.5 );

48. RTCP and Intermedia Synchronization RTP timestamps are offset with random intervals (Rnd ini values); RTP timestamp counter may not tick at nominal rate; SRs correlate “real” time (wallclock time, i.e. NTP) with RTP TS; (ini+tm0) = NTP0 RTP TS (video)  NTP  RTP TS (audio)

49. Application Examples ---RTCP-Based Rate Control Mechanism Principles: <1> Packet loss is mainly caused by the network congestion; <2> As for each node, media rate should be dynamically adjusted to packet loss rate;

50. Application Examples ---RTCP-Based Rate Control Mechanism How – to: <1> RTCP analysis. The receiver reports of all receivers are analyzed and statistics of packet loss, packet delay jitter and roundtrip time are computed. <2> Network state estimation. The actual network congestion state seen by every receiver is determined as unloaded, loaded or congested. This is used to decide whether to increase, hold or decrease the bandwidth requirements of the sender. <3> Bandwidth adjustment. The bandwidth of the multimedia application is adjusted according to the decision of the network state analysis. The user can set the range of adjustable bandwidth, i.e., specify the minimum and maximum bandwidth.