1. Multimedia Over IP Networks -- II Hao Jiang Computer Science Department Boston College Nov. 8, 2007 CS335 Principles of Multimedia Systems

2. Real-time Transport Protocol • Real-time transport protocol (RTP) is an Internet-standard protocol for transmitting real-time multimedia over IP network. • RTP is (often) built on top of UDP. • A RTP packet is embedded in a UDP packet and provides additional information to support real-time multimedia communications • Timestamp. • Sequence number and source id. UDP header RTP header Multimedia Payload CS335 Principles of Multimedia Systems

3. The RTP Packet Header Timestamp is a 32bit integer placed on the RTP packet. It is incremented based on the local system clock. For video it is usually 90kHz. Sequence number is a 16bit integer incremented by 1 for each generated RTP packet. PT: payload type. For example, 7 is for LPC audio, and 34 is for H263 video. CS335 Principles of Multimedia Systems

4. Example: Motion JPEG Payload Packet • Timestamp is incremented at 90KHz. The marker M bit is set at the last packet for each frame. Main JPEG header following the RTP header There are other headers to provide information like quantization table, etc. CS335 Principles of Multimedia Systems

5. RTCP • Real Time Control Protocol (RTCP) packets can be generated by both a sender and a receiver. • RTCP uses a different UDP port from RTP. • RTCP packets can be used to generate reports for network statistics such as round trip time, jitter and to synchronize different information sources. V P IC PT Length Format Specific Information RCTP Packet Format CS335 Principles of Multimedia Systems

6. RTCP Receiver Report • Payload type = 201. • RTCP RR packet reports include • Loss fraction: ratio of lost packets # / expect packets #. • Cumulative number of packets lost. • Highest sequence number received. • Inter-arrival jitter D(i) = (R(i) – R(i-1)) – (S(i) - S(i-1)) J(i+1) = J(i) * 15/16 + | D(i) | * 1/16 • Last send report timestamp (LSR) and delay since last send report (DLSR) [NTP time] round trip time = current timestamp - LSR - DLSR sender LSR Current Timestamp DLSR receiver CS335 Principles of Multimedia Systems

7. RTCP Sender Report • Payload type = 200. • RTCP SR packet contains • NTP timestamp. • RTP timestamp. • Reporter SSRC and others. • The most important usage of RTCP SR packet is to synchronize different media streams. CS335 Principles of Multimedia Systems

8. RTP Example Application encoding buffer Audio Codec, Packing, Depacking capturing Internet Playing back Playback buffer RTP Packet buffer Client n Client k Client i Audio Conferencing based on RTP CS335 Principles of Multimedia Systems

9. RTP Media Sender Buffer until send time comes Adding sequence number and other fields to make RTP packets … … Compressed media frames with relative time stamps fragmentation Send to UDP module System clock … … Media buffer CS335 Principles of Multimedia Systems

10. RTP Media Receiver RTP packet with timestamp t > t1and t < t2 Packet buffer (Packet with timestamp t1) Play back buffer Scheduled to play out (Packet with timestamp t2) decompress We usually need another buffer for de-fragmentation. CS335 Principles of Multimedia Systems

11. Timing Sender clock 20ms audio 20ms audio 20ms audio encoding making a packet Across The network 20ms audio Receiver clock What happens if receiver clock and sender clock has different frequency? CS335 Principles of Multimedia Systems

12. Methods to Compensate for Clock Skew • A method is to directly compute the skew: sender timestamp n1 = (t1 – tx)/dtx packet arrive timestamp n2 = (t2 – ty)/dty Assume we have 2 packets a and b, then n1(a) – n1(b) = [t1(a) – t1(b)]/dtx n2(a) – n2(b) = [t2(a) – t2(b)]/dty => (n2(a) – n2(b)) / (n1(a) – n1(b)) = dtx/dty + D(a,b)/(t1(a) – t1(b))*dtx/dty If (t1(a) – t1(b)) is constant, we can do a long term average to remove the noise term. CS335 Principles of Multimedia Systems

13. Compensation for Jitter • The leaky bucket model: Similar to a leaky bucket, we can use a buffer to shape the network traffic to generate a smooth playback. Buffer … CS335 Principles of Multimedia Systems

14. Compensation for Jitter # of Accumulated Packets t1 t2 N/(t2-t1) CS335 Principles of Multimedia Systems

15. Compensation for Jitter # of Accumulated Packets Production Curve Consumption Curve t1 t2 N/(t2-t1) CS335 Principles of Multimedia Systems

16. Compensation for Jitter # of Accumulated Packets Production Curve Consumption Curve t1 t2 N/(t2-t1) CS335 Principles of Multimedia Systems

17. Compensation for Jitter # of Accumulated Packets Production Curve Consumption Curve t1 t2 CS335 Principles of Multimedia Systems

18. Media Synchronization • In RTP, different media streams (audio and video) are sent separately. • Media synchronization relies on the correspondence of timestamp and NTP time on the local machine. • All the information sources need to be synchronized so that they have the same NTP time base. • RCTP SR packet is used to align the timestamp with the NTP time in media synchronization. CS335 Principles of Multimedia Systems

19. Useful Links • http://www.realnetworks.com/support/education/sdk.html • http://www.w3.org/AudioVideo/Activity • http://www.networksorcery.com/enp/protocol/rtp.htm CS335 Principles of Multimedia Systems