IT 424 Networks2

1. IT 424 Networks2 Chapter 4: Multimedia Networks (ch 7 in book) Part 1: Streaming Stored Multimedia Ack.: Slides are adapted from the slides of the book: “Computer Networking” – J. Kurose, K. Ross

2. Multimedia & QoS Introduction to Audi and Video Streaming Stored Media Overview Streaming Protocols Streaming Live – Interactive Apps

3. To describe the characteristics and requirements of multimedia applications 1 To explain the problem of multimedia transmission over the Internet 2 To define the challenges of streaming stored video/audio 3 To differentiate between the different schemes and protocols for streaming multimedia 4 To understand and analyze the RTSP protocol To understand the difference between streaming stored and live multimedia 5 6 Learning Outcomes

4. Multimedia & QoS Introduction to Audi and Video Streaming Stored Media Overview Streaming Protocols Streaming Live – Interactive Apps

5. Multimedia & QoS QoS network provides application with level of performance needed for application to function. Multimedia and Quality of Service: What Is It? multimedia : network audio and video (“continuous media”)

6. Multimedia & QoS - Applications MM Networking Applications • MM Application Types: • Streaming, stored audio, video • Streaming: can begin playout before downloading entire file • Stored (at server): which allows pause, rewind or fast-forward (implies storing/buffering at client) • E.G., Youtube, netflix, hulu • Streaming live audio, video • E.G., Live sporting event (football) • Conversationalvoice/video over IP • This type allows people to use audio/video to communicate with each other in real time • Interactive nature of human-to-human conversation limits delay tolerance • E.G., Skype

7. Multimedia & QoS - Applications MM Networking Applications Fundamental characteristics: • Delay sensitive • End-to-end delay • Delay jitter • Loss tolerant: • infrequent losses cause minor glitches • Different from data, which are loss intolerant but delay tolerant. Jitter is the variability of packet delays within the same packet stream

8. Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Overview Streaming Protocols Streaming Live – Interactive Apps

9. Introduction to Audio and Video - Audio Multimedia: Audio quantization error quantized value of analog value • Analog audio signal is converted to a digital signal as follows: • Analog audio signal sampled at constant rate • Telephone: 8,000 samples/sec • CD music: 44,100 samples/sec • Each sample quantized, i.e., Rounded • E.G., 28=256 possible quantized values • Each quantized value represented by fixed number of bits, • e.g., 8 bits for 256 values analog signal audio signal amplitude sampling rate (N sample/sec) time

10. Introduction to Audio and Video - Audio Multimedia: Audio quantization error quantized value of analog value • Example: 8,000 samples/sec, 256 quantized values, what is the bit rate? • Answer: 64,000 bps • Receiver converts bits back to analog signal: • Some quality reduction example rates • CD: 1.411 Mbps • MP3: 96, 128, 160 kbps • Internet telephony: 5.3 kbps and up analog signal audio signal amplitude sampling rate (N sample/sec) time

11. Introduction to Audio and Video - Video Multimedia: Video ……………………...… ……………………...… spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N) • Video: sequence of images displayed at constant rate • E.G. 24 images/sec • Digital image: consists of array of pixels • Each pixel represented by bits • Coding: use redundancy within and between images to decrease # bits used to encode image • Spatial (within image) • Temporal (from one image to next) frame i temporal coding example: instead of sending complete frame at i+1, send only differences from frame i frame i+1

12. Introduction to Audio and Video - Video Multimedia: Video ……………………...… ……………………...… spatial coding example: instead of sending N values of same color (all purple), send only two values: color value (purple) and number of repeated values (N) • CBR: (constant bit rate): video encoding rate fixed • VBR: (variable bit rate): video encoding rate changes as amount of spatial, temporal coding changes • Examples: • MPEG 1 (CD-ROM) 1.5 Mbps • MPEG2 (DVD) 3-6 Mbps • MPEG4 (often used in Internet, < 1 Mbps) frame i temporal coding example: instead of sending complete frame at i+1, send only differences from frame i frame i+1

13. Introduction to Audio and Video - Video ? ? ? ? ? ? ? But you said multimedia apps requires QoS and level of performance to be effective! ? ? ? ? Today’s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss Multimedia Over Today’s Internet TCP/UDP/IP: “best-effort service” • no guarantees on delay, loss

14. Multimedia & QoS - Applications How Should The Internet Evolve To Better Support Multimedia? Integrated services philosophy: • Fundamental changes in internet so that apps can reserve end-to-end bandwidth • Requires new, complex software in hosts & routers Laissez-faire • No major changes • More bandwidth when needed • Content distribution, application-layer multicast • Application layer Differentiated services philosophy: • Fewer changes to internet infrastructure, yet provide 1st and 2nd class service What’s your opinion?

15. Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Overview Streaming Protocols Streaming Live – Interactive Apps

16. Streaming Stored Video - Challenges Streaming Stored Multimedia • Stored multimedia: • Media stored at source • Transmitted to client • Streaming: • client playout begins before all data has arrived • Timing constraint for still-to-be transmitted data: on time for playout

17. Streaming Stored Video 2. video sent • video • recorded (e.g., 30 frames/sec) 3. video received, played out at client (30 frames/sec) Streaming: at this time, client playing out early part of video, while server still sending later part of video Streaming Stored Video Cumulative data network delay (fixed in this example) time

18. Streaming Stored Multimedia - Interactivity Streaming Stored Multimedia: Interactivity • VCR-like functionality: client can pause, rewind, FF, push slider bar • 10 sec initial delay OK • 1-2 sec until command effect OK • Timing constraint for still-to-be transmitted data: in time for playout

19. Streaming Stored Video - Challenges Streaming Stored Video: Challenges • Continuous playout constraint: once client playout begins, playback must match original timing • Network delays are variable (jitter), so will need client-side buffer to match playout requirements • Other challenges: • Client interactivity: pause, fast-forward, rewind, jump through video • Video packets may be lost, retransmitted

20. Streaming Stored Video client video reception constant bit rate video playout at client variable network delay buffered video client playout delay Streaming Stored Video: Client Buffering constant bit rate video transmission Cumulative data time • Client-side buffering and playout delay: compensate for network-added delay, delay jitter

21. Streaming Stored Video: Buffering Client-Side Buffering, Playout buffer fill level, Q(t) drain rate d to decompression and playout, e.g., CBR r variable fill rate, x(t) client application buffer, size B video server client • Client-side buffering and playout delay: compensate for network-added delay, delay jitter

22. Streaming Stored Video: Buffering Client-Side Buffering, Playout playout rate, e.g., CBR r buffer fill level, Q(t) variable fill rate, x(t) client application buffer, size B video server client 1. Initial fill of buffer until playout begins at tp 2. Playout begins at tp, 3. Buffer fill level varies over time as fill rate x(t) varies and playout rate r is constant

23. Streaming Stored Video - Challenges Streaming Stored Video • Application-level streaming techniques for making the best out of best effort service: • Client-side buffering • Use of UDP versus TCP • Multiple encodings of multimedia Media Player • Jitter removal • Decompression • Error concealment • Graphical user interface w/ controls for interactivity

24. Streaming Stored Video - Challenges Internet Multimedia: Simplest Approach • Audio or video stored in file • Files transferred as HTTP object • Received in entirety at client • Then passed to player Audio, video not streamed: • No, “pipelining,” long delays until playout!

25. Streaming Stored Video - Challenges Internet Multimedia: Streaming Approach • Browser gets metafile • Browser launches player, passing metafile • Player contacts server • Server streams audio/video to player

26. Streaming Stored Video - Challenges Streaming from a Streaming Server • HTTP is not optimized for streaming • This scheme allows for non-HTTP protocol between server, media player • It also allows for UDP or TCP for step (3)

27. Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Overview Streaming Protocols Streaming Live – Interactive Apps

28. Streaming Protocols Streaming Multimedia: UDP or TCP? UDP • Server sends at rate appropriate for client (oblivious to network congestion !) • Often send rate = encoding rate = constant rate • Then, fill rate = constant rate - packet loss • Short playout delay (2-5 seconds) to remove network jitter • Error recover: time permitting TCP • Send at maximum possible rate under TCP • Fill rate fluctuates due to TCP congestion control • Larger playout delay: smooth TCP delivery rate • HTTP/TCP passes more easily through firewalls

29. Streaming Protocols Streaming Multimedia: Client Rate(s) 1.5 Mbps encoding 28.8 Kbps encoding Q: How to handle different client receive rate capabilities? • 28.8 Kbps dialup • 100 Mbps Ethernet • A: Server stores, transmits multiple copies of video, encoded at different rates

30. Streaming Protocols - RTSP User Control of Streaming Media: RTSP HTTP • Does not target multimedia content • No commands for fast forward, etc. RTSP: RFC 2326 • Client-server application layer protocol • User control: rewind, fast forward, pause, resume, repositioning, etc… • Out-of-band control • RTSP control messages use different port numbers than media stream: out-of-band - port 554 What it doesn’t do: • Doesn’t define how audio/video is encapsulated for streaming over network • Doesn’t restrict how streamed media is transported (UDP or TCP possible) • Doesn’t specify how media player buffers audio/video

31. Streaming Protocols - RTSP RTSP Example Scenario: • Metafile communicated to web browser • Browser launches player • Player sets up an RTSP control connection, data connection to streaming server

32. Streaming Protocols - RTSP Metafile Example <title>Twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src="rtsp://video.example.com/twister/video"> </group> </session>

33. Streaming Protocols - RTSP RTSP Operation

34. Multimedia & QoS Introduction to Audio and Video Streaming Stored Media Overview Streaming Protocols Streaming Live – Interactive Apps

35. Multimedia Applications Streaming Live Multimedia Examples: • Internet radio talk show • Live sporting event Streaming (as with streaming stored multimedia) • Playback buffer • Playback can lag tens of seconds after transmission • Still have timing constraint Interactivity • Fast forward impossible • Rewind, pause possible!

36. Multimedia Applications Real-time Interactive Applications • PC-2-PC phone • Skype • PC-2-phone • Dialpad • Net2phone • Skype • Video conference with webcams • Skype • Polycom • Going to look at Voice-over- IP in detail

37. Conclusion Conclusion • Internet provides best effort service only which is not suitable for multimedia applications • MM applications can be classified into three types: • Streaming, stored audio, video • Conversational voice/video over IP • Streaming live audio, video • Multimedia traffic is bursty in nature and requires stringent delay guarantees • Streaming mulimedia can be improved using UDP protocol, buffering content, adapt compression level and send redundant packets.

38. References References • Computer Networking: A Top-Down Approach Featuring the Internet by James Kurose and Keith Ross, Addison Wesley, 2012 (chapter 7 ) • Dynamic Adaptive Streaming over HTTP – Design Principles and Standard