Chapter 7 End-to-End Data

1. Chapter 7 End-to-End Data Application Application data data Presentation Presentation encoding decoding ■ ■ ■ Message Message Message • Data Manipulating Functions (Affecting Throughputs) • How to encode the message into different formats in presentation format? (MPEG, JPEG, GIF…) • How to perform data compression in the message (more redundancy vs. more simplify) Fig. 7.1 – Presentation formatting involves encoding and decoding application data. CPSC558: Advanced Computer Networks

2. Chapter 7 End-to-End Data • Argument Marshalling (encoding) and Un-marshalling (decoding) is difficult because • Computer represents same data in different ways (e.g. Big/Little-endian format.) • Application programs are written in different languages (Compiler/Machine structure may be different.) So, what do we consider when dealing with encoding/decoding process? (next slide please…) CPSC558: Advanced Computer Networks

3. Facts to consider about data encoding/decoding • Data Types • Base types : integer, floating-point, characters • Flat types : structures and arrays • Complex types : for examples, trees (has Pointers) • Serializing • A network must be able to serialize the complex types (transfer complex types data) via the network. (Or ‘flatten’ the complex type data). Because complex type of data contains pointers (address of data) which may not the same in different machines. CPSC558: Advanced Computer Networks

4. Data Encoding (Argument Marchalling Strategies • A. Canonical Intermediate Form • Sender translates message from internal representation to an external representation for each type. • B. Receive-makes-right • Sender does not convert the base types but transfer data in its own internal types to receiver. The receiver is responsible for translating the data from sender into receiver’s own local format. (But use this strategy, every host must be ready to do the translate work – N-by-N solution.) • NOTE: if both sender and receiver are the same type of machine, then really there is no need to do the translation. CPSC558: Advanced Computer Networks

5. Tags (Encoding issues) • How the receiver knows what kind of data is contained in the message it receives? • Use tagged data approach • Type tag : data is a integer, floating point or character • Length tag : no. of integer in an array or size of an integer • Architecture tag • Use untagged data approach • In a Remote Procedure call, the message is programmed to send to receiver and receiver already knows/expect certain format or kind of data will be transmitted from sender. CPSC558: Advanced Computer Networks

6. Stubs (Encoding issues) • We use stubs (a piece of code) that perform argument marshalling (data encoding). • Stubs are typically used to support RPC (remote procedure calls) • On client side : stubs marshalling (encode) the procedure arguments into a message that can be transmitted by means of RPC protocol. • On server side : stubs un-marshalling (decode) the message back into a set of variables that can be used as arguments to call remote procedure. CPSC558: Advanced Computer Networks

7. Stubs (Encoding issues) Figure 7.5 CPSC558: Advanced Computer Networks

8. Multimedia Outline Compression RTP Scheduling CPSC558: Advanced Computer Networks

9. Compression Overview • Encoding and Compression • Huffman codes • Lossless • data received = data sent • used for executables, text files, numeric data • Lossy • data received does not != data sent • used for images, video, audio CPSC558: Advanced Computer Networks

10. Lossless Algorithms • Run Length Encoding (RLE) • example: AAABBCDDDD encoding as 3A2B1C4D • good for scanned text (8-to-1 compression ratio) • can increase size for data with variation (e.g., some images) • Differential Pulse Code Modulation (DPCM) • example AAABBCDDDD encoding as A001123333 • change reference symbol if delta becomes too large • works better than RLE for many digital images (1.5-to-1) CPSC558: Advanced Computer Networks

11. Dictionary-Based Methods • Build dictionary of common terms • variable length strings • Transmit index into dictionary for each term • Lempel-Ziv (LZ) is the best-known example • Commonly achieve 2-to-1 ration on text • Variation of LZ used to compress GIF images • first reduce 24-bit color to 8-bit color • treat common sequence of pixels as terms in dictionary • not uncommon to achieve 10-to-1 compression (x3) CPSC558: Advanced Computer Networks

12. Image Compression • JPEG: Joint Photographic Expert Group (ISO/ITU) • Lossy still-image compression • Three phase process • process in 8x8 block chunks (macro-block) • grayscale: each pixel is three values (YUV) • DCT: transforms signal from spatial domain into and equivalent signal in the frequency domain (loss-less) • apply a quantization to the results (lossy) • RLE-like encoding (loss-less) CPSC558: Advanced Computer Networks

13. Quantization and Encoding • Quantization Table 3 5 7 9 11 13 15 17 5 7 9 11 13 15 17 19 7 9 11 13 15 17 19 21 9 11 13 15 17 19 21 23 11 13 15 17 19 21 23 25 13 15 17 19 21 23 25 27 15 17 19 21 23 25 27 29 17 19 21 23 25 27 29 31 • Encoding Pattern CPSC558: Advanced Computer Networks

14. MPEG • Motion Picture Expert Group • Lossy compression of video • First approximation: JPEG on each frame • Also remove inter-frame redundancy CPSC558: Advanced Computer Networks

15. MPEG (cont) • Frame types • I frames: intrapicture • P frames: predicted picture • B frames: bidirectional predicted picture • Example sequence transmitted as I P B B I B B CPSC558: Advanced Computer Networks

16. MPEG (cont) • B and P frames • coordinate for the macroblock in the frame • motion vector relative to previous reference frame (B, P) • motion vector relative to subsequent reference frame (B) • delta for each pixel in the macro block • Effectiveness • typically 90-to-1 • as high as 150-to-1 • 30-to-1 for I frames • P and B frames get another 3 to 5x CPSC558: Advanced Computer Networks

17. MP3 • CD Quality • 44.1 kHz sampling rate • 2 x 44.1 x 1000 x 16 = 1.41 Mbps • 49/16 x 1.41 Mbps = 4.32 Mbps • Strategy • split into some number of frequency bands • divide each subband into a sequence of blocks • encode each block using DCT + Quantization + Huffman • trick: how many bits assigned to each subband CPSC558: Advanced Computer Networks

18. RTP • Application-Level Framing • Data Packets • sequence number • timestamp (app defines “tick”) • Control Packets (send periodically) • loss rate (fraction of packets received since last report) • measured jitter CPSC558: Advanced Computer Networks

19. Transmitting MPEG • Adapt the encoding • resolution • frame rate • quantization table • GOP mix • Packetization • Dealing with loss • GOP-induced latency CPSC558: Advanced Computer Networks

20. Layered Video • Layered encoding • e.g., wavelet encoded • Receiver Layered Multicast (RLM) • transmit each layer to a different group address • receivers subscribe to the groups they can “afford” • Probe to learn if you can afford next higher group/layer • Smart Packet Dropper (multicast or unicast) • select layers to send/drop based on observed congestion • observe directly or use RTP feedback CPSC558: Advanced Computer Networks

21. Real-Time Scheduling • Priority • Earliest Deadline First (EDF) • Rate Monotonic (RM) • Proportional Share • with feedback • with adjustments for deadlines CPSC558: Advanced Computer Networks