Rate-Control in Video Codec

1. Rate-Control in Video Codec Heejune AHN Embedded Communications Laboratory Seoul National Univ. of Technology Fall 2008 Last updated 2008. 11. 16

2. Agenda • Basic Concepts • Rate-Distortion Theory • Practical Rate-Control Algorithms • Other issues

3. 1. Rate Control Concepts • Key players • coded video quality: (usu. Lower limits) • bit-rate: some constraints (usu. upper limits) • input video sequence: (noisy, degree of details, motion complexity) • coding parameters (standards, coded mode, quantization step) • Rate-optimization • select the parameters for best video quality under constraints • Poor algorithm : low quality, large fluctuation, dropped frames Rate Control network/ storage Coded Video Coding para. selection Input Video Bit-rate contraints encoder

4. Video stream Bit-rate characteristics • Factors • Encoding algorithms (intra/inter, forward/bi-direction, DCT/Wavelet , standards, etc) • Input video • Encoding parameters • Q step, picture &mb mode, mv search area, GOP structures

5. Bit-rate Constraints • Typical constraints parameters • Average (mean) bit-rate (with duration) • Peak (maximum) bit-rate • Maximum variation in bit rate • Prevention of underflow and overflows at encoder &decoder buffer • Required latency • Examples • DVD video • Total 4.7GB (play time 3 hours), peak rate => 3.5Mbps with some peak limits • Video conferencing in ISDN or Circuit switching network • Constant transmission bit-rate and delay => Based on buffer size, peak, average rate is calcuated • Vide streaming over internet and packet switched network • Network condition is variable. Should be adaptive rate control

6. 2. Rate Control Theory • Revisit to Quantization • For Uniform distributed Input & Uniform quantizer • Rate Distortion Theory • General trends are same as above distortion ~ input signal variance and exp. desc.(bitrate) !!

7. Rate-Distortion Curve • Rate decreases as distortion increases • depends on input statistics (complex image => large variance) • PSNR measure is 10log10(2552/D)

8. Rate-Distortion Optimization • Motivation • Which subject I have to study if I gets 90pts in Math, 50pts in Eng. ? • Image coding • Condition: we can use 10KBytes for one picture and each parts have different complexity (i.e, simpler and complex, variance) • Which parts I have to assign more bits? simpler blocks More complex blocks

9. Lagrange multiplier • Constrained Optimization Problem • Minimize distortion with bits no larger than max bits • Cannot use partial differential for minima and maxima • Lagrange Multiplier technique • Insert one more imaginary variable (called Lagrange) • We have multivariable minimization problem

10. 1 1 Side note: why not partial differential • Simple example • min subject to • Solution • By substitution • minimum ½ at x = y = 1/sqrt(2) • Wrong Solution • With complex contraints, we cannot use substitution of variable method • We cannot change partial different with ordinary difference • If we check, • x = y = 0 ?

11. An Example: Lagrange Multiplier • Optimization Problem • multiple independent Gaussian variables • Lagrange multiplier optimization

12. Solution • M 개를 모두 곱하고 M square 를 하면 • Bit-allocation • Lagrange value

13. RDO in Image Coding • Bit allocation problem • Modified Lagrange Method • First proposed by H. Everett, then signal processing domain by Shoham and Gersho • Theorem • For any l > 0, the solution B*(l) to the ‘unconstrained’ problem • is also a solution to the ‘constrained’ problem subject to • Here B = {r1,r2,. . ., rN}, ie. Bit allocaton each ri such that

14. RDO algorithm • How to find such l, that results to • Algorithm • Set l • Get Ri for each i independently ie. tagent to slope of RD curve • Get R (e.g. sum of ri’s) • Repeat until R = Rc, changing l (note: large l small b) • Diverse Constraints • Sum Ri < R • Embedded sum Ri < Rx • Buffer controls • Dependency condition

15. RDO algorithm • Intuitive understanding • Reverse Water filling theory • larger bits for larger variance block • The returns will be same at the equivalent point Large variance RD curve small variance RD curve RS* RL*

16. Practical Rate Control • Problem in Lagrange optimization • too much computational complexity for real-time use • Rate control Model • Buffer occupancy level B at the end of i-th frame • BiPRE = Bi-1 + Rv – Rc • Bi = BiPRE if 0< BiPRE <Bmax = Bmax if overflow (BiPRE >Bmax) = 0 if underflow (BiPRE < 0) • Rate Control (a negative feedback loop) : see Fig. 10.9 • High level => large Q step => smaller Rv => level down • Low level => smaller Q step => larger Rv => level up Rate Control Quant step Input Video Encoder Rc Rv

17. Practical Rate Control • H263 Test Model Rate control • Frame Level • Bit allocation for preventing buffer underflow and overflow (some times even frame skip) • Macroblock level • Allocate bits for each MB • A : # of pixles in a MB, si : standard deviation, Qi : quant. step • Step 1: calculate the MB complexity si • Step 2: calculate Qi from K, A, C, ai, si • Step 3: encode MB • Step 4: update the model parameters K, C

18. MPEG-2 Test Model 5 • Bit allocation • Assign a target bits to current GOP • Assign to each current picture • Based on Complexity of previous I, P, B pictures (Q-Step and resultant bit) • Rate control • Compare coded bit with expected values, and adjust Q-Step • Modulation • block activity, increase Q-step

19. 4. Other Rate Control Issue • Variable Complexity Algorithm • Constraints on computational power • Motion search algorithm & search range • Rate Control Mode selection • Frame rate (skipping) • Trade off • Computation complexity v.s. video quality • Also depends on input image (time-varying) • Usage • Computation limited Mobile device (usu. Software based codec) • Iain E.G. Richardson, the athor of textbook,’s interest • Performance Measure • PSNR is not enough (for subject quality measure) • Some objective measures are under development • SSIM(Structural SIMiliarity)

20. Appendix Rate distortion theory

21. Rate-Distortion Theory • Distortion of X’ from X • D = H(X) – H(X|X’) • X 전체의 정보양에서 X’을 통하여 확보한 X의 정보양을 뺀것 (make sense? Yeh ~~~) • 참고 : mutual information : I (X:X’) = H(X:X’) = H(X’:X) • Information theoretic definition D = H(X) – I(X:X’) I(X:X’) H(X) H(X’) Note: 붉음색은 X’ = Quant(X) 인경우

22. Rate Distortion • Rate-distortion function • Inverse relation between Information/Data Rate and Distortion • Example with Gaussian R.V. (known as most difficult distribution) • Refer to element of Information theory

23. Derivation