1 / 38

Modeling and Evaluating Feedback-Based Error Control for Video Transfer

Modeling and Evaluating Feedback-Based Error Control for Video Transfer. PhD Candidate: Yubing Wang - Computer Science, WPI, EMC Corp. Committee: Prof. Mark Claypool - Computer Science, WPI Prof. Robert Kinicki - Computer Science, WPI Prof. Dan Dougherty - Computer Science, WPI

elma
Télécharger la présentation

Modeling and Evaluating Feedback-Based Error Control for Video Transfer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Modeling and Evaluating Feedback-Based Error Control for Video Transfer PhD Candidate: Yubing Wang - Computer Science, WPI, EMC Corp. Committee: Prof. Mark Claypool - Computer Science, WPI Prof. Robert Kinicki - Computer Science, WPI Prof. Dan Dougherty - Computer Science, WPI Prof. Ketan Mayer-Patel – Computer Science, UNC at Chapel Hill Ph.D. Dissertation Defense

  2. Video Transfer 5 4 3 2 1 Too Late 5 3 1 5 4 3 2 1 Frame Loss Video Frames Server Error Propagation Internet Capacity Constraint Delay Constraint Client

  3. Error Control 5 4 3 2 1 3 Retransmission Video Frames Server Change Coding Parameter Internet Local Concealment Client 3 NACK

  4. Motivation • Frame loss degrades video quality • Feedback-based error control techniques use information from decoder to repair • Feedback indicates damage location. • Encoder and decoder cooperate in error control process. • Better than error control techniques where no interaction between encoder and decoder • Major techniques: RPS, Intra Update, Retransmission • Choice and Effectiveness depends on packet loss, RTT, video content and GOP size • No systematic exploration and comparison of impact of video and network conditions on the performance of feedback-based error control techniques

  5. The Dissertation • Analyze video quality with feedback based error control • Develop analytical models to predict quality of videos streamed with RPS NACK, RPS ACK, Intra Update or Retransmission • Conduct systematic study of effects of reference distance on video quality • Validate analytical models through simulations • Analysis of loss rate, round-trip time, video content, Group Of Pictures (GOP) • Determine choice between RPS NACK, RPS ACK, Intra Update or Retransmission • Publications • “Impact of Reference Distance for Motion Compensation Prediction on Video Quality”, MMCN07 • “An Analytic Comparison of RPS Video Repair”, MMCN08 • “Modeling RPS and Evaluating Video Repair with VQM”, IEEE Transactions on Multimedia, 2009, (to appear)

  6. Outline • Introduction • Background • RPS ACK • RPS NACK • Intra Update • Retransmission • Impact of Reference Distance on Video Quality • Analytical Models and Results • Model Validations • Conclusions

  7. Reference Picture Selection (ACK) • The decoder acknowledges all correctly received frames • Only the acknowledged frames are used as a reference • Error propagation is avoided entirely • Distance from reference frame is reference distance • Reference distance increases with round-trip delay • Coding efficiency decreases as reference distance increases • Video quality degrades as coding efficiency decreases 6 7 1 3 4 5 2 ACK(1) ACK(2) ACK(3)

  8. Reference Picture Selection (NACK) • The previous frame is used as a reference for encoding during the error-free transmission. • Reference distance is always 1 regardless of RTT • The decoder sends a NACK for the erroneous frame along with a reference frame number • Error propagation • Impact of loss increases with RTT 5 6 7 8 1 3 4 2 NACK(3)

  9. Intra Update NACK(4) • Upon receiving a NACK from the decoder, encodes the current frame with intra mode • Frame is independently encoded without using any information from previous frames • Coding efficiency is reduced because of intra coding Intra-coded 1 3 4 5 6 7 8 9 2

  10. Retransmission 3 NACK(3) Encoder • Retransmission of lost frames needs extra bandwidth • Packets arriving after their display times are not discarded but instead are used to reduce error propagation 5 6 7 8 9 1 3 4 2 Decoder 5 6 7 8 9 1 3 4 2

  11. Outline • Introduction • Background • Impact of Reference Distance on Video Quality • Hypothesis • Methodology • Results and Analysis • Analytical Models and Results • Model Validations • Conclusions

  12. Impact of Reference Distance on Video Quality • RPS selects one of several previous frames as a reference frame during compression • Distance from selected frame is reference distance • Higher reference distance, lower quality and vice versa • How reference distance affects video quality has not been quantified • A systematic study of the effects of reference distance on video quality • Data is needed for modeling RPS

  13. Hypothesis • Low Motion: • The similarities among frames are high; • More macro-blocks are inter-coded; • High motion: • The similarities among frames are low; • More macro-blocks are intra-coded; • The y-intersect is determined by motion and scene complexity. • High-motion video sequences starts with low quality, degrade slower. • Low-motion video sequence starts with high quality, degrade faster.

  14. Methodology • Select a set of non-compressed video clips with a variety of motion content. • All in YUV 4:2:2, CIF (352x288) • Each video sequence contains 300 video frames with a frame rate of 30 fps. • Change reference distances for each selected video sequence • Encode the video clips using H.264 • Measure video quality using • Peak-Signal-to-Noise-Ratio (PSNR) • Video Quality Metric (VQM) • Analyze the results.

  15. PSNR vs. Reference Distance The relationship between PSNR and reference distance can be characterized using a logarithmic function:

  16. VQM vs. Reference Distance The relationship between VQM and reference distance can be characterized using a linear function:

  17. Outline • Introduction • Background • Impact of Ref. Distance on Video Quality • Analytical Models and Results • Assumptions • RPS ACK • RPS NACK • Intra Update • Retransmission • Result & Analysis • Model Validations • Conclusions

  18. Assumptions 1 2 3 4 5 6 7 • Each GOB is independent from other GOBs in the same frame. • An independent video sub-sequence is referred to as a reference chain. • Each GOB is carried in a single network packet. • Reliable transmission of feedback messages are assumed. • Erroneously-decoded GOBs are repaired by local concealment. • Make no assumption on specific local concealment techniques. Assume independent packet loss with a random loss distribution. In this talk, GOB and Frame is exchangeable.

  19. Model Parameters

  20. Modeling of RPS ACK 1 3 4 5 2 ACK(1) ACK(2) The probability of decoding GOB (n) correctly using GOB (n-δ-i) as a reference: The probability of GOB (n) being successfully decoded is:

  21. RPS ACK Modeling (cont.) The expected video quality for n-th GOB:

  22. RPS NACK -- Model root C p 1- p GOB 1 [1] B p 1- p 1 3 4 5 2 GOB 2 (1) [1] p 1- p p 1- p A GOB 3 D [2] (1) (1) (2) NACK(1) NACK(2) p 1- p p 1- p 1- p p 1- p p GOB 4 (1) (2) (2) (3) [3] [1] (1) [1] • The probability of GOB (n) being successfully decoded: --- the probability of decoding GOB (n) correctly using GOB (n- δ -i) as a reference GOB Dependency Tree

  23. Intra Update -- Model root p 1- p C GOB 1 B p 1- p F GOB 2 p 1- p p 1- p A E GOB 3 p 1- p p 1- p 1- p p 1- p p D GOB 4 • The probability of GOB (n) being successfully decoded: -- the probability of decoding GOB (n) correctly using Intra coding Intra-coded 1 3 4 5 2 NACK GOB Dependency Tree

  24. Retransmission • Capacity constraint: • The n-th GOB in the reference chain being successfully decoded: • The expected video quality for GOB (n):

  25. Outline • Introduction • Background • Impact of Ref. Distance on Video Quality • Analytical Models and Results • Assumptions • RPS ACK • RPS NACK • Intra Update • Retransmission • Result & Analysis • Model Validations • Conclusions

  26. Analytic Experiments • Our analytical models consider a number of factors that may affect feedback-based repair performance: • Reference distance change • Loss probability • Round-trip time • Bitrate constraint • Video content • GOP Size • Select a set of video clips with a variety of motion content

  27. Quality versus Round-Trip Time RPS ACK RPS NACK • Quality degrades with round-trip time increase • NACK resistant to degradation with round-trip time for low loss • ACK degrades uniformly with round-trip time

  28. Quality versus Loss Rate RPS NACK RPS ACK • Quality degrades with loss rate increase • NACK degrades faster with high round trip times • ACK uniform degradation

  29. RPS NACK vs. RPS ACK • Above trend line, ACK better. Below trend line, NACK better • Crossover points for low-motion are higher than for high-motion • Error propagation more harmful to quality than reference distance

  30. Comparison • RPS NACK performs best in low loss • RPS ACK performs best in high loss • RPS ACK performs worst in low loss • Retransmission performs worst in high loss • Intra Update performs as well as RPS NACK as RTT increases RTT=80 ms RTT=240 ms

  31. Outline • Introduction • Background • Impact of Ref. Distance on Video Quality • Analytical Models and Results • Model Validations • Methodology • Results • Conclusions

  32. Validation -- Methodology 1(I) 2(P) 5(P) 6(P) 7(P) • Randomly drop controllable number of frames in input sequence based on given loss probability • Based on given round-trip time and randomly selected lost frames, regenerate video sequence • Encode video sequence generated in step 2 using H.264 • Measure average PSNR and VQM for encoded H.264 video sequence • Calculate average PSNR and VQM based upon video quality measured in step 4 RPS NACK, round-trip time = 2 frames, frame 3 is lost

  33. Validation – RPS NACK • Error bar represents 95% confidence interval • As loss probability or round-trip time increases, the variance is increased • Simulation results are consistent with values predicted by analytical model for both PSNR and VQM

  34. Outline • Introduction • Background • Impact of Ref. Distance on Video Quality • Analytical Models and Results • Model Validations • Conclusions

  35. Major Contributions • Systematic study of effects of reference distance on video quality for a range of video coding conditions • Two utility functions that characterize impact of reference distance on video quality based upon study • Modeling prediction dependency among GOBs for RPS NACK and Intra Update using binary tree • Analytical models for feedback-based error control techniques including Full Retransmission, Partial Retransmission, RPS ACK, RPS NACK and Intra Update • Simulations that verify accuracy of our analytical models • Analytic experiments over a range of loss rates, round-trip times and video content using our models

  36. Future Work • Explore and incorporate other existing video quality metrics or develop a new quality metric • Investigate how local concealment may affect the choice of feedback-based repair techniques • Investigate the impact of the extra bandwidth consumed by feedback messages on performance • Build a videoconference system that automatically adapts to the best repair techniques

  37. Conclusions • Degree of video quality degradation is affected by video content • High-motion video sequences starts with lower quality, degrade slower. • Low-motion video sequences starts with higher quality, degrade more rapidly. • Mathematical Characterization of the relationship between video quality and reference distance: • PSNR: • VQM: • Analytical models reveal: • RPS NACK performs best in low loss • RPS ACK performs best in high loss, worst in low loss • RPS NACK outperforms RPS ACK over a wider range for low motion videos than for high motion videos • Retransmission performs worst in high loss • Intra Update performs as well as RPS NACK as RTT increases

  38. Acknowledge • Prof. Claypool and Prof. Kinicki • Prof. Dougherty • Prof. Mayer-Patel from UNC at Chapel Hill • Faculty/Staff of Computer Science Dept., WPI • Huahui Wu, Mingze Li, Feng Li, and everyone from PEDS and CC groups • Attendees today • My Family

More Related