The Effectiveness of a QoE - Based Video Output Scheme for Audio-Video IP Transmission

1. The Effectiveness of a QoE - Based Video Output Scheme for Audio-Video IP Transmission Shuji Tasaka, Hikaru Yoshimi, Akifumi Hirashima, Toshiro Nunome Department of Computer Science and Engineering, Graduate School of Engineering, Nagoya Institute of Technology ACM Multimedia 2008

2. Issue • Conceal the impairment typified by packet loss, error, and delay in IP networks • Two ways these impairments can be remedied at the receiver are: • Error-concealment • Video frame skipping • These techniques approach this issue with a unique tradeoff • Temporal vs. Spatial • Typically the benefits are exclusive

3. Scheme • Switching between error Concealment and frame Skipping (SCS) utilizes this tradeoff between spatial and temporal quality to cope with video packet loss • SCS aims to improve Quality of Experience (QoE) as it depends on both spatial and temporal quality by mixing error concealment and frame skipping

4. Outline • Introduction • SCS theory • Aims of study • QoE measurement • QoE estimation • Threshold selection • Conclusions

5. Principle • SCS switches from error-concealment to frame skipping when a percentage of video slices error-concealed in a frame (Rc) exceeds a threshold value (Th) • Frame skipping continues until a new intra-coded frame is decoded • Optimal Th is dependent on the content type

6. Error Concealment Strategy • I Frames • Missing block is interpolated from its neighboring blocks in the current frame • P Frames • Missing block is replaced by the corresponding block in the previous output frame • Instead of motion copy

7. Outline • Introduction • SCS theory • Aims of study • QoE measurement • QoE estimation • Threshold selection • Conclusions

8. Previous Limitations • No audio accompanies the video • Output quality determined by PSNR, which is a QoS parameter (non-perceptual) • PSNR evaluates no temporal qualities • No real-time estimation of QoE • Full Reference (objective QoE) models compare stream with the original though not in real-time

9. Aims • Find QoE through subject testing • Want to estimate QoE through estimation equations, which can allow threshold values to be evaluated in real-time • See how accurate estimations are to the subjective measurements • Measure the percentage of the selected threshold by way of multiple regression lines

10. Outline • Introduction • SCS theory • Aims of study • QoE measurement • QoE estimation • Threshold selection • Conclusions

11. Setup • 6 videos • Video dominant (sports) • Audio dominant (music video) • Lower frame rate (animation) • The second of each with greater Temporal perceptual Information (TI) value • Recorded streams output by the media recipient were used as stimuli (432 total): • Six different levels of average web traffic (20, 30, 40, 50, 75, 100) • Lossy environments > 20 web client processes • Four Th (100, 40, 20, 0) • Three picture patterns (I, IPPPP, IPPPPPPPPPPPPPP)

12. Setup • Based on an interval scale derived from: • Rating scale • Law of categorical judgment • Impairment rating-scale: 5 Imperceptible 4 Perceptible, but not annoying 3 Slightly annoying 2 Annoying 1 Very annoying • Stimuli which gave large errors of Mosteller’s test are removed • Ensures goodness of fit (psychological scale)

13. Sport 2 (I) • Pure frame skipping achieves the highest QoE • All other contents performed similarly for I • Notice that no other frames are dropped with only (I)

14. Sport 2 (IPPPP) • In almost all lossy environments, Th 20% & 40% provide higher QoE than 0%

15. Animation 2 (IPPPP) | Music Video 1 (IPPPP) • Th 0% is better than nonzero values in lossy environments • Reason: Animation has less than 30 frames & the music video is audio dominant with low video motion

16. Music Video 1 (IPPPPPPPPPPPPPP) • At 0%, Th is now less successful • Greater QoE at lossy envioronments with a higher Th, as frame skip loses all succeeding P frames

17. Outline • Introduction • SCS theory • Aims of study • QoE measurement • QoE estimation • Threshold selection • Conclusions

18. QoE Estimation • Psychological scale is achieved by QoS mapping between the user-level and the application level • Mapping is accomplished via multiple regression analysis • QoS parameters = independent • Psychological scale = dependent

19. Application level QoS parameters • These parameters represent both temporal and spatial quality • To best estimate QoE, the variables chosen chosen should have low cross-correlations

20. QoE estimation • Principle component analysis allows us to find cross-correlations between introduced independent variables • Variables that correlate strongly (a cumulative contribution rate> 90%) are placed in one of the five classes (A-E) • Then we calculate a multiple regression line for every combination • The line chosen is the one with the greatest multiple correlation coefficient adjusted for degrees of freedom (R*) • Picture pattern is not taken into account • One parameter from each class must be selected

21. QoS parameter classes based on PCA

22. Accuracy of estimation

23. Outline • Introduction • SCS theory • Aims of study • QoE measurement • QoE estimation • Threshold selection • Conclusions

24. Setting the threshold • In order to maximize QoE, the appropriate threshold must be chosen • This is accomplished by implementing a “learning period” • Th is first set to 100%, while the formal Th is computed by estimating the psychological scale values for each threshold • At the end of the learning period, the Th with the max psychological value is chosen • If there are two Th’s with the same value, the larger Th is chosen

25. Sport 2 (I) • Pure frame skipping is chosen in a lossy environment • This was also true in subjective tests

26. Sport 2 (IPPPP)

27. Music Video 1 (IPPPP)

28. Music Video 1 (IPPPPP…)

29. Limitations Realized • Content contains audio and video • QoE is a perceptual QoS • Psychological scale uses QoS parameters • In QoE estimation, QoS parameters account for spatial and temporal characteristics • Estimation with learning period provide real-time QoE assessment

30. Conclusions • The effectiveness of estimation and human subject testing for SCS’s was examined • Subject testing of these estimated SCS should be done • With picture pattern’s I and IPPPP the measured and estimated QoE’s are quite similar to each other when utilizing nonlinear multiple regression analysis • Picture pattern I favored frame dropping, while IPPP… favored error concealment • Threshold value selection must be further investigated • The effects of motion copy should be used in future tests • Is there a need for QoE estimation?

31. Thank you