1. Perceived video quality measurement Muhammad Saqib Ilyas CS 584 Spring 2005

2. Agenda • Motivation • Subjective vs. objective measures • Mean Square Error based metrics • Recommended framework • Perceived video quality metrics • Structural distortion based metrics • Other video quality metrics • Conclusions

3. Motivation • Background in voice over IP • PSQM (Perceived Speech Quality Measurement) • Improving voice over IP quality over slow WAN links

4. Subjective measures • Mean opinion score • Most reliable • Slow • Expensive

5. MSE based metrics • Mean Square Error • Peak SNR

6. Recommended framework

7. Pre-processing • Temporal alignment • Compression, processing and transmission • Color space transformation • Visual blur in HVS • Linear space invariant lowpass characterized by PSF (Point Spread Function) • Light adaptation • Weber’s law

8. Contrast Sensitivity Function • Models sensitivity to different spatial and temporal frequencies • Bandpass in nature • Implementation: • Filter • Weighting factors after channel decomposition • Mostly lowpass filters used • More robust to changes in viewing distance • Temporal filters

9. Channel decomposition • Visual cortex neurons tuned to • spatial and temporal frequencies • Orientation • Direction of motion • Neuron modeled as 2D Gabor function • Collection of neurons modeled as octave band Gabor filter bank • Spatial spectrum sampled at • Octave intervals in radial frequency dimension • Uniform intervals in orientation dimension

10. Channel decomposition • Visual cortex neuron output saturates with increase in contrast • Typically modeled based on application and computational constraints • Sophisticated channel decomposition • Wavelet transforms • DCT

11. Error normalization and masking • Masking/facilitation • Presence of one image component will decrease/increase visibility of another image component at the same spatial location • Strongest when two signals have the same frequency components and orientation • Implementation • Gain control – space-varying visibility threshold for the particular channel

12. Error normalization and masking • Base error threshold for every channel is elevated to account for the presence of the reference signal • Elevated threshold used to normalize error signal into JND

13. Error pooling • Combines error values from various channels into one • Typically Minkowski pooling is used ei,k is the normalized and masked error of the k-th coefficient in the i-th channel β is a constant typically with a value between 1 and 4

14. Video distortion meter • Image quality assessment • Cognitive emulator • Asymmetric tracking • Humans detect quality transition from good to poor more readily

15. Multi-metric MPEG quality • Combines • Error sensitivity based metric • Blockiness detection

16. Digital Video Quality (DVQ) • Simplicity is key consideration • LC: Local contrast is ratio of DCT amplitudes to DC amplitude for a block • CM: Contrast masking

17. Others • Moving picture quality metric • Color space based metric • Blocking artifact based metric

18. Critique • Computational complexity • Memory requirement • Viewing resolution • Resolution of display device • Digital pixel – luminance value non-linear relationship • Viewing distance • Reliance on linear channel decomposition • Correlation between channels modeled using sophisticated masking techniques • Current masking models inaccurate

19. Structural Similarity Index Metric (SSIM) • Different kinds of image distortion have different perceived quality • Metrics discussed so far measure error • Error and structural distortion agree quite often • But the same amount of error may lead to different structural distortion • Bottom-up approach • Simulate the hypothesized functionality of the overall HVS

20. Reduced reference metrics Discussed in proposal presentation Based on temporal and spatial dissimilarity information

21. No reference / blind • Complications • Unquantifiable factors when reference is not available include but not limited to: • Aesthetics • Cognitive relevance • Learning • Visual context • Philosophy • All images/videos are perfect unless distorted during: • Acquisition • Processing • Reproduction

22. No reference • Determining the possible distortion introduced during these stages • Reference is “perfect” natural images/videos • Measured with respect to a model best suited to a given distortion type or application • E.g., natural images/videos do not contain blocking artifacts • To improve prediction • Some HVS aspects are also modeled • Texture and luminance masking

23. Other metrics • Marker bits hidden in video frames • Marker bits additionally transmitted on an aux channel • Watermarking

24. Conclusion • Perceptual quality measures not yet mature • Computational complexity a big hurdle especially for real-time applications • Accuracy of models also doubtful • where pin-point accuracy is not required • Less accurate RR or NR metrics may be used in