A/V Quality in Television: Challenges & Solutions in the Digital World

1. The State of Television in Our Crazy World What’s Your A/V Quality?

2. We’re Digital • We regained the 700MHz spectrum • which we sold in 2008 to balance the Federal budget. • Now our cell phone coverage is way better • Everyone has those digital converter boxes for their TVs • Everything is fine, right?

3. Hmm, Maybe • Why doesn’t the picture fill the screen? • Why does the picture look blurry & blocky? • Sometimes I have no picture at all

4. What Happened? • With every change, there is a challenge • As SMPTE members, let’s see we can fix some of these problems

5. History – Let’s get into our WABAC1 Machine 1Wavelength Acceleration Bidirectional Asynchronous Controller

6. First, are we re-inventing the wheel? • Movies until the 1950s were shot in 35mm (4:3) • Movie attendance decreased in the 1950s • Introduction of wide-screen (1.67:1, 1.85:1, and 2.35:1) • Movie attendance decreased in the 2000s • Introduction of home theater

7. In the early 80s – most people didn’t notice • Just display it • But how? • Put borders • Anamorphic scaling • Center cut • Who cares? • It’s just an old movie after all

8. With Home Theaters - Film • We Care • Great picture quality • Sound fidelity • We Complain • Twitter • Facebook • Blog

9. What Has Happened to Viewers • People want to watch • at home • while mobile • in a theater • The experience must be great everywhere

10. Why doesn’t the picture fill the screen?

11. Different Devices and Formats • How do we display the formats • Label the format at the origin so we try to get it right • We need a plan

12. SMPTE Working Group - 2016 - AFD • Dr Kerns Powers – Sarnoff • He cut out little squares and tried to fit all of the aspects ratios on a grid • 1.33:1 (4:3) – (35mm) • 1.67:1 (European Wide) -(16mm) • 1.85:1 (American Wide) • 2.20:1 (70mm/PanaVision) • 2.35:1 (Anamorphic Wide Screen/CinemaScope)

13. The Discovery • 16:9 is the geometric mean • Everything could be converted to it and from it • Most new movies are shot in CinemaScope using Center Cut so the action stays in the 1.78:1 middle

14. SMPTE 2016 – Specification • Active Format Description • The AFD is most commonly on line 11 in the Video Index Information (HD recommendation) • Can be on line 9 through VANC-Y (last line)

15. AFD Right 4:3 Source, 16:9 TV, AFD #9 16:9 Source (4:3) Protected, 4:3 TV, AFD #15 16:9 Source, 4:3 TV, AFD #10

16. 4:3 Source to a 16:9 TV

17. 16:9 Source to a 4:3 TV

18. Next Step Labeling • Benefits of labeling the aspect ratio • Save bandwidth • Compress the original aspect ratio. Play the others. • Put the station logos and graphic overlays in the right place • Display the image properly • Get the up/down conversion correct

19. What Does this mean • AFD Must Be • Tightly coupled with the video asset • Frame accurate • Survive work-flow processing – compression, insertion, transmission, etc. • Updated when scaling • AFD Must Not be • In the Active Video • Set your VTRs, disk recorders, and frame grabbers properly • AFD Can be • Held as meta data per frame

20. Where does it lead • Every pixel may not light up • Black on the top/bottom for 16:9 content shown on a 4:3 TV • Black on the sides for 4:3 content shown on a 16:9 TV • Anamorphic Scaling can fix some of this with the correct AFD settings • The user must stop fiddling with the knobs • They will continue to fiddle until the picture automatically does the right thing • Set and preserve the AFD!

21. System Considerations • AFD must be carried through the entire workflow • Production • Processing • Distribution • Tape decks, Capture devices must • Preserve the VANC data • Not call it active video • If the AFD is missing, • Pick a safe, repeatable behavior

22. After All That, Does it Work? • NO! • At home, the consumer has too many knobs to twiddle • MAYBE! • Set-top boxes (STB or IRD) are becoming AFD aware • The Head-End (HE) can set the artistic action • This is part of the consumer electronic (CE) specifications • There is HOPE!

23. System Considerations • Test the Behavior • Ensure devices maintain AFD • Ensure scalars change AFD correctly • Ensure scalars react properly when AFD is missing • Monitor/Test the result • Send out a known A/V sequence • Record the processed A/V sequence • Verify that the results are as expected frame-by-frame

24. So, We Better Monitor!

25. Why Monitor? • Save Time • Prevent angry tweets, calls • Save Money • Prevent the negative press • Ensure advertisement is shown • Altruistic • Ensure quality

26. How Do We Monitor • Employ students to watch • After a while they look like this • Then they make mistakes

27. Why Automated Monitoring? • Saves Time • Finds errors that you may have missed • Logs errors for further analysis • Saves Money • The employee can do something more valuable

28. Problems seen while Monitoring • Up/Down scaling problems • Loss of A/V quality • Loss of A/V sync • Encoding, transmitting, decoding problems • Loss of ancillary, PSIP Data

29. Types of Broadcast Monitoring • Full Reference • No Reference • Reduced Reference

30. Full Reference Monitoring • Compares Reference and Processed A/V streams • Identifies all errors • Problem • Both streams must be co-located • Must be temporally and spatially aligned • Usage • Best in a lab or controlled environment

31. No Reference • Evaluates the processed sequence • Identifies • Blocking, contouring, blurring • Loss of high frequency • Quantization noise • Packet loss • Problem • Wrongly identifies “Artistic Content” • Usage • Used when the reference is not present • No need for synchronization

32. What is Artistic Content? • Does anyone remember Jay Leno doing this?

33. Reduced Reference • Compares some extracted information from the reference and processed • Identifies most errors • Problem • Extracted information must be co-located • Extracted information must be aligned • Usage • Best when a back-channel is present and when full reference cannot be used

34. What is SMPTE up to? • Working Group 22TV • In-service monitoring • Lip-Sync detection • Audio quality • Video quality • Based on finger printing technology • Reduced Reference

35. What is Finger Printing • Extract information from the audio and video separately • Resilient to data processing/ compression • Independent of resolution, frame rate, sampling • Independent of logo insertion, graphic overlay, etc. • Computed at the reference point when the data is known to be correct • Compared at the processing point

36. What is Finger Printing (continued) • Standards-based algorithm which everyone can use • Reduced Reference • Requires us to go back and finger print old movies and shows

37. What Finger Printing is NOT! • Watermarking • This is carried within the audio/video and can be destroyed by compression • Proprietary approach • All vendors must work together with broadcasters • A short-term fix • We must go back and finger print old movies

38. Hope • In-service dynamic monitoring • Check A/V quality • Check A/V offset (lip-sync) • Alert on errors • Errors not identified • AFD correct – or any other control variables • It does check if the picture has been scaled though

39. It Should See this Error • Can you see the error – yes I am playing it slow?

40. How about if I just show you this • Frames 1-4

41. Conclusion • You may not fill the entire screen • Live with it • Monitor everything to make sure it works • Find and fix problems before your customers LEAVE • SMPTE is working and with your help it can do more

42. What Does Video Clarity Do? • Full Reference Monitoring • We find all errors • Save sequences around every error • A/V Quality Analysis • Quantitative measurements • Perceptual measurement • View and hear