1 / 87

AVS – Technology, IPR and Applications

AVS – Technology, IPR and Applications. Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology, Peking University Oct.12, 2010. Video coding…. Compress high bit-rate digital video signal as low bit-rate stream

alan-joseph
Télécharger la présentation

AVS – Technology, IPR and Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AVS – Technology, IPR and Applications Prof. Huang Tiejun Audio and Video coding Standard Working Group of China Institute of Digital Media Technology, Peking University Oct.12, 2010

  2. Video coding… • Compress high bit-rate digital video signal as low bit-rate stream • SD: from ~250Mbps to < 5Mbps • HD: for >1Gbps to < 20Mbps • is the reason makes DTV possible at 1994 • and always a core concern for Digital TV system and any other digital video applications

  3. Video coding Standards • ISO/IEC recommended MPEG series standards • MPEG-1, MPEG-2, MPEG-4 • ITU-T recommended H.26X series standards • H.261, H.262, H.263, H.263+, H.263++ 1G standards 2G standards H.264 / MPEG-4 AVC China MPEG-China AVS

  4. MPEG-2: the 1st Generation • ISO/IEC 13818 / ITU-T H.262 @ 1994 • SDTV at ~5Mbps • HDTV at ~20Mbps • Makes DTV became reality • Europe DVB at 1997 • USA ATSC at 1997 • Japan ISDB at 1998 • China DTMB at 2006

  5. AVS – one of the 2nd Generation In Feb2006, AVS video issued as Chinese national standard GB/T 20090.2-2006 Effective from Mar2006 In Aug2006, DTMB issued as Chinese national standard GB/T 20600-2006 Mandatory from Aug2007 5

  6. 2ndG Video – ITU-T IPTVSelection • ITU-T IPTV FG decided to recommended the following video format at May2007 • H.264/ MPEG-4 AVC • MPEG-2 (for Legacy) • AVS – GB/T 20090.2 • SMPTE VC-1 • ITU-T Q6-WP3-SG16 meeting at last week (10July09) decided to release upper document as the ITU Technical Paper on “Media coding toolbox for IPTV”

  7. Video Compression Performance Improvement Compression Ratio AVS-?/MPEG-?/others-? 250 HD: 6-8Mbps SD:1-2.5Mbps AVS MPEG-4 AVC/H.264 VC-1 150 HD: 20 Mbps SD:4-6Mbps MPEG-2 50 MPEG-1 1991 1994 2003 2030

  8. Outline • Licensing problem for AV standards • AVS approach for dealing with IPR • AVS Video technologies • AVS Products • AVS Applications nowadays • Conclusion and Discussion

  9. Higher Performance, High Cost • Gold age is far away… • MPEG-1,MPEG-2 • H.261,H.262, H.263,H.263+/++ • Face to complicated situation nowadays • H.264,VC-1

  10. MPEG Standards • 1991：MPEG-1 • Royalty Free • 1994：MPEG-2 • DVD, DTV, Emmy Award • 2.5$ per device • 1996：MPEG LA—A licensing agency • 1999：MPEG-4 SP (Part 2) • 0.25 $/device, coding time based charge (2 cents/ hours) • Objected by AOL-Times Warner • 2003：H.264/MPEG-4 Part 10 AVC • Nov. 17,2003: Licensing policy – title based licensing • Objected by EBU（EBU Technical Statement-D 96-2003) • May.20,2004: Policy finalized, little revision

  11. First Effort to deal with patent issue for MPEG-2 • Leonardo Chiariglione《Patents and standards》 • http://www.chiariglione.org/ride/patents_and_standards.htm • If Gutenberg had lived in a culturally more advanced city like Venice or Florence, which had already started issuing litterae patentes ("open letters", whence the word "patent" comes from) since a few decades, he could have protected his invention with a patent. • Some time later, MPEG had to deal with MPEG-2 patents. Many companies had participated in the MPEG-1 development not so much because they necessarily had plans to make products in that space, but because they were keen to be around and help tune up the MPEG machine for the big MPEG-2 deal. • In January 1993 I took the initiative to invite to the Rome meeting Mike Smith, head of the Information Technology Task Force (ITTF), the group inside the ISO Central Secretariat in Geneva looking after JTC1 matters. Mike provided advice on how this difficult issue could be treated without trespassing the limits set by the ISO/IEC directives. • In the event, an MPEG-2 Patent Pool was established, managed by MPEG LA. This organisation acts as a "one-stop shop" for MPEG-2 Video and Systems patents (Audio is not part of the licensing), but there are a few companies licensing their technology separately. The result is reasonable (4$ - 2.5$/device)

  12. 网址 http://www.mpegla.com

  13. Participation Fees Where End User pays Subscription Title-by-Title Codec Manufacturers Internet Broadcast Participation Fees where Remuneration is from Other sources Free Television AVC/H.264 licensing structure Source:AVC/H.264 license terms http://www.avs.org.cn

  14. AVC/H.264 License Terms:Participation Fees • Where End User pays for AVC Video • Subscription(not limited by title) – 100,000 or fewer subscribers/yr = no royalty; > 100,000 to 250,000 subscribers/yr = $25,000; >250,000 to 500,000 subscribers/yr = $50,000; >500,000 to 1M subscribers/yr = $75,000; >1M subscribers/yr = $100,000 • Title-by-Title - 12 minutes or less = no royalty; >12 minutes in length = lower of (a) 2% or (b) $0.02 per title.  • Where remuneration is from other sources • Free Internet Broadcast (not title-by-title, not subscription) – no royalty during first term (through 2010; not greater than free television thereafter) • Free Television - (a) one-time $2,500 per transmission encoder or (b) annual fee starting at $2,500 for > 100,000 HH rising to maximum $10,000 for > 1,000,000 HH • Enterprise cap: $3.5M/yr 2006-07, $4.25M/yr 2008-09, $5M/yr 2010 • Royalties begin January 1, 2006 Source:AVC/H.264 license terms http://www.avs.org.cn

  15. Effort for 2G: RF JVT baseline AVS Strategy turned in 2003For JVT baseline + essential enhanced toolsTo a dependent, compact, patent clear standard

  16. MPEG LA专利池构成与风险分析

  17. AVS Mission • To develop a 2nd Generation video coding standard with same/better coding performance than others • Avoids licensing risk based on clearly analysis of related patents in last 50 years • To help DTV, IPTV and new media operators in China and out of China leave from more and more dangerous licensing mire • AVS was born at Mar.2002

  18. Outline • Licensing problem for AV standards • AVS approach for dealing with IPR • AVS Video technologies • AVS Products • AVS Applications nowadays • Conclusion and Discussion

  19. AVS Working Group • Set up by MII (Ministry of Information Industry) in June of 2002 • In charge of national standardization of audio-video coding for Chinese A/V industry • Also in charge of organizing the delegation of China National Body to MPEG(ISO/IEC JTC1/SC29/WG11)

  20. AVS membership

  21. 国内企业 Local Members

  22. MNC or Out of China外资企业和国外企业

  23. AVS Subgroups • System • Video • Audio • Digital Media Rights Management • Requirement • Conformance • IPR • Implementation

  24. AVS IPR Experts Group formation • Late 2003 the President of AVS asked that a group of experts be assembled to create a recommendation for an IPR policy for AVS • Create policies that: • reflect and respect China law and culture • reflect WTO requirements • reflect successful global practices • balance the rights of the inventor and needs of the implementers • innovative and forward looking • evolves with practice and law in China

  25. Mission of AVS IPR Group • Make a friendly environment for technical society (standard body), industry, and end user • Key issue to work out • Reforming the IPR policy making by only IP holder, to by IP holder + industry + consumer

  26. AVS IPR Experts Group • Experienced members:经验丰富的成员 • Patent Attorneys专利律师 • Technical Experts技术专家 • Contract Attorneys处理合同的律师 • Previous experience in setting up patent pools具有建立专利池相关经验的成员 • Members are from:成员来自 • Asia 亚洲 • Europe 欧洲 • North America 北美 • Members represent: 成员代表了 • Consumer Electronics 消费电子产品界 • Information Technology 信息技术界 • Microelectronics微电子产业界

  27. AVS IPR Experts group • Members • Broadcom • IBM • Intel • Matsushita/Panasonic • Microsoft • Nokia • Sony • Sun Microsystems

  28. Licensing Obligations • Non-Participant: • RAND RF • POOL • RAND • No License • Participant: • RAND RF • POOL • RAND • Join AVS • Sign Member Agreement • Declare default option • Participate in Subgroup • Commit to license • License options • RAND RF • POOL • RAND • Submit Contribution • Disclose patents • Declare license option • China Patent: • RAND RF • POOL • Worldwide Patent: • RAND RF • POOL • RAND

  29. Disclosure Obligations • When making a contribution • Unpublished, published and granted patents • Ongoing • When a patent is discovered • When a Patent Application is published • During Final Draft Standard Review • 90 days to declare licensing option, OR: • Default applies http://www.avs.org.cn

  30. AVS Licensing • 1 RMB concept • Only one charge, to device/software/chip • No charge to operator • No charge to end-user • Will be handled by AVS PPA • PPA is a non-profit organization • Founded By #1 institute of MII and ICT of CAS, in 2005 http://www.avs.org.cn

  31. Three Independent Organizations of AVS AVS Working Group AVS Patent Pool Council AVS Industry Alliance Technical Evaluation Licensing principle AVS Licensing Cooperation & Development IPR Principle Essential evaluation One-Stop license Applicatio Standard Better Implem. AVS Standards • AVS Industry Chain • ICs,software • Device,System • Operating,Service • AVS Industry Cluster • HDTV • IPTV • DTH • MobileTV • …… Public knowledge • Patent： • -Royalty Free • AVS Pool • RAND AVS Patent Pool R&D IPR Industry

  32. Amoisonic Broadcom Changhong Haier Huaju Huawei Langchao NSCC Panstar Semi SVA Skyworth Spectrum Communication Sunnywell TCL USstarcom ZTE AVS Industry Alliance http://www.avs.org.cn

  33. Outline • Licensing problem for AV standards • AVS approach for dealing with IPR • AVS Video technologies • AVS Products • AVS Applications nowadays • Conclusion and Discussion

  34. AVS1 Family

  35. MPEG-2Video Control Control Data Trans./ Quant Coeff. - IQ/IT Decoder Entropy Coding 0 Motion compensation Inter/Intra MV Motion estimation Oct.20, 2004

  36. AVSVideo Control Control Data Trans./ Quant Coeff. - IQ/IT Decoder Entropy Coding Intra-frame Prediction Deblock Motion compensation Inter/Intra MV Motion estimation Oct.20, 2004

  37. AVSVideo Control Control Data Trans./ Quant Coeff. - IQ/IT Decoder Entropy Coding Intra-frame Prediction Deblock Motion compensation Inter/Intra MV Motion estimation Oct.20, 2004

  38. Video Object Planes Variable Block-size Motion Compensation Macroblocks Transform Coding Multiple Reference DPCM P-frames B-frames DCT Generic B-pictures 1950 1999 ~1988 Advanced Deblocking Filter Hybrid Coding Scene Adaptive Coder Integer Transform Interlace Huffman Coding Motion Vector Prediction Block Motion Estimation Object-Based Scalability Error Resilience Modified from C Reader, 2002 Periods of key video coding tools contributed http://www.avs.org.cn

  39. Coding tools and its contribution Transform and bit allocation Color space Conversion Entropy coding ME and MC 2 * 3 * 3 * 3≈ 50 1st G 2nd G 2 * 3 * 6 * 3≈ 100

  40. AVS Video Tools • AVS1 video accepted 42 technical proposals from more than 200. • Major tools • Transform – 16bit-implemented 8x8 integer transform • Quantization and scaling - scaling only in encoder • Intra prediction – 5 modes • Motion compensation – 16x16/16x8/8x16/8x8 modes • Quarter-pel interpolation – 4-taps interpolation filter • Deblocking • Entropy coding • Minor tools • Motion vector prediction • Skipped mode and Coded block pattern • Adaptive scan Oct.20, 2004

  41. 1970s 1960s Ahmed, 1974 Andrews, 1968 Pratt, 1969 Chen, 1973 Fourier Transform Hadamard Transform Slant Transform Cosine Transform Pratt 1969 Huang 1966 Statistical Coding Zonal Coding Habibi, 1971 Tasto 1971 Block Classification Huang 1963 Max, 1960 Adaptive Block Coding Non-Uniform Quantizer Block Quantization Fano 1949 Seyler 1962 Huffman 1952 Chen 1981 VLC Video Variable-Length Coding VLC Scene Adaptive Coder Woods 1969 Dead-Zone Quantizer Uniform Quantizer Pratt 1969 Tescher, 1976 Threshold Coding Pratt, 1970 ZZ-Run-Length Coding Run-Length Coding Tescher, 1975 Zigzag Scan Tescher, 1977 C Reader, 2002 Rate Buffer Spatial Coding Tools

  42. Y RGB YUV U DCT 179 -2 0 0 0 0 0 0 4 0 0 0 0 0 0 0 V 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1430 -32 -4 -1 4 -4 0 -2 64 -2 -9 -4 -1 0 -3 -3 0 0 0 0 0 0 0 0 0 -7 3 -6 -6 0 -4 0 0 0 0 0 0 0 0 0 VLC/ CABAC 7 3 -16 -3 5 1 -2 -1 Quant -12 8 -8 -2 -1 -1 4 -1 -3 0 0 2 -2 0 2 0 3 -4 2 0 -4 6 3 0 173 184 190 198 194 192 192 192 5 -4 -1 -10 4 -1 -3 2 7 7 011010001011101... 180 187 182 189 193 191 194 196 C(h,v)=C(v)C(h)ΣΣ[P(x,y)COS{πv(2y+1)/16}COS{πh(2x+1)/16}] 180 180 181 182 184 189 196 199 y=0 x=0 178 174 174 175 180 186 175 177 17 18 24 47 99 99 99 99 16 16 16 16 16 16 16 16 8 16 19 22 26 27 29 34 16 11 10 16 24 40 51 61 17 18 24 47 99 99 99 99 16 16 16 16 16 16 16 16 8 16 19 22 26 27 29 34 16 11 10 16 24 40 51 61 170 171 176 184 185 175 172 181 C(0)=1/2√2 C(v)=C(h)=1/2 [v,h≠0] 18 21 26 66 99 99 99 99 16 16 22 24 27 29 34 37 16 16 16 16 16 16 16 16 12 12 14 19 26 58 60 55 18 21 26 66 99 99 99 99 16 16 22 24 27 29 34 37 16 16 16 16 16 16 16 16 12 12 14 19 26 58 60 55 173 167 170 176 178 181 179 177 24 26 56 99 99 99 99 99 16 16 16 16 16 16 16 16 14 13 16 24 40 57 69 56 19 22 26 27 29 34 34 38 24 26 56 99 99 99 99 99 16 16 16 16 16 16 16 16 14 13 16 24 40 57 69 56 19 22 26 27 29 34 34 38 165 167 165 168 174 171 175 181 47 66 99 99 99 99 99 99 16 16 16 16 16 16 16 16 14 17 22 29 51 87 80 62 22 22 26 27 29 34 37 40 47 66 99 99 99 99 99 99 16 16 16 16 16 16 16 16 14 17 22 29 51 87 80 62 22 22 26 27 29 34 37 40 162 163 163 158 162 169 172 175 99 99 99 99 99 99 99 99 16 16 16 16 16 16 16 16 22 26 27 29 32 35 40 48 18 22 37 56 68 109 103 77 99 99 99 99 99 99 99 99 16 16 16 16 16 16 16 16 22 26 27 29 32 35 40 48 18 22 37 56 68 109 103 77 99 99 99 99 99 99 99 99 26 27 29 32 35 40 48 58 16 16 16 16 16 16 16 16 24 35 55 64 81 104 113 92 99 99 99 99 99 99 99 99 26 27 29 32 35 40 48 58 16 16 16 16 16 16 16 16 24 35 55 64 81 104 113 92 26 27 29 34 38 46 56 69 99 99 99 99 99 99 99 99 16 16 16 16 16 16 16 16 49 64 78 87 103 121 120 101 26 27 29 34 38 46 56 69 99 99 99 99 99 99 99 99 16 16 16 16 16 16 16 16 49 64 78 87 103 121 120 101 27 29 35 38 46 56 69 83 99 99 99 99 99 99 99 99 16 16 16 16 16 16 16 16 72 92 95 98 112 100 103 99 27 29 35 38 46 56 69 83 99 99 99 99 99 99 99 99 16 16 16 16 16 16 16 16 72 92 95 98 112 100 103 99 DCT-based Compression

  43. AVS Innovation Example(1)DCT Transform Matrix in AVS Differs from the one in MPEG-2 (floating value) and H.264 (4X4 and 8X8) that was patented with similar performance (or even better in some cases )

  44. 1960s 1970s Limb 1975 Netravali 1979 Seyler 1962 Brofferio 1970 Motion Measurement PRA Motion Estimation Temporal Statistics Motion Modeling Seyler 1962 Candy 1971 Netravali 1979 Conditional Update (Spatial; Pixel) Motion Compensated DPCM (Pixel) 3D (Temporal) DPCM H.120 Harrison 1952 Rocca 1969 Koga 1981 Conditional Update (Spatial; Block) DPCM Motion Compensated DPCM (Block) Giorda 1975 Taki 1974 Variable – Size BMA Motion Est. BMA Motion Estimation Jain 1979 H.261 MPEG1 MPEG2 Etc. Schroeder 1970 Transform in DPCM Loop with MC Hybrid Coding Roese 1975 Netravali 1979 Reader 1973 Pratt 1968 Motion Compensated Transform Hybrid Coding (Transform) Conditional Update (Transform) Transform Coding (Intra) Jones 1978 Interlaced Field/Frame Coding MPEG2 C Reader, 2002 Temporal Coding Tools

  45. Skip, Direct and Symmetry Modes in B-frame • Same: • Use both forward and backward reference frame to predict • Different: • Skip: no coded MVD, no coded coefficient • Direct: no coded MVD, with coded coefficient • Symmetry: with 1 coded MVD and coded coefficient

  46. AVS B-frame Compared with Other Standards • AVS • Bi-prediction • Skip, direct • No motion vector • Symmetric • 1 coded motion vector • Higher compression ratio • Other Standards • Bi-prediction • Skip, direct • No motion vector • Other cases • 2 coded motion vector

  47. AVS vs. H.264 vs. MPEG-2

  48. AVS vs. H.264 vs. MPEG-2 (cont’d)

  49. AVS Video Profile and Level • JiZhun (基准)Profile (Dec. 2003) • 4 levels • 4.0 : up to Standard Definition with 4:2:0 • 4.2 : up to Standard Definition with 4:2:2 • 6.0 : up to High Definition with 4:2:0 • 6.2 : up to High Definition with 4:2:2 • AVS Zengqiang Profile (2008) • for Storage media such as movie • AVS Shensuo Profile (2008) • for video surveillance

More Related