100 likes | 182 Vues
The meeting aims to facilitate scientific exchange beyond subjective quality assessment, emphasizing objective metrics, with a focus on MPEG demanding, JPEG AIC initiative, and patent pool policies. The process involves testing new modules, dataset evaluation, profile definition, and validation, enhancing understanding of new algorithms' benefits.
E N D
Goals • Scientific exchange not only limited to subjective quality assessment • Links with other communities who have the opposite problem : share on objective metric not on subjective • Context : • MPEG demanding • JPEG AIC initiative
Joint Effort approach • Starting with generic model framework • Testing only new modules' benefit • Source code available • no training possibility • no need for secret sequences • each CE member can run its own tests and report results • better understanding of each new algo benefits
Policies, IPR issues • Among possibilities JVT (H264 …) example : • Contribution and IPR declaration and statement at the same time • Contribution Form : source code • Issue: reference software • Patent pool
Policies : dataset • Dataset (e.g. materials and subjective data) … two steps : • Common Working Dataset WD (known by contributors) to test the contributions • Possibility to present analysis based on non public dataset • Goal : identify limits of contribution in order to further improvement • Validation dataset VD at final stage (joined the WD after)
Process example • Step 1:Scope (limited), rules (performance evaluation), basic framework, call for Evidence • Step 2: WD completed • Step 3: contributions and evaluation on WD => decision (JEG labs) • Step 4: VD design and evaluation • T0 + 3 • T0 + 9 • T0 + 21 • T0 + 27
Possible Scope • Previous test plan • Current • Non existing
Profiles and levels • Name (NR, RR, Hybrid ..): VQM1-NR • Profile: use and context (coding errors, tranmission errors, accuracy and quality range) • Level: resolution, frame rate, coder … • Name: VQM1-H264 • Prolile: NR, RR, FR, Hybrid • Level: resolution, …
First test plan proposal • VQM1-H264 • Profile: NR V (PVS only after decoding) S (Stream only) H (PVS and Stream) • At least : Stream at NAL level • Possibility to define extended profile to cover protocol level parameters (UDP, RTP, DVB …) • Level: from HD to QCIF, transmission error at several bit rate
VQM1-H264 NRV profile Tools that have access to PVS only NRH profile NRS profile Tools that have access to stream only