1 / 32

MPEG Video Coding II — MPEG-4, 7 and Beyond

MPEG Video Coding II — MPEG-4, 7 and Beyond. Outline. Overview of MPEG-4 Object-based Visual Coding in MPEG-4 Synthetic Object Coding in MPEG-4 MPEG-7 and Further Exploration. Review: Object-based Visual Coding. Review: Synthetic Object Coding. MPEG Standards. MPEG-1 :

lada
Télécharger la présentation

MPEG Video Coding II — MPEG-4, 7 and Beyond

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MPEG Video Coding II— MPEG-4, 7 and Beyond

  2. Outline Overview of MPEG-4 Object-based Visual Coding in MPEG-4 Synthetic Object Coding in MPEG-4 MPEG-7 and Further Exploration

  3. Review: Object-based Visual Coding

  4. Review: Synthetic Object Coding

  5. MPEG Standards • MPEG-1: • Storage of moving picture and audio on storage media (CDROM) • MPEG-2: • Digital television • MPEG-4: • Coding of natural and synthetic media objects for multimedia applications • MPEG-7???

  6. Problem Definition • How to find the desired content in a multimedia database? • Keywords • “Apple”, “IPAD” • Semantic • “IPAD was announced by Apple” • Query by example • “Find objects like these… ” • Hierarchical approach • Low-level features, e.g., texture, color, motion, etc. and domain specific high-level information • Try http://images.google.com

  7. Query • Query by example

  8. MPEG-7 • The main objective of MPEG-7 is to serve the need of audio-visual content-based retrieval (or audiovisual object retrieval) in applications • Growth of digital aduiovisual information • To find a video clip of Clinton’s speech on Internet • To find a motocycle like the terminator II • To record TV programs that a viewer like • “Multimedia Content Description Interface” to standardize the description of various types of multimedia content • Still pictures, graphics, 3D models, audio, speech, video, and composition information • Special cases: facial expression, personal characteristics

  9. MPEG-7 (cont.) • To enable fast and efficient search and retrieval • From text-based search (e.g., keywords) to content-based search (e.g., color, motion) • MPEG-1/2/4 vs. MPEG-7 • MPEG-1/2/4: Representation of data • MPEG-7: Representation of “metadata” (information about data) • MPEG-7 may use the shape description in MPEG-4 or the motion vector field in MPEG-1/2

  10. Scope of MPEG-7 ??? Feature extraction is outside MPEG-7 Search and query are outside MPEG-7

  11. Scope of MPEG-7 (cont.) • “Standardize the minimum” • Analysis should not be standardized • Can keep improving • Room for competition • Search engine should not be standardized • Application dependent • Room for competition • Description for the same content may be different for different use domains and different applications

  12. Concepts in MPEG-7 • MPEG-7 has developed Descriptors (D), Description Schemes (DS) and Description Definition Language (DDL). The following are some of the important terms: • Feature — characteristic of the data, e.g., color, motion • Descriptor — definition (syntax and semantics) of the feature, e.g. histogram, motion vectors • Description Scheme (DS)— specification of the structure and relationship between Ds and between DSs. • Description • An instantiation of a DS • Combination of Descriptors and DS’s • DDL —A language to define, modify, and combine DS’s • So, MPEG-7 will standardize a set of descriptors and DS’s, DDL, a scheme for coding the descriptions

  13. Applications Supported by MPEG-7 • Fig. 1: Possible Applications using MPEG-7.

  14. Description Definition extension Language Definition Tags DS1 < scene id=1> < time> .... D1 DS2 Instantiation < camera>.. < annotation </scene> D3 D2 DS4 D3 Structuring DS3 D6 D4 D5 Encoding & Description Schemes Delivery 10101 1 0 Example Relations Between D’s DS’s D2 D1 D7 D8 D6 D4 D10 D9 D5 Descriptors: (Syntax & semantic of feature representation)

  15. Descriptors (D) • Localization • Region Locator • Spatio-Temporal • Locator • Color Descriptors Dominant Color Scalable Color Color Layout Color Structure • Texture Descriptors • Shape Descriptors Region Shape Contour Shape 3D Shape • Motion Descriptors for Video • Camera Motion • Motion Trajectory • Parametric Motion • Motion Activity • Other • Face Recognition

  16. Camera Motion • Fig. 2: Camera motions: pan, tilt, roll, dolly, track, and boom.

  17. Motion Trajectory • Used to define the localization, time, and space of an object • Consists of a set of key points along with a set of optional interpolating functions used to describe the path of the object between two key points

  18. Description Scheme (DS)

  19. Description Scheme (DS) • Basic elements • Basic datatypes: types provide a set of extended data types and mathematical structures such as vectors and matrices, which are needed by the DSs for describing • Schema tools: assist in the formation, packaging, and annotation of MPEG-7 descriptions • Content Management • Creation and production • Title, creator, creation locations, creation time, texture annotations • Subject, purpose, language, … • Media coding, storage and file formats • Format, compression, … • Content usage • Usage right, usage records, • Content Description • Structural Description. A Segment DS, for example, can be implemented as a class object. It can have five subclasses: Audiovisual segment DS, Audio segment DS, Still region DS, Moving region DS, and Video segment DS. The subclass DSs can recursively have their own subclasses.

  20. MPEG-7 for Content Description

  21. Fig. 3: MPEG-7 video segment.

  22. Content structural aspects (Segment Relationship DS graph) Temporal segments

  23. Content Organization • Collections: Collection Structure DS • Model: Model DSs provide tools for modeling the attributes and features of AV content

  24. Description Scheme (DS) • Navigation and access • Summaries. • Partitions and Decompositions. • Describe different decompositions of the audio-visual signal in space, time and frequency • Describe different views of the audio-visual data, which is important for multi-resolution access and progressive retrieval • User Interaction • User Preference

  25. Description Definition Language (DDL) • MPEG-7 adopted the XML Schema Language initially developed by the WWW Consortium (W3C) as its Description Definition Language (DDL).

  26. How to Build and Deploy MPEG-7 Descriptions A description A Description Scheme (structure) . + in DDLanguage A set of Descriptor Values (instantiation of a Descriptor for a given data set) MPEG-7 Description Tools are a library of standardized Descriptions and Description Schemes Adopting the XML Schema as the basis for the MPEG-7 DDL and the resulting XML-compliant instances (Descriptions in MPEG-7 textual format)

  27. Example Applications • Digital libraries • E.g., image/video catalog, musical dictionary • Multimedia directory services • e.g., yellow pages • Broadcast media selection • e.g., radio channels, TV channels • Multimedia authoring • e.g., personalized news services, digital photo/video albums

  28. Example Uses (cont. ) • Movement • On a give set of objects, describe movements and relations between objects • Retrieve a list of animations fulfilling the described temporal and spatial relations • Scenarios • On a given content, describe actions and get a list of scenarios where similar actions happen • Voice • Using an excerpt of Pavarotti’s voice to retrieve a list of Pavarotti’s records or video clips

  29. Further Exploration • Text books: • Multimedia Systems, Standards, and Networks by A. Puri and T. Chen • The MPEG-4 Book by F. Pereira and T. Ebrahimi • Introduction to MPEG-7: Multimedia Content Description Interface by B.S. Manjunath et al. • Web sites: • The MPEG home page • The MPEG FAQ page • Overviews, tutorials, and working documents of MPEG-4 • Tutorials on MPEG-4 Part 10/H.264 • Overviews of MPEG-7 and working documents for MPEG-21 • Documentation for XML schemas that form the basis of MPEG-7 DDL

  30. IBM Image Retrieval System based on MPEG-7 http://mp7.watson.ibm.com/

  31. Backup

  32. Texture regularity (periodic to random) Coarseness (grain to coarse) Directionality (/300)

More Related