The Moving Picture Experts Group

Multimedia Description Schemes

Part number: 
Activity status: 

MPEG-7 Multimedia Description Schemes (MDS)


MPEG doc#: N7418
Date: July 2005
Author: José M. Martínez (UAM-EPS-GTI, ES)

MPEG-7 Multimedia Description Schemes (ISO/IEC 15938-5) specifies the different description tools that are not visual and audio ones, that is, generic and multimedia ones. The MDS specification comprises therefore the major number of description tools of the MPEG-7 standard from the basic structures allowing to create the structure of the description to the description of collections and user preferences, including also the hooks for adding the audio and visual description tools defined in ISO/IEC 15938-4 and ISO/IEC 15938-3 respectively.

MPEG-7 MDS Description Tools (either Descriptors or Description Schemes) can be grouped in terms of the functionality they provide, although in practice they are used in a structured way making use of the topo-level elements and the selected profile. The following figure presents the different functional groups of MDS description tools.

The following list present short descriptions of the different functional groups:

  • The Basic Elements are the generic entities that are used as building blocks by different Description Tools. They include basic datatypes (e.g., numbers, matrices, vectors, country), links and locators (time, media locators and referencing tools), and other basic Description Tools for places, persons, textual annotations, controlled vocabularies, etc.
  • The Schema Tools comprise the tools (root and top-level elements) for wrapping Description Tools for use by applications and the package tools for grouping related Description Tools into folders for personalizing applications.
  • The Content Description Tools are the tools for the representation of perceivable information, comprising structural aspects (structure Description Tools) and conceptual aspects (semantic Description Tools). The Structure Description Tools allow the description of the content in terms of spatio-temporal segments organized in a hierarchical structure (allowing the definition of a table of contents or an index). Audio, visual, annotation and content management Description Tools can be attached to the segments to describe them in detail. The Semantic Description Tools allow the description of the content from the viewpoint of real-world semantics and conceptual notions: objects, events, abstract concepts and relationships. The semantic and structure Description Tools can be further related by a set of links.
  • The Content Description Tools are the tools that are related to classical archival of content, providing information about media features, creation and usage of the multimedia content. The Media Description Tools allow the description of the storage media, the coding format, the quality and the transcoding hints for adapting the content to different networks and terminals. The Creation Description Tools allow the description of the creation process (title, agents, materials, places, dates, ...), classification (genre, subject, parental rating, languages, ...) and related materials. The Usage Description Tools allow the description of the conditions for use (rights, availability, ...) and the history of use (financial results, audience, ...).
  • The Content Organization Description Tools allow the creation and modeling of collections of multimedia content and descriptions. Each collection can be described as a whole by their attribute values characterized by models and statistics.
  • The Navigation and Access Description Tools allow the specification of summaries, partitions and decompositions, and variations of the multimedia content for facilitating browsing and retrieval. Summaries Description Tools provide both hierarchical and sequential navigation modes in order to provide efficient preview access to the multimedia material. Partitions and Decompositions Description Tools provide the tools for multi-resolution and progressive access in time, space and frequency. Variations Description Tools provide the tools for describing pre-existing views of the multimedia content: summaries, media modalities (e.g., image to text), scaled versions, ...
  • The User Interaction Description Tools allow the description of user preferences (for personalized filtering) and usage history pertaining to the consumption of the multimedia content. These Description Tools have been also adopted within MPEG-21 DIA (ISO/IEC 21000-7).

Most of the MPEG-7 MDS Description Tools are specified in ISO/IEC 15938-5 (2001), but there are additionally one corrigenda –ISO/IEC 15938-5 Cor.1 (2004)- and two amendments:

  • ISO/IEC 15938-5 Amd.1 (2003), specifies some new basic elements and additional Linguistic Description Tools.
  • ISO/IEC 15938-5 Amd.2 (2004), specifies extensions to the User Interactions Descriptions Tools for compatibility with MPEG-21 DIA


ISO/IEC 15938-5:2003, “Information Technology – Multimedia Content Description Interface – Part 5: Multimedia Description Schemes”, 2003.

ISO/IEC 21000-7:2004, “Information Technology – Multimedia Framework – Part 7: Digital Item Adaptation”, 2004.

J.M. Martínez, R. Koenen, F. Pereira, “MPEG-7: the generic Multimedia Content Description Standard”, IEEE Multimedia, 9 (2):78-87, April-June 2002.

J. M. Martínez, “MPEG-7: Overview of Description Tools”, IEEE Multimedia, 9 (3):83-93, July-September 2002.