Last update: 2012/01/12

The Moving Picture Experts Group (MPEG)

 

MPEG achievements

 

1. Introduction 

MPEG is a working group of ISO, the International Organisation for Standardisation. Its formal name is ISO/IEC JTC 1/SC 29/WG 11. The title is: Coding of moving pictures and audio. The are of work assigned to it is: Development of international standards for compression, decompression, processing, and coded representation of moving pictures, audio, and their combination, in order to satisfy a wide variety of applications. The full MPEG terms of reference can be found here.

MPEG has developed the following standards

MPEG held its first meeting in Ottawa, ON (CA) on 88/05/10-12. Until January 2010 when it met in Kyoto (JP), MPEG has met as a group 75 times. The full list of meeting dates and venues can be found here.

2. MPEG-1 (ISO/IEC 11172)

The first MPEG work item was: Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s. In practice this meant a standard for efficient storage and retrieval of audio and video on compact disc. Parts 1 (Systems), part 2 (Video) and part 3 (Audio) of the standard were approved at the November 1992 meeting in London. The Systems part provide multiplexing and synchronisation support to elementary Audio and Video streams. The Video part provides efficient encoding of non-interlaced pictures with roughly VHS quality at 1,15 Mbit/s. The Audio part provides encoding of stereo audio with transparency (i.e. subjective quality similar to the original streo) at 384, 256 and 192  kbit/s per Layer II, II and III respectively. A more comprehensive description of the standard can be found here.

Part 4 of the standard "Conformance Testing" provides methods and reference bitstreams that can be used to assess conformance of a bitstream or of a decoder, was approved one year later. Part 5 of the standard "Reference Software" was approved in 1994. The latter contains the C-code implementation of a Systems multiplexer/demultiplexer and of encoders and decoders for Audio and Video.

This is the table of MPEG-1 parts:

MPEG-1 has been and is being used by many industries in a variety of products, services and applications and has triggered the start of a number of others.

There are many versions of a full MPEG-1 audiovisual players that can be used in a software environment. These utilise all 3 parts of Audio standard with Audio typically in Layer II. Many software packages exists that are capable of encoding audio and video in MPEG-1 and editing the resulting files.

The Video CD is a full application of MPEG-1 that is typically used to encode movies on 2 CDs. Several hundreds million hardware Video CD decoders have been sold worldwide and billions of Video CD discs have been printed. Software Video CD decoders are also avilable from multiple sources.

MPEG-1 Audio Layer III, also known as MP3, has been implemented in manifold ways. Many software packages exist to rip a track from a CD Audio and compress it in MP3. This has given rise to innovative ways of consuming music, such as the ability to create compilations to one's liking that can then downloaded to light non-mechanical MP3 players. With the arrival of MP3 the music world has been changed without recognition.

2. MPEG-2 (ISO/IEC 13818)

The Porto meeting in July 1990 was the first to address the MPEG-2 standard called "Generic coding of Moving Pictures and Associated Audio" and the Singapore meeting in November 1994 was the one that approved the first 3 parts: Systems, Video and Audio. Conformance (part 5) was approved one year later and Reference Software in 1996. The Systems part, in its "Transport Stream" version, provides support for efficient transmission over error-prone delivery systems, while the "Program Stream" version, similar to MPEG-1 Systems, is more useful for digital storage media. The Video part provides support for efficient coding of interlaced pictures ad different spatial resolution. The Audio part provides support to encoding of multi-channel audio in such a way that an MPEG-1. A more comprehensive description of the standard can be found here.

MPEG-2 has more parts than MPEG-1. Part 6 "Digital Storage Media Command and Control' or DSM-CC provide protocols for session set up across different networks and for remote control of a server containing MPEG-2 content. Part 7 "Advanced Audio Coding" or AAC provides a new multichannel audio coding that is not backward compatible with MPEG-1 Audio. Part 8 was intended to support video coding when samples are represented with an accuracy of more than 8 bits, but its development was discontinued when the interest of the industry that had requested it did not materialise. Part 9 "Real Time Interface" provides a standard interface between an MPEG-2 Transport Stream and a decoder.

13818 (MPEG-2)	Generic coding of moving pictures and associated audio
Part 1	Systems
Part 2	Video
Part 3	Audio
Part 4	Conformance testing
Part 5	Software simulation
Part 6	System extensions - DSM-CC
Part 7	Advanced Audio Coding
Part 8	VOID - (withdrawn)
Part 9	System extension RTI
Part 10	Conformance extension - DSM-CC
Part 11	IPMP on MPEG-2 Systems

Parts 1, 2 and 3 (this last one sometimes replaced with a proprietary solution) are used in the some 50 million digital television set top boxes and 10 million Digital Versatile Discs (DVD). Some MPEG-2 encoders are very costly professional equipment and some are very inexpensive PC board that are sold with video editing software.

Several examples of DSM-CC is widely used in set top boxes for satellite and cable. This part of the standard is also at the basis of provision of other set top box functionalities by other standards bodies and industry consortia. AAC has been adopted by Japan for a national digital television standard and by several manufacturers of secure digital music.

Over the years several amendments, i.e. extensions, of the standard have been developed. One of the most important is the 4:2:2 profile that extends the use of MPEG-2 into the television studio.

A number of patents are thought to be relevant for implementing the MPEG-2 standard. As MPEG is prevented by ISO rules to deal with patent issues, there is at least one organisation known that handles the licensing of MPEG-2 Systems and Video and another of MPEG-2 Audio.

3. MPEG-4 (ISO/IEC 14496)

Work on the MPEG-4 standard "Coding of audio-visual objects" began in July 1993 in New York, NY and the first set of standards (so-called version 1) was approved at the Atlantic City, NJ meeting in October 1998. A major extension of the standard (so-called version 2) was approved at the Maui, HI meeting in December 1999.

The first 6 parts of the standard correspond roughly to those of MPEG-2. The title of the first 5 is the same as MPEG-2, the title of the 6th is Delivery Multimedia Integration Framework. There are, however, a number of significant differences of content.

MPEG-4 enables the coding of individual objects. This means that the video information needs not be of rectangular shape as MPEG-1 and MPEG-2 Video assume. The same applies for audio, which provides all tools to encode speech and audio ad different rates and with different functionalities, including an extension of AAC. The systems part, therefore, contains, in addition to the traditional parts of MPEG-1 and MPEG-2 Systems, also the "composition function. Further, since a composition object can be also of synthetic nature, MPEG-4 Systems also contains standard technology to represent time-varying synthetic 3D information. A framework to deal with management and protection of rights arising from individual objects is also provided by MPEG-4 Systems. Finally a file format has been standardised. Part 5 is a complete software implementation of both encoders and decoders. Compared with the reference software of MPEG-1 and MPEG-2 whose value is purely informative, the MPEG-4 Reference Software has the same normative value as the textual parts of the standard. The software may also be used for commercial products and the copyright of the software is licensed at no cost by ISO/IEC for products conforming to the standard. Part 6 “Delivery Multimedia Integration Framework” (DMIF) provides a standard interface to access various transport mechanisms and an abstraction from the underlying delivery mechanism. A more comprehensive description of these MPEG-4 parts can be found here.

Part 7 “Optimised software for MPEG-4 tools” that provides examples of reference software that not just implement the standard correctly but also in optimised form. Part 8 “4 on IP framework” complements the generic MPEG-4 RTP payload defined by IETF as RFC 3640. Part 9 is “Reference Hardware Description” providing "reference software" in VHSIC Hardware Description Language (VHDL) for synthesis of VLSI chips.

Part 10, Advanced Video Coding (AVC), has been produced by the Joint Video Team (JVT) has roughly twice the compression capability of MPEG-2 and MPEG-4 Visual and contains Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

Scene Description (part 11) provides technologies for the functionality of “composing” different information elements in a “scene”. This is called Binary Format for MPEG-4 Scenes (BIFS).

The ISO Base Media File Format (part 12) is designed to contain timed media information for a presentation in a flexible, extensible format that facilitates interchange, management, editing, and presentation of the media. These may be ‘local’ to the system containing the presentation, or may be via a network or other stream delivery mechanism. Part 14 “MP4 File Format” extends the File Format to cover the needs of MPEG-4 scenes while part 15 “AVC File Format” supports the storage of AVC and MVC bitstreams.

Tools for coding synthetic visual information for 3D graphics are specified in Part 2 - Face and Body Animation and 3D Mesh Compression, Part 11 - Interpolator Compression - and Part 16 - which is a complete framework, called Animation Framework eXtension (AFX), for efficiently coding the shape, texture and animation of interactive synthetic 3D objects.

Streaming Text Format (part 17) defines text streams that are capable of carrying Third Generation Partnership Program (3GPP) Timed Text. To transport the text streams, a flexible framing structure is specified that can be adapted to the various transport layers, such as RTP/UDP/IP and MPEG-2 Transport and Program Stream, for use in media such as broadcast and optical discs. Part 18 Font compression and streaming provides tools for the purpose indicated by the title. Part 19 Synthesized Texture Stream defines the representation of synthesised textures.

Part 20 “Lightweight Application Scene Representation” (LASeR) provides composition technology with similar functionalities is provided by BIFS. Part 21 MPEG-J Extension for rendering provides a Java powered version of BIFS called MPEG-J.

Part 22 Open Font Formatr is the well-known OpenType specification converted to an ISO standard.

Part 23 Symbolic Music Representation provides a standard for representing music scores.

Part 24 Audio-System interaction clarifies some Audio aspects in a Systems environment.

Part 25 3D Graphics Compression Model defines an architecture for 3D Graphics related applications.

Part 26 Audio Conformance and Part 27 3D Graphics Conformance collects all specifications of audio and 3D graphics.

The full list of MPEG-4 parts is

14496 (MPEG-4)	Coding of audio-visual objects
Part 1	Systems
Part 2	Visual
Part 3	Audio
Part 4	Conformance testing
Part 5	Reference Software
Part 6	Delivery Multimedia Integration Framework
Part 7	Optimised software for MPEG-4 tools
Part 8	4 on IP framework
Part 9	Reference Hardware Description
Part 10	Advanced Video Coding
Part 11	Scene Description and Application Engine
Part 12	ISO Base Media File Format
Part 13	IPMP Extensions
Part 14	MP4 File Format
Part 15	AVC File Format
Part 16	Animation Framework eXtension (AFX)
Part 17	Streaming Text Format
Part 18	Font compression and streaming
Part 19	Synthesized Texture Stream
Part 20	Lightweight Application Scene Representation
Part 21	MPEG-J Extension for rendering
Part 22	Open Font Format
Part 23	Symbolic Music Representation
Part 24	Audio-System interaction
Part 25	3D Graphics Compression Model
Part 26	Audio Conformance
Part 27	3D Graphics Conformance

4. MPEG-7 (ISO/IEC 15938)

Work on MPEG-7 "Multimedia Content Description Interface" standard started at the April 1997 meeting in Bristol. MPEG-7 is an audio-visual information representation that is different from the previous MPEG standards in the sense that what is represented is not the information itself but the information about the information. MPEG-7 is an 11-part standard:

15938 (MPEG-7)	Multimedia Content Description Interface
Part 1	Systems
Part 2	Description Definition Language
Part 3	Visual
Part 4	Audio
Part 5	Multimedia Description Schemes
Part 6	Reference Software
Part 7	Conformance
Part 8	Extraction and Use of MPEG-7 Descriptions
Part 9	Profiles
Part 10	Schema definition
Part 11	Profile schemas
Part 12	Query Format

The technical content of the standard is as follows:

1. Systems provides the architectural framework of the standard, the carriage of MPEG-7 content and the binarisation of MPEG-7 content
2. Description Definition Language allows to create descriptors and description schemes
3. Visual provides standard descriptors and description schemes that are purely visual
4. Audio provides standard descriptors and description schemes that are purely audio
5. Multimedia Description Schemes provides standard descriptors and description schemes that are neither visual nor audio
6. Reference software has the same normative value as the MPEG-4 reference software and may be used for products at the same conditions
7. Conformance is the means to test an implementation or data for conformity.
8. Part 8 describes how feature extraction can be implemented
9. Part 9 provides a set of profiles
10. Part 10 the MPEG-7 schema definition
11. Part 11 collects the profile schemas
12. Part 12 specifies the interface between a requester for and a responder of multimedia content retrieval systems

5. MPEG-21 (ISO/IEC 21000)

Work on MPEG-21 "Multimedia Framework" standard started at the May-June 2000 meeting in Geneva. MPEG-21 provides a multimedia framework and sets out a vision for the future of an environment where delivery and use of all content types by different categories of users in multiple application domains will be possible.

MPEG-21 assumes that there are Users (anybody in the value network) and Digital Items (assembly of content) on which Users execute Actions that generate other Digital Items that can become object of Transactions. In order to make this possible a number of technologies are needed that fall under the following categories

1. Digital Item Declaration
2. Digital Item Identification
3. Intellectual Property Management and Protection
4. Terminals and Networks
5. Digital Item Management and Usage
6. Digital Item Representation
7. Event Reporting

The current table of MPEG-21 standards is

21000 (MPEG-21)	Multimedia Framework
Part 1	Vision, Technologies and Strategy
Part 2	Digital Item Declaration
Part 3	Digital Item Identification
Part 4	IPMP Components
Part 5	Rights Expression Language
Part 6	Rights Data Dictionary
Part 7	Digital Item Adaptation
Part 8	Reference Software
Part 9	File Format
Part 10	Digital Item Processing
Part 11	Evaluation Tools for Persistent Association
Part 12	Test Bed for MPEG-21 Resource Delivery
Part 13	VOID - (to MPEG-4 part 10)
Part 14	Conformance
Part 15	Event reporting
Part 16	Binary format
Part 17	Fragment Identification
Part 18	Digital Item Streaming
Part 19	Media Value Chain Ontology
Part 20	Contract Expression Language

1. Part 1 Vision, Technologies and Strategy lays down the scope and development plan of the project.
2. Part 2 Digital Item Declaration (DID) defin es a structure that can flexibly accommodate the many components of a multimedia object (resources, identifiers, metadata, encryption keys, licenses etc.).
3. Part 3 Digital Item Identification (DII), a standard to handle identifiers in Digital Items. 
4. Part 4 Intellectual Property Management and Protection (IPMP) Components specifies the component technologies to make elements of a Digital Item available in a form that can be processed by a machine.
5. Part 5 Rights Expression Language (REL) defines a language to express machine readable rights in a rich form that is comparable to the richness of the human language.
6. Part 6 Rights Data Dictionary (RDD) defines a standard semantics for verbs commonly used in the media environment, especially for use by Part 5..
7. Part 7 Digital Item Adaptation (DIA) specifies the syntax and semantics of the tools that may be used to assist in the adaptation of Digital Items, metadata and resources.
8. Part 8 Reference Software provides the reference software implementation of the relevant MPEG-21 standards.
9. Part 9 File Format defines a standard file format for Digital Items.
10. 10 Digital Item Processing (DIP) provides the tools to enable a Digital Item creator to suggest how a user can interact with the Digital Item.
11. Part 11 Evaluation Tools for Persistent Association provides the means to evaluate the performance of a given Persistent Association Technology to see how well it fulfils the requirements of the intended application.
12. Part 12 Test Bed for MPEG-21 Resource Delivery is a software test bed that has been developed to enable experimentation with different means of resource delivery.
13. Part 13 - withdrawn
14. Part 14 Conformance provides test methodologies and suites to assess the conformity of a bitstream (typically an XML document) and a decoder (typically a parser) to the relevant MPEG-21 standard.
15. Part 15 Event Reporting (ER) provides the technology to generate an event every time an action specified in the “Event Report Request” (ERR) contained in a Digital Item is made on a resource.
16. Part 16 Binary format references the technology specified in MPEG-B Part 1 “Binary MPEG format for XML” (BiM).
17. Part 17 Fragment Identification (FID) specifies a normative syntax for URI Fragment Identifiers to be used for addressing parts of a resource from a number of Internet Media Types.
18. Part 18 Digital Item Streaming (DIS) provides the technology to achieve this when the streaming mechanism employed is MPEG-2 Transport Stream and RTP/UDP/IP.

6. MPEG-A (ISO/IEC 23000)

 The current table of MPEG-A standards is

23000 (MPEG-A)	Multimedia Application Formats
Part 1	Purpose for Multimedia Application formats
Part 2	Music Player Application Format
Part 3	Photo Player Application Format
Part 4	Musical Slide Show Player Application Format
Part 5	Media Streaming Application Format
Part 6	Professional Archival Application Format
Part 7	Open Access Application Format
Part 8	Portable Video Player Application Format
Part 9	Digital Multimedia Broadcasting Application Format
Part 10	Video Surveillance Application Format
Part 11	Stereoscopic Video Application Format
Part 12	Interactive Music Application Format

1. Part 1 Vision, Technologies and Strategy down the scope and development plan of the MPEG-A project.
2. Part 2 “Music Player Application Format” enables an augmented experience of sound resources by providing an “extended MP3 format”.
3. Part 3 “Photo Player Application Format” enables an augmented experience of their photo resources by adding more information to the ubiquitous JPEG File Format, namely
4. Part 4 “Musical Slideshow Application Format” builds on top of the Music Player and the Photo Player Application Formats and is a superset of these two Application Formats.
5. Part 5 “Media Streaming Application Format” specifies how to use specific MPEG technologies to build a full-fledged media player for streaming governed content.
6. Part 6 “Professional Archival Application Format” provides a standard packaging format for carriage of digital multimedia content, metadata to describe context information related to digital multimedia content and the logical structure of how the digital multimedia content is stored, identification of processing tools and data protection and integrity tools, data governance tools, and data compression tools.
7. Part 7 “Open Access Application Format” defines a format designed for users who own rights to a piece of content and have an interest in releasing it in such a way that other users can freely access it but without making it public domain.
8. Part 8 “Portable Video Application Format” defines a format for the use of video files on portable devices giving users the possibility to use the content interactively.
9. Part 9 “Digital Multimedia Broadcasting Application Format” defines a standard file format that can be used to store in and exchange DMB content between DMB terminals.
10. Part 10 “Video Surveillance Application Format” provides a lightweight wrapper to the video content from the MPEG technologies, video coding, related metadata and file format, suitable for video surveillance.
11. Part 11 “Video Stereoscopic Application Format” enables more realistic experiences (with or without special glasses).

7. MPEG-B (ISO/IEC 23001)

 The current table of MPEG-B standards is

23001 (MPEG-B)	MPEG Systems Technologies
Part 1	Binary MPEG format for XML
Part 2	Fragment Request Unit
Part 3	XML Representation of IPMP-X messages
Part 4	Codec Configuration Representation
Part 5	Bitstream Syntax Description Language

1. Part 1 “Binary MPEG format for XML” (BiM) provides a standard set of generic technologies to transmit and compress XML documents, addressing a broad spectrum of applications and requirements.
2. Part 2 “Fragment Request Unit” specifies a technology enabling a terminal to request XML fragments of immediate interest.
3. Part 3 “XML Representation of IPMP-X Messages” provides an XML representation of the IPMP-X messages defined in MPEG-4 part 13 with extensions.
4. Part 4 “Codec Configuration Representation” provides a compressed digital representation of a video decoder and of the corresponding bitstream, assuming that the receiving terminal shares a library of video coding tools with the transmitter.
5. Part 5 “Bitstream Syntax Description Language” provides a normative grammar to describe, in XML, the high-level syntax of a bitstream.

8. MPEG-C (ISO/IEC 23002)

 The current table of MPEG-C standards is

23002 (MPEG-C)	MPEG Video Technologies
Part 1	Accuracy specification for implementation of integer-output IDCT
Part 2	Fixed point implementation of DCT/IDCT
Part 3	Auxiliary Video Data Representation
Part 4	Video Tool Library

1. Part 1 “Accuracy specification for implementation of integer-output IDCT” specifies the IDCT accuracy that is equivalent to or extends the IEEE 1180 standard which has been withdrawn.
2. Part 2 “Fixed-point 8x8 inverse discrete cosine transform and discrete cosine transform” specifies a particular fixed-point approximation to the ideal 8x8 IDCT and DCT function, fulfilling the 8x8 IDCT conformance requirements for the MPEG-1, MPEG-2 and MPEG-4 part 2 video coding standards.
3. Part 3 “Auxiliary Video Data Representation” specifies how auxiliary data such as pixel-related depth or parallax values, are to be represented when encoded by MPEG video standards in the same way as ordinary picture data.
4. Part 4 “Video Tool Library” contains a collection of descriptions of video coding tools, called Functional Units, as referenced in MPEG-B Part 4.

9. MPEG-D (ISO/IEC 23003)

 The current table of MPEG-D standards is

23003 (MPEG-D)	MPEG Audio Technologies
Part 1	MPEG Surround
Part 2	Spatial Audio Object Coding
Part 3	Unified Speech and Audio Coding

Part 1 “MPEG Surround” provides an efficient bridge between stereo and multichannel presentations in low-bitrate applications by supporting very efficient parametric coding of multi-channel audio signals, so as to permit transmission of such signals over channels that typically support only the transmission of stereo (or even mono) signals.

10. MPEG-E (ISO/IEC 23004)

 The current table of MPEG-E standards is

23004 (MPEG-E)	MPEG Multimedia Middleware
Part 1	Architecture
Part 2	Multimedia API
Part 3	Component Model
Part 4	Resource and Quality Management
Part 5	Component Download
Part 6	Fault Management
Part 7	System Integrity Management
Part 8	Reference Software and Conformance

1. Part 1 “Architecture” describes the M3W architecture and APIs.Part 2 “Multimedia API” specifies access to the functionalities provided by conforming multimedia platforms such as Media Processing Services (including coding, decoding and trans-coding), Media Delivery Services (through files, streams, messages), Digital Rights Management (DRM) Services, Access to data (e.g. media content) and Access to, Edit and Search Metadata.Part 3 “Component Model” specifies a technology enabling cost effective software development and an increase in productivity through software reuse and easy software integration. Part 4 “Resource and Quality Management” specifies a framework for resource management aiming to optimise and guarantee the Quality of Service that is delivered to the end-user in a situation where resources are constrained.Part 5 “Component Download” specifies a download framework enabling controlled download of software components to a device. Part 6 “Fault Management” specifies a framework for fault management with the goal to have a dependable/reliable system in the context of faults. These can be introduced due to upgrades and extensions out of the control of the device vendor, or because it is impossible to test all traces and configurations in today’s complex software systems.Part 7 “System Integrity Management” specifies a framework for integrity management with the goal to have controlled upgrading and extension, in the sense that there is a reduced chance of breaking the system during an upgrade/extension or to provide the ability to restore a consistent configuration.
2. Part 8 “Reference Software and Conformance” is the usual complement as with the other MPEG standards.

11. MPEG-M (ISO/IEC 23006)

The current table of MPEG-M standards is

23006 (MPEG-M)	MPEG Extensible Middleware
Part 1	Architecture and Technologies
Part 2	Application Programming Interface
Part 3	Reference Software and Conformance
Part 4	Elementary Services
Part 5	Service Aggregation
Part 6	Usage guidelines

12. MPEG-U (ISO/IEC 23007)

The current table of MPEG-U standards is

23007 (MPEG-U)	MPEG Rich Media User Interface
Part 1	Widgets
Part 2	Advanced User Interaction Interface
Part 3	Reference Software and Conformance

13. MPEG-V (ISO/IEC 23005)

The current table of MPEG-V standards is

23005 (MPEG-V)	Media Context and Control
Part 1	Architecture
Part 2	Control Information
Part 3	Sensory Information
Part 4	Virtual World Object Characteristics
Part 5	Data Formats for Interaction Devices
Part 6	Common Types and Tools
Part 7	Reference Software