The Moving Picture Experts Group

Multimedia API

Part number: 
Activity status: 

This standard specifies the MPEG-E API. A (non-exhaustive) list of such APIs contains

  • Media Processing Services (including coding, decoding and trans-coding),
  • Media Delivery Services (through files, streams, messages),
  • DRM Services,
  • Access to data (which could be media content)
  • Access to Metadata, Metadata Edit, Metadata Search

MPEG Multimedia Middleware (M3W) API


MPEG doc#: N8688
Date: October 2006
Authors: Johan Muskens & Jean Gelissen (Philips)


1.    Introduction

MPEG, a working group in ISO/IEC, has produced many important international standards (for example: MPEG-1, MPEG-2, MPEG-4, MPEG-7, and MPEG-21). MPEG feels that it is now important to standardize an Application Programming Interface (API) for Multimedia Middleware (M3W) that is based on a defined set of requirements [1]. This API provides a uniform view to a multimedia middleware platform that can be realized by a number of different vendors. In order to guarantee the interoperability of multiple parts of M3W realization provided by different parties, the same realization technology shall be used.

ISO/IEC 23004 Part 2 [2] – Multimedia API specifies interoperable access to the functionalities provided by conformant multimedia platforms. A (non-exhaustive) list of such APIs contains Media Processing Services (including coding, decoding and trans-coding), Media Delivery Services (through files, streams, messages), DRM Services, Access to data (which could be media content) and Access to Metadata, Metadata Edit, Metadata Search.

2.    Multimedia API

Many home multimedia products are already becoming multifunctional, supporting all kinds of media content, and sharing similar media and signal processing requirements such as compression, decompression, decryption, and reformatting media streams (MPEG-2, MPEG-4, DivX, etc.). When applications or middleware such as MHP, OCAP, DVD Navigator, or DVD Recording is ported to these products, it must control the audio and video tasks required to implement functionality such as tuning, de-multiplexing, media decoding, video mixing, etc. The M3W Multimedia API will remove the complexity of programming these underlying tasks by providing a multimedia control interface that is completely hardware independent from a programmer’s perspective.

The M3W Multimedia APIs offer a high-level API that is independent of the target hardware. By standardizing things that are common, the Multimedia APIs let developers focus on innovation and reduce time to market without limiting product differentiation. It simplifies the work of middleware and application developers because it is designed from what they need, rather than simply reflecting how a particular product’s hardware and software happens to be implemented.

The Multimedia APIs provides the following classes of functionalities:


  • Access to discovery services for Applications, Services and Multimedia resources
  • Multimedia Resource delivery services for Files and Streams
  • Access to Multimedia Resources like Metadata Resources, Media Resources and information on possible/available functional decomposition of requirements
  • Access to IPMP Services like Trust management Services, Authentication Services, Key management Services, Encryption Services and Decryption Services
  • Access to processing of Multimedia Resources like the Authoring of Multimedia Resources & Rights Expressions and Metadata resources
  • Access to processing of Media Resources like Coding, Transcoding, Filtering, Compositing, Splitting and Concatenating of Media Resources
  • Access to the rendering of Multimedia Resources including Rights Expressions, Metadata resources, Media Resources
  • Management of Multimedia Resources like Moving, Copying, Backup and Restore of Multimedia Resources

The Multimedia APIs are intended to give the requesting application or middleware access to multimedia functionalities provided by middleware components. The following diagram illustrates the position of the Multimedia APIs (MM API) in the overall M3W systems structure.

The Multimedia APIs provide unified access to the functionalities provided by the underlying platforms. It simplifies software development for the related products domain and promotes the reuse of multimedia middleware software components and by this improves the market place for independent (middleware and application) software providers.

 An example illustrating the direct use of the Multimedia API for a portable application is given below. The connection manager builds, on request of the application or other middleware, a media processing graph composed out of functionalities provided by the multimedia API like tuning, channel decoder, media decoder & splitter followed by parallel audio and video rendering functions.

3.    Summary

The M3W Multimedia API for multimedia applications simplifies the development of multimedia applications and middleware using a multimedia platform. It uses a simple programming model based on connected component networks called use cases that are set up by a Connection Manager and controlled through interfaces. Furthermore the M3W Multimedia API enormously simplifies portability of applications and middleware written for a realization of the M3W middleware to a different realization.

The M3W Multimedia API provides a flexible interoperable set of media functions suitable for use in multiple products with different capabilities and in multiple application domains. It can be deployed, by composing a set of the functionalities into a media processing graph, for a variety of product families like analogue television, a digital television, and digital set-top boxes and the like.

3.    References

[1]   The Multimedia Middleware (M3W) requirements are in the annex to the Multimedia Middleware (M3W) Requirements Document Version 2.0 (ISO/IEC JTC1/SC29/WG11 N6981). A Call for Proposals derived from these requirements was issued at the 70th MPEG Meeting in Palma, Mallorca

[2]   ISO/IEC 23004, Information Technology — Multimedia Middleware, The M3W standard.