INTERNATIONAL ORGANISATION FOR STANDARDISATION

ORGANISATION INTERNATIONALE DE NORMALISATION

ISO/IEC JTC1/SC29/WG11

CODING OF MOVING PICTURES AND AUDIO

ISO/IEC JTC1/SC29/WG11

N9586

Antalya, Turkey - January 2008

Source:     Video and ISG Subgroup

Status:      Draft

Title:         Whitepaper on Reconfigurable Video Coding (RVC)

Author:     Euee S. Jang, Jens Ohm, Marco Mattavelli

MPEG Reconfigurable Video Coding (RVC) is a new initiative motivated by the following observations:

• The video coding usage has changed a lot in the last two decades. -  Nowadays a terminal (or multimedia device) quite often should support multiple codecs (e.g.: JPG, MP1-V, MP1-A, MP2-V, AAC, MP4-V (ASP), AVC, SVC, etc… …) and their multiple profiles (Simple, Main, Core, High, etc … …). Although many of these codecs share common and/or similar coding tools there is currently no standard way at the level of the specification or the implementation to exploit such commonalities.
• MPEG Video Standards compete in the market place with non-MPEG standards. – After MPEG-2, MPEG video standards can not be called as the ‘unique’ solution for the market. MPEG-4 is now facing challenges in the video coding area from non-MPEG standards. Therefore MPEG must innovate the way it develops its standards so as to be able to include improvements and innovations and continue to stay ahead of competition.
• So far the specification of video coding standards has been done case by case providing textual and reference SW specifications, but without sufficient consideration for the problems of establishing an effective implementation flow process that, starting from the standard specification itself, provides efficient paths to software or hardware implementations. One of the main obstacles for the solution of  this problem is the growing complexity of the specification of recent MPEG standards. Although their normative specifications (typically in generic C or C++) are very precise, they present a number of limitations. Large portions of compression technology (i.e. video coding tools) are common across all MPEG standards, yet there is no direct way to recognize or exploit this commonality. Additionally, the sequential C/C++ descriptions do not expose the potential parallelism that is intrinsic to the algorithms constituting the codecs. They have also become excessively large (in terms of code size), making it extremely labor intensive, for example, to transform the sequential reference software into a VHDL implementation or to map it onto the new generation of multicore platform. In other words, the complex sequential C/C++ specifications no longer constitute a good starting point for the implementation processes of standard video codecs on current and future platforms.

Thus the goal of the new RVC standard is to offer a more flexible use and faster path to innovation of MPEG standards in a way that is competitive in the current dynamic environment, thereby enabling MPEG to continue serving the needs of the industry in terms of video coding standards. An additional challenge taken by the RVC framework is to provide a high level specification model for direct and efficient software and hardware synthesis [1].

To achieve this goal, MPEG by standardizing RVC specification intends to provide a framework allowing a dynamic development, implementation and adoption of standardized video coding solutions with features of higher flexibility and reusability.

The exploratory work of RVC was started in March 2004 at Munich meeting looking at all common elements among MPEG-1 and MPEG-2 Video, MPEG-4 Visual and AVC standards. As one result of the exercise, after more than two years of work, it was found that while the architecture of all codecs end up to be very similar and most of the basic blocks from the different standards presents the same data flows, their specification in terms of existing reference SW specifications remained completely different.

At 76^th Montreux MPEG meeting, a Call for Proposals aiming at collecting technologies for providing unified descriptions of the MPEG video technology has been issued, and at 77^th Klagenfurt meeting, proposals to build a MPEG Reconfigurable Video Coding framework have been received.

The development of the normative and informative (i.e. supporting technologies) components of RVC started at 77^th Klagenfurt meeting and is currently on going with continuous progresses at each meeting.   

The essential elements of the RVC framework are the following:

• A normative library of video coding tools, also called Functional Units (FUs) covering at the moment MPEG-2 simple profile and main profile, MPEG-4 SP, MPEG-4 AVC baseline profile, SVC baseline profile (the so called “MPEG Toolbox”). This library is specified with a textual normative specification and a corresponding reference SW. Such reference SW specification is provided using RVC-CAL as specification language for each library component. RVC-CAL is a language standardized by MPEG and used to define the behavior of dataflow components called actors, which is a modular component that encapsulates its own state such that an actor can neither read nor modify the state of any other actor. The only interaction between actors is via messages (known in RVC-CAL as tokens) which are passed from an output of one actor to an input of another. The behavior of an actor is defined in terms of a set of actions, transactional program fragments, at most one of which may be active at any point in time. The operations an action can perform are to consume (read) input tokens, modify internal state, produce output tokens, and interact with the underlying platform on which the actor is running.
• A normative language that provides the description of video codec representations called Decoder Description Language (DDL). This is an XML dialect that describes an interconnected network and parameterization of standard library components (i.e. FUs from the MPEG toolbox), which together represent a complete decoder. DDL can also be used recursively; that is, an actor may be defined as a composition of other actors, with the interconnections specified by DDL. In this case, the DDL itself declares input and output ports. DDL provides a facility for declaring parameters, and passing parameters to actors in the network. This is useful for declaring values that are constant for a particular instantiation of an actor, but may vary between different instantiations. An “abstract model” is constituted by the instantiation of a codec configuration using the Decoder Description Language and the MPEG Toolbox. Figure 1 depicts the process of instantiating an “abstract decoder model” in RVC.
• A normative language that specify the bistream syntax of a new configurations of a RVC decoder. Such language called RVC-BSDL is a profile of MPEG-21 BSDL that includes normative extensions used within the RVC framework. A RVC-BSDL schema is the information that is provided to be able to synthesize a bistream parser (component that is not existing in the RVC MPEG toolbox) to be connected with the RVC decoder configuration specified by DDL and by the MPEG toolbox.
  • Non-normative tools capable of: Validating the RVC-BSDL schema versus a bistream instantiation
  • Generating a CAL parser from a validated BSDL schema description
  • A simulator verifying and validating the behavior of the “abstract model”
  • A tool generating automatically C software from the abstract description model
  • A tool generating automatically hardware descriptions of the abstract model.

   

Figure 1. Graphical representation of the conceptual process of deriving a RVC « abstract decoder model » (i.e. an asynchronous data flow model made of a network of RVC-CAL actors) in the MPEG RVC specification.

Figure 2. Overview of the different tool families conceived to support the high level RVC codec specifications.

The RVC framework is covered by two parts of MPEG standards:

• ISO/IEC 23001-4 Codec Configuration Representation, which mainly covers the specification of the Decoder Description Language, associated bitstream representation, the abstract model and the formal description language used for the representation of their interfaces and data flows.
• ISO/IEC 23002-4 Video Tools Library, which contains the description of dedicated Functional Units,.

The current RVC workplan foresees (accompanied by a series of on-going Core Experiments to prove the concepts) Committee Drafts of both parts was approved in October 2007,  Final Committee Drafts for April 2008, Final Draft international Standards for October 2009, and International Standards for April 2009.