Conformance testing

MPEG-4 Conformance

MPEG doc#: N7649
Date: Oct. 2005
Author:

Parts 1, 2 and 3 of ISO/IEC 14496 specify a multiplex structure and coded representations of audio-visual information. Parts 1, 2 and 3 of ISO/IEC 14496 allow for large flexibility, achieving suitability of ISO/IEC 14496 for many different applications. The flexibility is obtained by including parameters in the bitstream that define the characteristics of coded bitstreams. Examples are the audio sampling frequency, picture size, picture shape, picture rate, bitrate parameters, synchronisation timestamps, the association of bitstreams and synthetic objects within objects, the association of objects within scenes, the protection of bitstreams, objects and scenes. Part 6 of ISO/IEC 14496 specifies a framework for uniform delivery of MPEG-4 content according to the requested associated QoS, irrespective of their location and the transport technology.  This part of ISO/IEC 14496 specifies how tests can be designed to verify whether bitstreams and decoders meet the requirements as specified in parts 1, 2, 3 and 6 of ISO/IEC 14496 and allow interoperability with remote terminals in interactive, broadcast and local (with stored contents) sessions. These tests can be used for various purposes such as:

• manufacturers of encoders, and their customers, can use the tests to verify whether the encoder produces bitstreams compliant with parts 1, 2 and 3 of ISO/IEC 14496.
• manufacturers of decoders and their customers can use the tests to verify whether the decoder meets the requirements specified in parts 1, 2 and 3 of ISO/IEC 14496 for the claimed decoder capabilities.  manufacturers and customers of terminals supporting interactive, broadcast and local sessions over a multitude of transport protocols and networks, can use the tests to verify whether the claimed functionalities are compliant with ISO/IEC 14496-6.
• manufacturers of test equipments, and their customers can use the tests to verify compliance with parts 1, 2 and 3 of ISO/IEC 14496.

This part of ISO/IEC 14496 specifies how tests can be designed to verify whether bitstreams and decoders meet requirements specified in parts 1, 2 and 3 of ISO/IEC IEC 14496 and for part 6 of ISO/IEC 14496 it specifies how tests can be designed for bitstream delivery over various delivery technologies in an interoperable transparent manner to parts 1, 2 and 3. In this part of ISO/IEC 14496, encoders are not addressed specifically. An encoder may be said to be an ISO/IEC 14496 encoder if it generates bitstreams compliant with the syntactic and semantic bitstream requirements specified in parts 1, 2 and 3 of ISO/IEC 14496.

Characteristics of coded bitstreams and decoders are defined for parts 1, 2 and 3 of ISO/IEC 14496. The characteristics of a bitstream define the subset of the standard that is exploited in the bitstream. Examples are the applied values or range of the picture size and bitrate parameters. Decoder characteristics define the properties and capabilities of the applied decoding process. An example of a property is the applied arithmetic accuracy. The capabilities of a decoder specify which coded bitstreams the decoder can decode and reconstruct, by defining the subset of the standard that may be exploited in decodable bitstreams. A bitstream can be decoded by a decoder if the characteristics of the coded bitstream are within the subset of the standard specified by the decoder capabilities.

Procedures are described for testing conformance of bitstreams and decoders to the requirements defined in parts 1, 2 and 3 of ISO/IEC 14496. Given the set of characteristics claimed, the requirements that must be met are fully determined by parts 1, 2 and 3 of ISO/IEC 14496. This part of ISO/IEC 14496 summarises the requirements, cross references them to characteristics, and defines how conformance with them can be tested. Guidelines are given on constructing tests to verify bitstream and decoder conformance. This document gives guidelines on how to construct bitstream test suites to check or verify decoder conformance. In addition, some test bitstreams implemented according to those guidelines are provided as an electronic annex to this document. The procedures and signaling messages for session and channel establishment are defined in part 6 of ISO/IEC 14496.

Conformance with the signaling messages and procedures in this part of ISO/IEC 14496 are defined in accordance to the specifications in part 6 of ISO/IEC 14496. This specification allows the manufacturer to identify the conformance of the signaling message in a static review and provides abstract test cases to test the conformance to the procedures in a dynamic review of an implementation as defined in ISO/IEC 9646 Conformance Testing standard.