MPEG Compact Descriptors for Visual Search
MPEG doc#: N12041
Date: March 2011
Author:
Camera equipped mobile devices, such as mobile phones or tablets are becoming ubiquitous platforms for deployment of visual search and augmented reality applications. A visual database is typically stored on remote servers. Hence, for a visual search, information must be either uploaded from the mobile device to the server, or downloaded from the server to the mobile device. With relatively slow wireless links, the response time of the system critically depends on how much information must be transferred.
MPEG is considering standardizing technologies that will enable efficient and interoperable design of visual search applications. In particular we are seeking technologies for visual content matching in images or video. Visual content matching includes matching of views of objects, landmarks, and printed documents that is robust to partial occlusions as well as changes in vantage point, camera parameters, and lighting conditions.
There are a number of component technologies that are useful for visual search, including format of visual descriptors, descriptor extraction process, as well as indexing, and matching algorithms. As a minimum, the format of descriptors as well as parts of their extraction process should be defined to ensure interoperability.
It is envisioned that a standard for compact descriptors will
- ensure interoperability of visual search applications and databases,
- enable high level of performance of implementations conformant to the standard,
- simplify design of visual search applications,
- enable hardware support for descriptor extraction and matching functionality in mobile devices,
- reduce load on wireless networks transmitting visual search-related information.
It is envisioned that such standard will provide a complementary tool to the suite of existing MPEG standards, such as MPEG-7 visual descriptors. To build full visual search application this standard may be used jointly with other standards, such as MPEG Query Format, HTTP, XML, JPEG, JPSec, and JPSearch.
Related documents
[1] N12038, CDVS: Call for Proposals, March 2011, Geneva, CH
[2] N11531, CDVS: Requirements, July 2010, Geneva, CH.
[3] N11531, CDVS: Context and Objectives, July 2010, Geneva, CH.
[4] N11529, CDVS: Applications and Use Scenarios, July 2010, Geneva, CH.