by Philip Merrill - June 2016
Movie special effects are commonly used to make the imaginary look real, but by processing images from different sources new sides of what is visible can be presented. The feeling of "being there" in 3D scenes has been suggested by camerawork since the beginning of cinema, but today's imaging applications permit the construction of infinite vantage points from information captured by a limited number of cameras. The parable of the blind men and the elephant introduced the idea of differing perspectives to the ancient world, but now it suggests how the group would have been assisted by position and directional data for each investigator. In our real world, free navigation supplies our visual desire to move around scenes composed from camera-captured source material, something vaguely comparable to the ability gamers have to move across and around geometrically generated play levels.
At the 115th MPEG meeting in Geneva, researchers from Poland's Poznan University of Technology submitted an archive of several dozen source video sequences and their parameters, as well as several free navigation sequences based on them. One example, of two gentlemen sitting at a table making detailed structures out of toy blocks, illustrates a potential entertainment scenario; another, a sequence with two fencers engaged in rapid lunging and parrying, gives a sense of free navigation's potential for sports reporting. Each was captured by ten cameras arranged in pairs along an arc with common radius and center. The cameras in each pair were separated from each other by about a meter or less, and the five pairs were separated from each other by 15 degrees for a total arc of 60 degrees. As the vantage point in the derived sequences navigates freely in a synthesized tracking shot, the visible information alternates between the camera-captured information and synthesized interstitial data that must be generated.
Free navigation is at a preliminary and exciting stage where the technologies that will eventually make possible a standard are still evolving. The input archive from Geneva represents a store of source material academically licensable for others to try their interstitial best to synthesize scene material that blends the pixels in a way that seems true. The accuracy of the toy blocks' shadings or the rapidly moving foils provide a wide range of artistic possibilities, as the granular usually does. But these are details of image synthesis and the broad possible reasons for a standard are more exciting as these could potentially bring the interstitial math broadly to life and to the public.
Logical considerations in the past have believed interoperability and standardizing some minimum, required to ensure that interoperability, allows a system of developers to compete separate from the standard development process as private entities. On some fundamental level it is irrelevant where all these MPEG experts comes from because once gathered together they enable this process that functions differently from how corporations or pure academia function. There are rules that are strictly followed in this and there are also areas where flexibility provides the opportunities to pursue the lively MPEG process.
On a technical level free navigation still requires broad and considerable research. It will take some time before a decision is reached. Anything is possible within the scope of what can be engineered, and the MPEG process explores precisely that, investigating whether there are good opportunities to standardize a minimum required (or informative) for interoperability.
News extracted from: M. Domański, A. Dziembowski, A. Grzelka, D. Mieloch, O. Stankiewicz, K. Wegner, “Multiview test video sequences for free navigation exploration obtained using pairs of cameras”, ISO/IEC JTC 1/SC 29/WG 11 Doc. M38247, Geneva, Switzerland, May 2016.
Video sequences are owned by: Poznan University of Technology, Chair of Multimedia Telecommunications and Microelectronics, Polanka 3, PL 60-965 Poznan, Poland
Go to the MPEG news page