The Moving Picture Experts Group

Workshop on coding technologies for immersive audio/visual experiences

Date: 
Wednesday, 10 July 2019 - 1:00pm to 6:30pm
Venue: 
Clarion Post Hotel
Drottningtorget 10
411 03 Gothenburg
Sweden

MPEG aims at standardizing coding solutions for the digital representation of immersive audio and visual content. The objective is to support immersive applications - virtual and augmented reality - with the highest level of perceived audio/visual realism and with visual comfort with parallax cues that are required in natural vision.

Since any digital capture will subsample the light field in which we are immersed, various capture technologies (omnidirectional and plenoptic cameras, camera arrays, etc.) as well as display technologies (head mounted devices, light field displays, integral photography displays, etc.) lead to a wide range of coding technologies to be explored. Some will be selected for further standardization.

Representation of immersive audio for 6DoF will be based on audio object, channel bed and Higher Order Ambisonics (HOA) signals, all at positions in the VR space relative to the user. MPEG-I Immersive Audio technology will be the coded audio and metadata for transmission or storage; and decoding and rendering for presentation to the user via headphones or possibly loudspeakers. Rendering supports full 6DoF user movement and also audio elements responsive to user interaction. It is expected that the standard will be ready in 2022.

MPEG has already standardized 360° video - also called omnidirectional video - in OMAF version 1, and is in the process of standardizing extensions in 3DoF+ OMAF version 2 by mid-2020, bringing natural vision parallax cues, albeit within a limited range of viewer motion. Full parallax of dynamic object is supported in the first version of Point Cloud Coding ready by early 2020, while coding of 6DoF virtual reality over large navigation volumes is further studied for standardization by 2023.

This workshop will cover the MPEG-I immersive Audio and Visual activities - past, present and future – calling participants to present demos and future requirements to the MPEG community.

This workshop will be publicly accessible with registration.

Title: Coding technologies to be standardized for immersive audio/visual experiences

Date: 10 July, 2019

Address: MPEG meeting venue

Clarion Post Hotel

Drottningtorget 10

411 03 Gothenburg, Sweden

1300-1315

Introduction (Lu Yu, Zhejiang University)

1315-1350

Overview of technologies for immersive visual experiences: capture, processing, compression, standardization and display (Marek Domanski, Poznan University of Technology)

1350-1425

Usecases and challenges about user immersive experiences

(Valerie Allie, Technicolor)

1425-1450

Demos:

(NHK: integral photography display (waiting for formal approval),

Technicolor: Realtime interactive demo with 3DoF+ content,

Tsinghua University: Plenoptic 2.0 video camera,

Poznan University of Technology: Low cost virtual navigation in 6DoF)

1450-1500

Coffee break

1500-1530

6DoF Immersive Audio (Schuyler Quackenbush, Audio Research Labs)

1530-1605

360°  and 3DoF+ video, (Bart Kroon, Philips)

1605-1640

Point cloud compression, (Marius Preda, Telecom SudParis, CNRS Samovar)

1640-1720

How can we achieve 6DoF video compression? (Joel Jung, Orange)

1720-1800

How can we achieve lenslet video compression? (Xin Jin, Tsinghua University)

1800-1830

Discussion