3D POINT CLOUD COMPRESSION SYSTEMS FOR DELIVERY AND ACCESS OF A SUBSET OF A COMPRESSED 3D POINT CLOUD

• US 20190318488A1
• Filed: 04/09/2019
• Published: 10/17/2019
• Est. Priority Date: 04/12/2018
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lim, Youngkwon, Bouazizi, Imed, Yip, Eric Ho Ching, Joshi, Rajan Laxman
• Granted Patent: US 10,984,541 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 21/85406 involving a specific file format, e.g. MP4 format
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

An encoding device, a decoding device and a method for point cloud encoding is provided. The method includes generating, for a 3D point cloud, frames corresponding to attributes of the 3D point cloud and an occupancy map. The method also includes encoding, the frames to generate one or more bitstreams. The one or more bitstreams include individual tracks and media content. The individual tracks include media tracks and a timed metadata track. The timed metadata track references the media tracks and the media tracks correspond respectively to the attributes. The method also includes generating a container including the one or more bitstreams transmitting the container.

Claims

1. A decoding device for point cloud decoding, comprising:

a communication interface configured to receive a container including one or more bitstreams; and

a processor operably coupled to the communication interface, wherein the processor is configured to;

identify, from the container, individual tracks including a timed metadata track and media tracks, wherein the timed metadata track references the media tracks and the media tracks correspond respectively to attributes of a three-dimensional (3D) point cloud,

decode the one or more bitstreams to generate frames based on the individual tracks and media content, wherein the frames correspond to the attributes and an occupancy map, and

generate the 3D point cloud using the frames and the timed metadata track.

2. The decoding device of claim 1, wherein:

the processor is configured to identify a point cloud compression header in a sample entry of the timed metadata track, and

the timed metadata track indicates a location of at least one of the occupancy map or an auxiliary patch within the media content.

3. The decoding device of claim 1, wherein the processor is further configured to:

identify correspondence information associated with the individual tracks,

determine that the 3D point cloud is segmented into multiple subsets,

wherein the correspondence information associates points of the 3D point cloud to pixels included in the of frames.

4. The decoding device of claim 3, wherein:

the pixels included in the frames include a first subset and a second subset, the first subset representing a higher level of detail than the second subset,

the communication interface is configured to receive the correspondence information prior to receiving the container

the processor is further configured to;

decode the first subset and the second subset, based on the correspondence information, and

generate the 3D point cloud using the frames and the timed metadata track corresponding to the first subset and the second subset.

5. The decoding device of claim 3, wherein:

the multiple subsets span the 3D point cloud,

a point, of the 3D point cloud, that is within a first subset, of the multiple subsets, is not within another subset of the multiple subsets, and

the multiple subsets correspond respectively to multiple 3D spatial regions.

6. The decoding device of claim 5, wherein the multiple 3D spatial regions are cuboids.

7. The decoding device of claim 5, wherein the pixels included in the frames are motion constrained tiles.

8. An encoding device for point cloud encoding, comprising:

a processor configured to;

generate, for a three-dimensional (3D) point cloud, frames corresponding to attributes of the 3D point cloud and an occupancy map;

encode, the frames to generate one or more bitstreams including individual tracks and media content, the individual tracks include media tracks and a timed metadata track, wherein the timed metadata track references the media tracks and the media tracks correspond respectively to the attributes;

generate a container including the one or more bitstreams; and

a communication interface operably coupled to the processor, the communication interface configured to transmit the container.

9. The encoding device of claim 8, wherein:

the processor is configured to place a point cloud compression header in a sample entry of the timed metadata track, and

the timed metadata track indicates a location of at least one of the occupancy map or an auxiliary patch within the media content.

10. The encoding device of claim 8, wherein the processor is configured to:

segment the 3D point cloud into multiple subsets;

generate correspondence information associating points of the 3D point cloud to pixels included in the frames; and

include the correspondence information in the individual tracks.

11. The encoding device of claim 10, wherein:

the pixels included in the frames include a first subset and a second subset, the first subset representing a higher level of detail than the second subset, and

the communication interface is configured to transmit the correspondence information, associated with the first subset and the second subset, prior to transmitting the container.

12. The encoding device of claim 10, wherein:

the multiple subsets span the 3D point cloud,

a point, of the 3D point cloud, that is within a first subset, of the multiple subsets, is not within another subset of the multiple subsets, and

the multiple subsets correspond respectively to multiple 3D spatial regions.

13. The encoding device of claim 12 wherein the multiple 3D spatial regions are cuboids.

14. The encoding device of claim 12, wherein the pixels included in the frames are motion constrained tiles.

15. A method for point cloud encoding, comprising:

generating, for a three-dimensional (3D) point cloud, frames corresponding to attributes of the 3D point cloud and an occupancy map;

encoding, the frames to generate one or more bitstreams including individual tracks and media content, the individual tracks include media tracks and a timed metadata track, wherein the timed metadata track references the media tracks and the media tracks correspond respectively to the attributes;

generating a container including the one or more bitstreams; and

transmitting the container.

16. The method of claim 15, further comprising:

placing a point cloud compression header in a sample entry of the timed metadata track, and

wherein the timed metadata track indicates a location of at least one of the occupancy map or an auxiliary patch within the media content.

17. The method of claim 15, further comprising:

segmenting the 3D point cloud into multiple subsets;

generating correspondence information associating points of the 3D point cloud to pixels included in the frames; and

including the correspondence information in the individual tracks.

18. The method of claim 17, wherein:

the pixels included in the frames include a first subset and a second subset, the first subset representing a higher level of detail than the second subset, and

the method further comprises transmitting the correspondence information, associated with the first subset and the second subset, prior to transmitting the container.

19. The method of claim 17, wherein:

the multiple subsets span the 3D point cloud,

a point, of the 3D point cloud, that is within a first subset, of the multiple subsets, is not within another subset of the multiple subsets, and

the multiple subsets correspond respectively to multiple 3D spatial regions.

20. The method of claim 19, wherein:

the multiple 3D spatial regions are cuboids, and

the pixels included in the frames are motion constrained tiles.