POINT CLOUD AND MESH COMPRESSION USING IMAGE/VIDEO CODECS

• US 20180268570A1
• Filed: 03/13/2018
• Published: 09/20/2018
• Est. Priority Date: 03/16/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Budagavi, Madhukar, Faramarzi, Esmaeil, Ho, Tuan
• Granted Patent: US 11,514,613 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: H04N 19/20 using video object coding

Abstract

An decoding device, an encoding device and a method for point cloud encoding is disclosed. The method includes generating, from a three-dimensional point cloud, multiple two-dimensional frames, the two-dimensional frames including at least a first frame representing a geometry of points in the three-dimensional point cloud and a second frame representing texture of points in the three-dimensional point cloud. The method also includes generating an occupancy map indicating locations of pixels in the two-dimensional frames that represent points in the three-dimensional point cloud. The method further includes encoding the two-dimensional frames and the occupancy map to generate a compressed bitstream. The method also includes transmitting the compressed bitstream.

Claims

1. A decoding device for point cloud decoding, the decoding device comprising:

a communication interface configured to receive a compressed bitstream; and

a processor operably coupled to the communication interface, wherein the processor is configured to;

decode the compressed bitstream into two-dimensional frames and an occupancy map, wherein the two-dimensional frames include at least a first frame representing a geometry of points in a three-dimensional point cloud and a second frame representing a texture of points in the three-dimensional point cloud;

identify the occupancy map indicating locations of pixels in the two-dimensional frames that represent each point in the three-dimensional point cloud; and

generate from the two-dimensional frames the three-dimensional point cloud using the occupancy map.

2. The decoding device of claim 1, wherein to generate from the two-dimensional frames the three-dimensional point cloud, the processor is further configured to:

map a location of each point of the first frame to reconstruct the geometry of points in a three-dimensional point cloud; and

map a texture attribute of each point in the second frame to reconstruct the texture of points in the three-dimensional point cloud,

wherein the texture attribute of each point is mapped based on the mapped location of each point that indicates the geometry of points in the three-dimensional point cloud.

3. The decoding device of claim 1, wherein to decode the compressed bitstream, the processor is further configured to:

decompress a first series of frames, each representing the geometry of points, to generate the first frame that includes the geometry of points; and

decompress a second series of frames, each representing the texture of points of the three-dimensional point cloud, to generate the second frame that includes the texture of points.

4. The decoding device of claim 1, wherein, the processor is further configured to identify auxiliary information to relate the geometry of points and the texture of points in the two-dimensional frames to the three-dimensional point cloud, the auxiliary information indicates a geometric location and texture attribute in the three-dimensional point cloud to each row in the first and second frame.

5. The decoding device of claim 1, wherein to generate from the two-dimensional frames the three-dimensional point cloud, the processor is further configured to:

analyze each row of the first frame to identify a first axis, wherein each point that is positioned in each row corresponds a point of the three-dimensional point cloud at a position along the first axis; and

generate three-dimensional point cloud by positioning each point at a second axis and a third axis that corresponds to each row of the first frame.

6. The decoding device of claim 5, wherein the processor is configured to:

identify that a longest dimension of the three-dimensional point cloud is the first axis; and

generate three-dimensional point cloud by positioning each point at the second axis and the third axis that corresponds to each row of the first frame along the longest dimension of the three-dimensional point cloud.

7. The decoding device of claim 1, wherein:

the second frame representing the texture of points indicates a color attributer and of the three-dimensional point cloud, and

the processor is further configured to store the color attribute as three components in a single frame representing a color of points in the three-dimensional point cloud.

8. An encoding device for point cloud encoding, the encoding device comprising:

a processor configured to;

generate, from a three-dimensional point cloud, multiple two-dimensional frames, the two-dimensional frames including at least a first frame representing a geometry of points in the three-dimensional point cloud and a second frame representing texture of points in the three-dimensional point cloud;

generate an occupancy map indicating locations of pixels in the two-dimensional frames that represent points in the three-dimensional point cloud; and

encode the two-dimensional frames and the occupancy map to generate a compressed bitstream; and

a communication interface operably coupled to the processor, wherein the communication interface is configured to transmit the compressed bitstream.

9. The encoding device of claim 8, wherein to generate the two-dimensional frames, the processor is further configured to:

map a location of each point in the three-dimensional point cloud onto the first frame to indicate the geometry of points in the three-dimensional point cloud; and

map a texture attribute of each point in the three-dimensional point cloud to indicate the texture of points onto the second frame,

wherein the texture attribute of each point is mapped based on the mapped location of each point that indicates the geometry of points in the three-dimensional point cloud.

10. The encoding device of claim 8, wherein to encode the two-dimensional frames, the processor is further configured to:

compress a first series of frames, each representing the geometry of points, to generate a first compressed frame that includes the geometry of points; and

compress a second series of frames, each representing the texture of points of the three-dimensional point cloud, to generate a second compressed frame that includes the texture of points.

11. The encoding device of claim 8, wherein the processor is further configured to generate auxiliary information to relate the geometry of points and the texture of points in the three-dimensional point cloud to the multiple two-dimensional frames, the auxiliary information indicates a geometric location and texture attribute in the three-dimensional point cloud to each row in the first and second frame.

12. The encoding device of claim 8, wherein to generate multiple two-dimensional frames, the processor is further configured to:

analyze the three-dimensional point cloud at a plurality of positions along a first axis; and

store each point in the three-dimensional point cloud along a second and third axis that correspond to each position along the first axis,

wherein each point is stored in row of the first frame that corresponds the position along the first axis.

13. The encoding device of claim 12, wherein the processor is further configured to:

determine a longest dimension of the three-dimensional point cloud;

analyze the three-dimensional point cloud at the plurality of positions along the longest dimension; and

store each point in the three-dimensional point cloud along a second and third axis that correspond to each position along the longest dimension of the three-dimensional point cloud.

14. The encoding device of claim 8, wherein:

the second frame representing the texture of points indicates a color attributer and of the three-dimensional point cloud, and

the processor is further configured to store the color attribute as three components in a single frame representing a color of points in the three-dimensional point cloud.

15. A method for point cloud encoding, the method comprising:

generating, from a three-dimensional point cloud, multiple two-dimensional frames, the two-dimensional frames including at least a first frame representing a geometry of points in the three-dimensional point cloud and a second frame representing a texture of points in the three-dimensional point cloud;

generating an occupancy map indicating locations of pixels in the two-dimensional frames that represent points in the three-dimensional point cloud; and

encoding the two-dimensional frames and the occupancy map to generate a compressed bitstream; and

transmitting the compressed bitstream.

16. The method of claim 15, wherein generating the two-dimensional frames, the method comprises:

mapping a location of each point in the three-dimensional point cloud onto the first frame to indicate the geometry of points in the three-dimensional point cloud; and

mapping a texture attribute of each point in the three-dimensional point cloud to indicate the texture of points onto the second frame,

wherein the texture attribute of each point is mapped based on the mapped location of each point that indicates the geometry of points in three-dimensional point cloud.

17. The method of claim 15, wherein encoding the two-dimensional frames, the method comprises:

compressing a first series of frames, each representing the geometry of points, to generate a first compressed frame that includes the geometry of points; and

compressing a second series of frames, each representing the texture of points of the three-dimensional point cloud, to generate a second compressed frame that includes the texture of points.

18. The method of claim 15, further comprises generating auxiliary information to relate the geometry of points and the texture of points in the three-dimensional point cloud to the multiple two-dimensional frames, the auxiliary information indicates a geometric location and texture attribute in the three-dimensional point cloud to each row in the first and second frame.

19. The method of claim 15, wherein generating the two-dimensional frames, the method comprises:

analyzing the three-dimensional point cloud at a plurality of positions along a first axis; and

storing each point in the three-dimensional point cloud along a second and third axis that correspond to each position along the first axis,

wherein each point is stored in row of the first frame that corresponds the position along the first axis.

20. The method of claim 19, further comprises:

determining a longest dimension of the three-dimensional point cloud;

analyzing the three-dimensional point cloud at the plurality of positions along the longest dimension; and

storing each point in the three-dimensional point cloud along a second and third axis that correspond to each position along the longest dimension of the three-dimensional point cloud.