ADAPTIVE DISTANCE BASED POINT CLOUD COMPRESSION

• US 20190311499A1
• Filed: 04/10/2019
• Published: 10/10/2019
• Est. Priority Date: 04/10/2018
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Mammou, Khaled, Kim, Jungsun, Valentin, Valery G., Robinet, Fabrice A., Su, Yeping, Tourapis, Alexandros
• Granted Patent: US 11,010,928 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame

Abstract

A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute for the point cloud. To compress the attribute information, attribute values are predicted using one of a plurality of prediction strategies, wherein a selected prediction strategy is selected based at least in part on attribute variability of points in a neighborhood of points. A decoder follows a similar prediction process. Also, attribute correction values may be determined to correct predicted attribute values and may be used by a decoder to decompress a point cloud, wherein the decoder applies the same prediction strategy applied at the encoder.

Claims

1. A system comprising:

an encoder configured to;

for a given point of a point cloud;

identify a set of neighboring points that neighbor the given point;

determine whether a variability of attribute values of the neighboring points exceeds a variability threshold;

determine respective predicted attribute values for one or more points of a sub-set of points of the point cloud that includes the given point according to;

a first prediction procedure if the variability of the attribute values of the set of neighboring points is less than the threshold; and

another prediction procedure if the variability of the attribute values of the set of neighboring points exceeds the threshold; and

determine respective attribute correction values for the one or more points of the sub-set points of the point cloud, based on comparing the respective predicted attribute values for respective ones of the one or more points of the sub-set to attribute information for corresponding respective points included in the point cloud prior to the point cloud being encoded; and

encode the determined attribute correction values.

2. The system of claim 1, wherein the encoder is configured to:

assign an attribute value to at least one point of an encoded version of the point cloud based on attribute information for the at least one point in the point cloud prior to the point cloud being encoded; and

encode the attribute value for the at least one point,

wherein the encoded attribute value for the at least one point and the encoded attribute correction values for other ones of the points of the encoded version of the point cloud are used, by a decoder, to determine attribute values for points of the point cloud when being reconstructed.

3. The system of claim 1, wherein the encoder is configured to predict the respective attribute values for respective ones of the one or more points of the sub-set based on predicted or assigned attribute values of neighboring points that neighbor the respective ones of the one or more points of the sub-set for which attribute values are being predicted,

wherein to predict the respective attribute values according to the first prediction strategy the encoder predicts the respective attribute values based on attribute values of a larger group of neighboring points that includes more neighboring points than is used to predict the respective attribute values according to the second prediction strategy.

4. The system of claim 1, wherein the encoder is configured to, for respective ones of the one or more points of the sub-set:

predict the respective attribute values, in accordance with the first prediction procedure, based on a group of neighboring points within a threshold distance of a given point for which a respective attribute value is being predicted; and

predict the respective attribute values, in accordance with the second prediction procedure, based on an assigned or predicted attribute value of a nearest point to a given point for which a respective attribute value is being predicted.

5. The system of claim 1, wherein the encoder is configured to, for respective ones of the one or more points of the sub-set:

predict the respective attribute values, in accordance with the first prediction procedure, based on a distance based interpolation for attribute values a group of neighboring points within a threshold distance of a given point for which a respective attribute value is being predicted; and

predict the respective attribute values, in accordance with the second prediction procedure, wherein the second prediction procedures utilizes a prediction procedure other than distance based interpolation.

6. The system of claim 5, wherein the second prediction procedure predicts the respective attribute values based on trellis quantization for a group of points whose attribute values are encoded jointly.

7. The system of claim 1, wherein the encoder is further configured to:

select a variability technique to be used to determine whether the variability of the attribute values of the neighboring points exceeds the variability threshold; and

signal the selected variability technique in a bit stream comprising the encoded determined attribute correction values,

wherein the variability technique is selected from one or more of;

a sum of square errors variability technique;

a distance weighted sum of square errors variability technique;

a sum of absolute differences variability technique;

or

a distance weighted sum of absolute differences variability technique.

8. The system of claim 1, wherein the encoder is further configured to:

determine the variability threshold to be applied to the neighboring points to determine whether the variability of the attribute values of the neighboring points exceeds the variability threshold; and

signal the determined variability threshold for the neighboring points in a bit stream comprising the encoded determined attribute correction values,

wherein the encoder is configured to determine different variability thresholds for different sets of neighboring points of the point cloud.

9. A method comprising:

identifying a set of neighboring points greater than a first distance and less than a second distance from a point of a point cloud;

determining whether a variability of attribute values of the neighboring points exceeds a variability threshold;

determining respective predicted attribute values for one or more points of a sub-set of respective points of the point cloud according to;

a first prediction procedure if the variability of the attribute values of the neighboring points is less than the threshold;

or

another prediction procedure if the variability of the attribute values of the neighboring points exceeds the threshold;

determining respective attribute correction values for the one or more points of the sub-set, based on comparing the respective predicted attribute values for respective ones of the one or more points of the sub-set to the attribute information for corresponding respective points included in the point cloud prior to the point cloud being encoded; and

encoding the determined attribute correction values.

10. The method of claim 9, wherein the variability of the attribute values of the neighboring points is determined according to:

a sum of square errors variability technique;

a distance weighted sum of square errors variability technique;

a sum of absolute differences variability technique;

or

a distance weighted sum of absolute differences variability technique.

11. The method of claim 9, further comprising;

assigning an attribute value to at least one point of the point cloud being encoded based at least in part on attribute information for a corresponding point in the point cloud prior to the point cloud being encoded; and

for each of respective other ones of the points of the point cloud being encoded, performing;

said identifying a set of neighboring points,

said determining whether a variability of the attribute values of the neighboring points exceeds the variability threshold,

said determining a respective predicted attribute value according to the first or other prediction procedure, and

said determining a respective attribute correction value.

12. The method of claim 9, wherein said determining the respective predicted attribute values for the one or more points of the sub-set of respective points according to the first prediction procedure or the other prediction procedure comprises:

predicting the respective attribute values, in accordance with the first prediction procedure, based on attribute values of a group of neighboring points within a threshold distance of a given point for which a respective attribute value is being predicted;

or

predicting the respective attribute values, in accordance with the second prediction procedure, based on an assigned or predicted attribute value of a nearest point to a given point for which a respective attribute value is being predicted.

13. The method of claim 9, wherein said determining the respective predicted attribute values for the one or more points of the sub-set of respective points according to the first prediction procedure or the other prediction procedure comprises:

predicting the respective attribute values, in accordance with the first prediction procedure, based on a distance based interpolation for attribute values of a group of neighboring points within a threshold distance of a given point for which a respective attribute value is being predicted;

or

predicting the respective attribute values, in accordance with the second prediction procedure, wherein the second prediction procedures utilizes a prediction procedure other than distance based interpolation.

14. A non-transitory computer-readable medium storing program instructions, that when executed by one or more processors, causes the one or more processors to:

receive compressed attribute information for a point cloud, the compressed attribute information comprising an assigned attribute value for at least one point of the point cloud and data indicating attribute correction values for attributes of other points of the point cloud;

predict attribute values for respective ones of the points of the point cloud, wherein to predict the attribute values for the respective ones of the points of the point cloud, the program instructions, when executed on the one or more processors, cause the one or more processors to;

identify a set of neighboring points greater than a first distance and less than a second distance from a respective point;

determine whether a variability of attribute values of the set of neighboring points exceeds a variability threshold;

determine respective predicted attribute values for the respective points according to;

a first prediction procedure if the variability of the attribute values of the neighboring points is less than the threshold;

or

another prediction procedure if the variability of the attribute values of the neighboring points exceeds the threshold; and

apply one or more attribute correction values to the predicted attribute values to determine attribute information for a reconstructed version of the point cloud.

15. The non-transitory computer-readable medium of claim 14, wherein the received compressed attribute information comprises data indicating the variability threshold, wherein the program instructions cause the one or more processors to determine whether the variability of the attribute values of the set of neighboring points exceeds the variability threshold included in the received compressed attribute information.

16. The non-transitory computer-readable medium of claim 15, wherein the received compressed attribute information further indicates that the variability threshold is to be applied to a sub-set of the points of the point cloud,

wherein the program instructions cause the one or more processors to:

determine whether the variability of the attribute values of respective sets of the neighboring points has been exceeded;

using the variability threshold included in the received compressed attribute information for the indicated sub-set of the points of the point cloud; and

using a default variability threshold for other points of the point cloud not included in the sub-set of the points.

17. The non-transitory computer-readable medium of claim 14, wherein the received compressed attribute information indicates a particular prediction strategy to be used for a sub-set of the points of the point cloud,

wherein the program instructions cause the one or more processors to:

apply the particular prediction strategy for the sub-set of points of the point cloud; and

apply the first prediction strategy or the other prediction strategy for other ones of the points of the point cloud not included in the sub-set, based on whether the variability of the attribute values of respective sets of neighboring points of the other points not included in the sub-set exceed the variability threshold.

18. The non-transitory computer-readable medium of claim 14, wherein to determine the respective predicted attribute values for the respective points, the program instructions cause the one or more processors to:

predict the respective attribute values, in accordance with the first prediction procedure, based on attribute values of a group of neighboring points within a threshold distance of a given point for which a respective attribute value is being predicted;

or

predict the respective attribute values, in accordance with the second prediction procedure, based on an assigned or predicted attribute value of a nearest point to a given point for which a respective attribute value is being predicted.

19. The non-transitory computer-readable medium of claim 14, wherein to determine the respective predicted attribute values for the respective points, the program instructions cause the one or more processors to:

predict the respective attribute values, in accordance with the first prediction procedure, based on a distance based interpolation for attribute values of a group of neighboring points within a threshold distance of a given point for which a respective attribute value is being predicted;

or

predict the respective attribute values, in accordance with the second prediction procedure, wherein the second prediction procedures utilizes a prediction procedure other than distance based interpolation.

20. The non-transitory computer-readable medium of claim 14, wherein to determine the respective predicted attribute values for the respective points, the program instructions cause the one or more processors to:

predict the respective attribute values, in accordance with the first prediction procedure, based on attribute values of a group of points comprising a number of neighboring points within a threshold distance of a given point for which a respective attribute value is being determined, wherein a value for the number of points to be included in the group is indicated in the received compressed attribute information;

or

predict the respective attribute values, in accordance with the second prediction procedure, based on fewer neighboring points than the number of points included in the group of points used in the first prediction procedure.