STRUCTURED LIGHT THREE-DIMENSIONAL (3D) DEPTH MAP BASED ON CONTENT FILTERING

• US 20150371393A1
• Filed: 06/18/2015
• Published: 12/24/2015
• Est. Priority Date: 06/19/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Ramachandra, Vikas, Atanassov, Kalin Mitkov, Nash, James Wilson
• Granted Patent: US 9,582,888 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/514 from specularities
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A structured light three-dimensional (3D) depth map based on content filtering is disclosed. In a particular embodiment, a method includes receiving, at a receiver device, image data that corresponds to a structured light image. The method further includes processing the image data to decode depth information based on a pattern of projected coded light. The depth information corresponds to a depth map. The method also includes performing one or more filtering operations on the image data. An output of the one or more filtering operations includes filtered image data. The method further includes performing a comparison of the depth information to the filtered image data and modifying the depth information based on the comparison to generate a modified depth map.

Claims

1. A method comprising:

receiving image data at a receiver device, wherein the image data corresponds to a structured light image;

processing the image data to decode depth information based on a pattern of projected coded light, wherein the depth information corresponds to a depth map;

performing one or more filtering operations on the image data, wherein an output of the one or more filtering operations includes filtered image data;

performing a comparison between the depth information and the filtered image data; and

modifying the depth information based on the comparison to generate a modified depth map.

2. The method of claim 1, wherein the one or more filtering operations modify the image data to identify regions within the structured light image that have similar reflectivity.

3. The method of claim 1, wherein the filtered image data is generated without determining an edge between two adjacent regions having differing reflectivity.

4. The method of claim 1, wherein the filtered image data corresponds to a filtered structured light image.

5. The method of claim 4, wherein the filtered structured light image is a segmentation map image.

6. The method of claim 1, further comprising identifying, based on the comparison, an incorrect depth value in the depth information.

7. The method of claim 1, wherein processing the image data to decode the depth information comprises:

identifying a codeword based on the image data; and

determining a displacement of the codeword from a location of the codeword in a projected code mask to a location of the codeword in the structured light image, wherein the structured light image corresponds to a reflection of the projected code mask;

and generating depth data based on the displacement.

8. The method of claim 1, wherein performing the one or more filtering operations comprises applying a blurring filter, a total variation filter, a low pass filter, or a combination thereof, to filter the image data to generate the filtered image data.

9. The method of claim 1, wherein the comparison corresponds to a pixel by pixel comparison between the depth information and the filtered image data.

10. The method of claim 9, further comprising identifying, based on the comparison, an incorrect depth value included in the depth information, wherein modifying includes correcting the incorrect depth value included in the depth information.

11. The method of claim 9, further comprising identifying, based on the comparison, a missing depth value associated with the depth information, wherein modifying includes populating the missing depth value to be included in the depth information.

12. The method of claim 1, further comprising:

identifying a particular pixel location of the depth map, wherein the particular pixel location corresponds to incorrect depth data included in the depth information; and

determining, based on the filtered image data, a particular filtered data value of the filtered image data that corresponds to the particular pixel location, wherein modifying the depth information includes modifying the incorrect depth data based on the particular filtered data value.

13. The method of claim 12, further comprising:

selecting an area associated with the depth map based on the particular pixel location, wherein the area is associated with a plurality of pixel locations that includes the particular pixel location; and

identifying a set of one or more pixels locations of the plurality of pixel locations, where each pixel associated with the set of one or more pixel locations corresponds to a filtered data value of the filtered image data that is similar to the particular filtered data value.

14. The method of claim 13, further comprising:

determining a particular depth value based on depth data that corresponds to the set of one or more pixel locations; and

updating the incorrect depth data corresponding to the particular pixel location, wherein the incorrect depth data is updated based on the particular depth value.

15. A device comprising:

a processor configured to process image data that corresponds to a structured light image to decode depth information based on a pattern of projected coded light and to perform a filtering operation on the image data to generate filtered image data, the processor further configured to modify the depth information based on a comparison between the depth information and the filtered image data; and

a memory configured to store the modified depth information.

16. The device of claim 15, wherein the device includes camera configured to capture the structured light image and to generate the image data.

17. The device of claim 15, wherein the depth information corresponds to a depth map associated with the structured light image, and wherein the pattern of projected coded light is associated with a code mask.

18. The device of claim 15, wherein the image data corresponds to a plurality of pixels, and wherein, for each pixel of the plurality of pixels, the image data includes a corresponding pixel intensity value.

19. The device of claim 18, wherein, for each pixel of the plurality of pixels, the depth information includes a corresponding depth value and the filtered image data includes a corresponding intensity value.

20. The device of claim 15, wherein the processor is further configured to perform the comparison between the depth information and the filtered image data, wherein the comparison includes a pixel by pixel comparison between the depth information and the filtered image data.

21. The device of claim 20, wherein the processor is further configured to identify, based on the comparison, an incorrect depth value, a missing depth value, or a combination thereof.

22. An apparatus comprising:

means for decoding depth information based on image data that corresponds to a structured light image and based on a pattern of projected coded light;

means for performing one or more filtering operations on the image data to generate filtered image data; and

means for modifying the depth information based on a comparison between the depth information and the filtered image data.

23. The apparatus of claim 22, further comprising means for capturing the structured light image.

24. The apparatus of claim 22, wherein the depth information is modified to correct incorrect depth data of the depth information, and wherein the incorrect depth data includes an incorrect depth value or a missing depth value.

25. A non-transitory computer-readable medium storing instructions that are executable by a processor to cause the processor to perform operations comprising:

processing image data corresponding to a structured light image to decode depth information based on a pattern of projected coded light, wherein the depth information corresponds to a depth map;

performing one or more filtering operations on the image data to generated filtered image data; and

modifying the depth information based on a comparison between the depth information and the filtered image data.

26. The non-transitory computer-readable medium of claim 25, wherein the image data is received from an image capture sensor that is configured to capture the structured light image.

27. The non-transitory computer-readable medium of claim 25, wherein performing the one or more filtering operations comprises performing a blurring filtering operation, a total variation filtering operation, or a combination thereof.

28. The non-transitory computer-readable medium of claim 25, wherein the instructions are further executable by the processor to cause the processor to perform operations comprising identifying, based on the comparison, an incorrect depth value included in the depth information, wherein modifying includes correcting the incorrect depth value included in the depth information.

29. The non-transitory computer-readable medium of claim 25, wherein the instructions are further executable by the processor to cause the processor to perform operations comprising identifying, based on the comparison, a missing depth value associated with the depth information, wherein modifying includes populating the missing depth value to be included in the depth information.

30. The non-transitory computer-readable medium of claim 25, wherein performing the one or more filtering operations comprises performing a low pass filtering operation, a band pass filtering operation, or a combination thereof.