Structured light depth imaging under various lighting conditions

• US 10,089,739 B2
• Filed: 06/04/2014
• Issued: 10/02/2018
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Appia, Vikram VijayanBabu
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Torres, Jose M
• Application Number:
  US14/296,172
• Publication Number:
  US 20150003684A1
• Time in Patent Office: 1,581 Days
• Field of Search: None
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method of image processing in a structured light imaging system is provided that includes receiving a captured image of a scene, wherein the captured image is captured by a camera of a projector-camera pair, and wherein the captured image includes a binary pattern projected into the scene by the projector, applying a filter to the rectified captured image to generate a local threshold image, wherein the local threshold image includes a local threshold value for each pixel in the rectified captured image, and extracting a binary image from the rectified captured image wherein a value of each location in the binary image is determined based on a comparison of a value of a pixel in a corresponding location in the rectified captured image to a local threshold value in a corresponding location in the local threshold image.

Claims

1. A method of image processing in a structured light imaging system, the method comprising:

receiving a captured image of a scene, wherein the captured image is captured by a camera of a projector-camera pair in the structured light imaging system, and wherein the captured image comprises a binary pattern projected into the scene by the projector;

rectifying the captured image to generate a rectified captured image;

applying a bilateral filter to the rectified captured image to generate a local threshold image, wherein the local threshold image comprises a local threshold value for each pixel in the rectified captured image;

extracting a binary image from the rectified captured image wherein a value of each location in the binary image is determined based on a comparison of a value of a pixel in a corresponding location in the rectified captured image to a local threshold value in a corresponding location in the local threshold image;

performing a matching algorithm to match the binary image to the binary pattern by measuring a Hamming distance;

determining a disparity map based on the matching algorithm; and

performing triangulation to determine a 3D point cloud based on disparities in the disparity map.

2. The method of claim 1, wherein extracting a binary image comprises setting a location in the binary image to one when a value of a pixel in a corresponding location in the rectified captured image is greater than a local threshold value in a corresponding location in the local threshold image and setting the value of the location in the binary image to zero when the value of the pixel is less than the local threshold value.

3. The method of claim 1, wherein the bilateral filter generates each local threshold value as a local mean of a corresponding pixel in the rectified image.

4. The method of claim 1, wherein an operation by the bilateral filter to generate the local threshold value is dependent on a distribution of binary numbers in the projected binary pattern.

5. The method of claim 4, wherein the bilateral filter generates each local threshold value as a local mean of a corresponding pixel in the rectified image when the distribution of the binary numbers in the projected binary pattern is equal.

6. The method of claim 1, wherein a kernel size of the bilateral filter is dependent on an information density in the projected binary pattern.

7. A structured light imaging system comprising:

an imaging sensor component configured to capture images of a scene;

a projector component configured to project a binary pattern into the scene;

means for rectifying a captured image to generate a rectified captured image;

means for applying a bilateral filter to the rectified captured image to generate a local threshold image, wherein the local threshold image comprises a local threshold value for each pixel in the rectified captured image;

means for extracting a binary image from the rectified captured image wherein a value of each location in the binary image is determined based on a comparison of a value of pixel in a corresponding location in the rectified captured image to a local threshold value in a corresponding location in the local threshold image;

means for performing a matching algorithm to match the binary image to the binary pattern by measuring a Hamming distance;

means for determining a disparity map based on the matching algorithm; and

means for performing triangulation to determine a 3D point cloud based on disparities in the disparity map.

8. The structured light imaging system of claim 7, wherein extracting a binary image comprises setting a location in the binary image to one when a value of a pixel in a corresponding location in the rectified captured image is greater than a local threshold value in a corresponding location in the local threshold image and setting the value of the location in the binary image to zero when the value of the pixel is less than the local threshold value.

9. The structured light imaging system of claim 7, wherein the bilateral filter generates each local threshold value as a local mean of a corresponding pixel in the rectified image.

10. The structured light imaging system of claim 7, wherein an operation by the bilateral filter to generate the local threshold value is dependent on a distribution of binary numbers in the projected binary pattern.

11. The structured light imaging system of claim 10, wherein the bilateral filter generates each local threshold value as a local mean of a corresponding pixel in the rectified image when the distribution of the binary numbers in the projected binary pattern is equal.

12. The structured light imaging system of claim 7, wherein a kernel size of the bilateral filter is dependent on an information density in the projected binary pattern.

13. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors cause the one or more processors to:

receive a captured image of a scene, wherein the captured image is captured by a camera of a projector-camera pair in a structured light imaging system, and wherein the captured image comprises a binary pattern projected into the scene by the projector;

rectify the captured image to generate a rectified captured image;

apply a bilateral filter to the rectified captured image to generate a local threshold image, wherein the local threshold image comprises a local threshold value for each pixel in the rectified captured image; and

extract a binary image from the rectified captured image wherein a value of each location in the binary image is determined based on a comparison of a value of pixel in a corresponding location in the rectified captured image to a local threshold value in a corresponding location in the local threshold image;

perform a matching algorithm to match the binary image to the binary pattern by measuring a Hamming distance;

determine a disparity map based on the matching algorithm; and

perform triangulation to determine a 3D point cloud based on disparities in the disparity map.

14. The computer-readable medium of claim 13, wherein the instructions that cause the one or more processors to extract a binary image include instructions that cause the one or more processors to set a location in the binary image to one when a value of a pixel in a corresponding location in the rectified captured image is greater than a local threshold value in a corresponding location in the local threshold image and setting the value of the location in the binary image to zero when the value of the pixel is less than the local threshold value.

15. The computer-readable medium of claim 13, wherein the bilateral filter generates each local threshold value as a local mean of a corresponding pixel in the rectified image.

16. The computer-readable medium of claim 13, wherein an operation by the bilateral filter to generate the local threshold value is dependent on a distribution of binary numbers in the projected binary pattern.

17. The computer-readable medium of claim 16, wherein the bilateral filter generates each local threshold value as a local mean of a corresponding pixel in the rectified image when the distribution of the binary numbers in the projected binary pattern is equal.

18. The computer-readable medium of claim 13, wherein a kernel size of the bilateral filter is dependent on an information density in the projected binary pattern.