Enhancement of Stereo Depth Maps

• US 20140198977A1
• Filed: 03/21/2013
• Published: 07/17/2014
• Est. Priority Date: 03/21/2012
• Status: Abandoned Application

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Narasimha, Rajesh, Villarreal, Jesse Gregory Jr., Raghuram, Karthik Jayaraman, Pacheco, Roman Joel
• Application Number:
  US13/848,697
• Publication Number:
  US 20140198977A1
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A method for computation of a depth map for corresponding left and right two dimensional (2D) images of a stereo image is provided that includes determining a disparity range based on a disparity of at least one object in a scene of the left and right 2D images, performing color matching of the left and right 2D images, performing contrast and brightness matching of the left and right 2D images, and computing a disparity image for the left and right 2D images after the color matching and the contrast and brightness matching are performed, wherein the disparity range is used for correspondence matching of the left and right 2D images.

Claims

1. A method for computation of a depth map for corresponding left and right two dimensional (2D) images of a stereo image, the method comprising:

determining a disparity range based on a disparity of at least one object in a scene of the left and right 2D images;

performing color matching of the left and right 2D images;

performing contrast and brightness matching of the left and right 2D images; and

computing a disparity image for the left and right 2D images after the color matching and the contrast and brightness matching are performed, wherein the disparity range is used for correspondence matching of the left and right 2D images.

2. The method of claim 1, wherein determining a disparity range comprises:

detecting the at least one object in a single 2D image selected from the left 2D image and the right 2D image; and

computing the disparity based on expected sizes of the at least one object at different distances.

3. The method of claim 1, wherein determining a disparity comprises:

detecting the at least one object in the left 2D image and the right 2D image; and

computing the disparity as a difference between a horizontal offset of a bounding box of the at least one object in the left 2D image and a horizontal offset of a bounding box of the at least one object in the right 2D image.

4. The method of claim 1, wherein performing color matching comprises:

computing an average R value, an average G value, and an average B value for each of a reference block of pixels in a reference image and a non-reference block of pixels in a non-reference image, wherein the reference image is one of the left 2D image and the right 2D image and the non-reference image is another of the left 2D image and the right 2D image, and wherein the non-reference block of pixels is a block of pixels in the non-reference image corresponding to the reference block of pixels;

computing an R gain, a G gain, and a B gain as respective ratios of the average R values of the non-reference block of pixels and the reference block of pixels, the average G values of the non-reference block of pixels and the reference block of pixels, and the average B values of the non-reference block of pixels and the reference block of pixels; and

applying the R gain, the G gain, and the B gain to the non-reference image.

5. The method of claim 4, wherein the reference block of pixels is the entire reference image and the non-reference block of pixels is the entire non-reference image.

6. The method of claim 4, wherein performing color matching further comprises:

searching a search area of the non-reference image to locate a block of pixels that best matches the reference block of pixels, wherein the search area is a subset of the non-reference image; and

selecting the block of pixels as the non-reference block of pixels.

7. The method of claim 1, wherein performing contrast and brightness matching comprises:

computing a reference luminance histogram of a reference image and a non-reference luminance histogram of a non-reference image, wherein the reference image is one of the left 2D image and the right 2D image and the non-reference image is another of the left 2D image and the right 2D image;

computing a mapping function to match the non-reference luminance histogram to the reference luminance histogram; and

applying the mapping function to the non-reference image.

8. The method of claim 1, wherein computing a disparity image further comprises:

computing a first disparity image at a first resolution;

computing a second disparity image at a second resolution, wherein the second resolution is lower than the first resolution;

upsampling the second disparity image to the first resolution; and

filling holes in the first disparity image by interpolating disparity values in selected areas in the first disparity image with disparity values in corresponding selected areas in the upsampled second disparity image.

9. The method of claim 8, wherein computing a disparity image further comprises:

computing a hole binary mask for the first disparity image, wherein the hole binary mask identifies holes in the first disparity image, and

using the hole binary mask to identify the selected areas.

10. A stereo image processing system comprising:

a first imaging component arranged to capture a left two-dimensional (2D) image of a scene;

a second imaging component arranged to capture a right 2D image of a scene;

means for performing color matching of the left and right 2D images, wherein performing color matching comprises;

computing an average R value, an average G value, and an average B value for each of a reference block of pixels in a reference image and a non-reference block of pixels in a non-reference image, wherein the reference image is one of the left 2D image and the right 2D image and the non-reference image is another of the left 2D image and the right 2D image, and wherein the non-reference block of pixels is a block of pixels in the non-reference image corresponding to the reference block of pixels;

computing an R gain, a G gain, and a B gain as respective ratios of the average R values of the non-reference block of pixels and the reference block of pixels, the average G values of the non-reference block of pixels and the reference block of pixels, and the average B values of the non-reference block of pixels and the reference block of pixels; and

applying the R gain, the G gain, and the B gain to the non-reference image.

means for performing contrast and brightness matching of the left and right 2D images; and

means for computing a disparity image for the left and right 2D images after the color matching and the contrast and brightness matching are performed.

11. The stereo image processing system of claim 10, wherein the reference block of pixels is the entire reference image and the non-reference block of pixels is the entire non-reference image.

12. The stereo image processing system of claim 10, wherein performing contrast and brightness matching comprises:

computing a reference luminance histogram of a reference image and a non-reference luminance histogram of a non-reference image, wherein the reference image is one of the left 2D image and the right 2D image and the non-reference image is another of the left 2D image and the right 2D image;

computing a mapping function to match the non-reference luminance histogram to the reference luminance histogram; and

applying the mapping function to the non-reference image.

13. The stereo image processing system of claim 10, further comprising:

means for determining a disparity range based on a disparity of at least one object in a scene of the left and right 2D images, and

wherein the means for computing a disparity map uses the disparity range for correspondence matching of the left and right 2D images.

14. The stereo image processing system of claim 13, wherein the means for determining a disparity comprises:

means for detecting the at least one object in the left 2D image and the right 2D image; and

means for computing the disparity as a difference between a horizontal offset of a bounding box of the at least one object in the left 2D image and a horizontal offset of a bounding box of the at least one object in the right 2D image.

15. The stereo image processing system of claim 10, wherein the means for computing a disparity image comprises:

means for computing a first disparity image at a first resolution;

means for computing a second disparity image at a second resolution, wherein the second resolution is lower than the first resolution;

means for upsampling the second disparity image to the first resolution; and

means for filling holes in the first disparity image by interpolating disparity values in selected areas in the first disparity image with disparity values in corresponding selected areas in the upsampled second disparity image.

16. The stereo image processing system of claim 10, wherein the means for computing a disparity image further comprises:

means for computing a hole binary mask for the first disparity image, wherein the hole binary mask identifies holes in the first disparity image, and

means for using the hole binary mask to identify the selected areas.

17. A non-transitory computer-readable medium storing software instructions that, when executed by a processor, perform a method for computation of a disparity map, the method comprising:

performing color matching of the left and right 2D images;

performing contrast and brightness matching of the left and right 2D images; and

computing a disparity image for the left and right 2D images after the color matching and the contrast and brightness matching are performed, wherein computing a disparity image comprises;

computing a first disparity image at a first resolution;

computing a second disparity image at a second resolution, wherein the second resolution is lower than the first resolution;

upsampling the second disparity image to the first resolution; and

filling holes in the first disparity image by interpolating disparity values in selected areas in the first disparity image with disparity values in corresponding selected areas in the upsampled second disparity image.

18. The computer-readable medium of claim 17, wherein computing a disparity image further comprises:

computing a hole binary mask for the first disparity image, wherein the hole binary mask identifies holes in the first disparity image, and

using the hole binary mask to identify the selected areas.

19. The computer-readable medium of claim 17, wherein the method further comprises:

determining a disparity range based on a disparity of at least one object in a scene of the left and right 2D images; and

wherein the disparity range is used for correspondence matching of the left and right 2D images when computing the first and second disparity images.

20. The computer-readable medium of claim 17, wherein performing color matching comprises:

computing an average R value, an average G value, and an average B value for each of a reference block of pixels in a reference image and a non-reference block of pixels in a non-reference image, wherein the reference image is one of the left 2D image and the right 2D image and the non-reference image is another of the left 2D image and the right 2D image, and wherein the non-reference block of pixels is a block of pixels in the non-reference image corresponding to the reference block of pixels;

computing an R gain, a G gain, and a B gain as respective ratios of the average R values of the non-reference block of pixels and the reference block of pixels, the average G values of the non-reference block of pixels and the reference block of pixels, and the average B values of the non-reference block of pixels and the reference block of pixels; and

applying the R gain, the G gain, and the B gain to the non-reference image.