Symmetric stereo model for handling occlusion

• US 7,599,547 B2
• Filed: 11/30/2005
• Issued: 10/06/2009
• Est. Priority Date: 11/30/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Yin, Kang, Sing Bing, Sun, Jian, Shum, Heung-Yeung
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): CHU, RANDOLPH I
• Application Number:
  US11/289,907
• Publication Number:
  US 20070122028A1
• Time in Patent Office: 1,406 Days
• Field of Search: 382/154, 345419-427, 356/12
• US Class Current: 382/154
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

The present symmetric stereo matching technique provides a method for iteratively estimating a minimum energy for occlusion and disparity using belief propagation. The minimum energy is based on an energy minimization framework in which a visibility constraint is embedded. By embedding the visibility constraint, the present symmetric stereo matching technique treats both images equally, instead of treating one as a reference image. The visibility constraint ensures that occlusion in one view and the disparity in another view are consistent.

Claims

1. A computing device, comprising:

a processor;

a memory into which a plurality of instructions are loaded, the plurality of instructions performing a method when executed by the processor, the method comprising;

iteratively estimating a minimum energy for both occlusion and disparity for at least two views of an image using belief propagation, wherein the minimum energy is based on an energy minimization framework in which a visibility constraint is embedded, wherein a smoothness strength parameter within the energy minimization framework is automatically set based on a similarity between each data term for neighboring pixels in the energy minimization framework, wherein the similarity between neighboring pixels is measured with a symmetric version of a Kullback-Lieber divergence and the average is computed over the entire image.

2. The computing device of claim 1, wherein the visibility constraint ensures that occlusion in one view and disparity in the other views are consistent.

3. The computing device of claim 1, wherein the at least two views comprise a left view and a right view for a two-frame stereo image.

4. The computing device of claim 1, wherein the belief propagation comprises a max-product version of belief propagation.

5. The computing device of claim 1, wherein the energy minimization framework further comprises a soft constraint for handling segmentation.

6. The computing device of claim 1, wherein the smoothness strength parameter is estimated by multiplying a constant by the average Kullback-Lieber divergence.

7. The computing device of claim 6, wherein the constant comprises a value of 5.75.

8. The computing device of claim 1, wherein iteratively estimating the disparity and the occlusion concludes upon a pre-defined iteration number.

9. The computing device of claim 3, wherein the visibility constraint ensures that an occluded pixel in the right view does not have a match in the left view and that a non-occluded pixel in the right view has at least one match in the left view.

10. A computer-implemented method comprising:

storing, in a memory, instructions for performing the method;

executing the instructions on a processor;

according to the instruction being executed;

iteratively estimating a minimum energy for both occlusion and disparity for at least two views of an image using belief propagation, wherein the minimum energy is based on an energy minimization framework in which a visibility constraint is embedded, wherein a smoothness strength parameter within the energy minimization framework is automatically set based on a similarity between each data term for neighboring pixels in the energy minimization framework, wherein the similarity between neighboring pixels is measured with a symmetric version of a Kullback-Lieber divergence and the average is computed over the entire image.

11. The computer-implemented method of claim 10, wherein the visibility constraint ensures that occlusion in one view and disparity in the other views are consistent.

12. The computer-implemented method of claim 10, wherein the at least two views comprise a left view and a right view for a two-frame stereo image.

13. The computer-implemented method of claim 10, wherein the belief propagation comprises a max-product version of belief propagation.

14. The computer-implemented method of claim 10, wherein the energy minimization framework further comprises a soft constraint for handling segmentation.

15. The computer-implemented method of claim 10, wherein the smoothness strength parameter is estimated by multiplying a constant by the average Kullback-Lieber divergence.

16. The computer-implemented method of claim 15, wherein the constant comprises a value of 5.75.

17. The computer-implemented method of claim 10, wherein iteratively estimating the disparity and the occlusion concludes upon a pre-defined iteration number.

18. The computer-implemented method of claim 12, wherein the visibility constraint ensures that an occluded pixel in the right view does not have a match in the left view and that a non-occluded pixel in the right view has at least one match in the left view.