Enhanced object reconstruction

• US 8,218,858 B2
• Filed: 05/27/2011
• Issued: 07/10/2012
• Est. Priority Date: 01/07/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gu, Jin
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/117,727
• Publication Number:
  US 20110262032A1
• Time in Patent Office: 410 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Processing images includes projecting an infra-red pattern onto a three-dimensional object and producing a first image, a second image, and a third image of the three-dimensional object while the pattern is projected on the three-dimensional object. The first image and the second image include the three-dimensional object and the pattern. The first image and the second image are produced by capturing at a first camera and a second camera, respectively, light filtered through an infra-red filter. The third image includes the three-dimensional object but not the pattern. Processing the images also includes establishing a first-pair correspondence between a portion of pixels in the first image and a portion of pixels in the second image. Processing the images further includes constructing, based on the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

Claims

1. A computer program product, stored using a non-transitory machine-readable medium, the computer program product comprising instructions that, when executed by a machine, cause the machine to:

mark an ending part on a pattern projected onto an object;

perform a disparity propagation routine in a first direction and a second direction, respectively, wherein;

the disparity propagation routine is halted in the first direction or the second direction based on encountering the ending part; and

construct a surface model of a three-dimensional surface using the performed disparity propagation routine.

2. The computer program product of claim 1 further comprising instructions, that when executed by the machine, cause the machine to:

perform a mismatch detection using the constructed surface model of the three-dimensional surface.

3. The computer program product of claim 1, wherein the first direction is horizontal and the second direction is vertical.

4. The computer program product of claim 1, wherein the instructions that, when executed by the machine, cause the machine to perform the disparity propagation routine in the first direction and the second direction, respectively, further comprise performing the disparity propagation routine in the first direction again after the second direction.

5. A method for constructing a surface model of a three-dimensional surface, the method comprising:

marking, by a computer system, an ending part on a pattern projected onto an object;

performing, by the computer system, a disparity propagation routine in a first direction and a second direction, respectively, wherein;

the disparity propagation routine is halted in the first direction or the second direction based on encountering the ending part; and

constructing, by the computer system, the surface model of the three-dimensional surface using the performed disparity propagation routine.

6. The method for constructing the surface model of the three-dimensional surface of claim 5, the method further comprising:

performing, by the computer system, a mismatch detection using the constructed surface model of the three-dimensional surface.

7. The method for constructing the surface model of the three-dimensional surface of claim 6, wherein performing the mismatch detection identifies a mismatch if a distance between a first three-dimensional point of a first pixel and a second three-dimensional point of a neighboring second pixel is greater than a threshold value.

8. The method for constructing the surface model of the three-dimensional surface of claim 5,wherein:

the ending part is an implicit ending part; and

the implicit ending part is adjacent to an explicit ending part of another portion of the pattern.

9. The method for constructing the surface model of the three-dimensional surface of claim 8,wherein:

the explicit ending part is not adjacent to another portion of the pattern; and

implicit ending parts and explicit ending parts each indicate existence of a depth discontinuity.

10. The method for constructing the surface model of the three-dimensional surface of claim 5, wherein the first direction is horizontal and the second direction is vertical.

11. The method for constructing the surface model of the three-dimensional surface of claim 5, wherein performing the disparity propagation routine in the first direction and the second direction, respectively, comprises performing the disparity propagation routine in the first direction again after the second direction.

12. The method for constructing the surface model of the three-dimensional surface of claim 5, wherein the pattern projected is a plurality of stripes.

13. The method for constructing the surface model of the three-dimensional surface of claim 5, wherein the object is a human face.

14. An apparatus for constructing a surface model of a three-dimensional surface, the method comprising:

means for marking an ending part on a pattern projected onto an object;

means for performing a disparity propagation routine in a first direction and a second direction, respectively, wherein;

the disparity propagation routine is halted in the first direction or the second direction based on encountering the ending part; and

means for constructing the surface model of the three-dimensional surface using the performed disparity propagation routine.

15. The apparatus for constructing the surface model of the three-dimensional surface of claim 14, wherein the first direction is horizontal and the second direction is vertical.

16. The apparatus for constructing the surface model of the three-dimensional surface of claim 14, wherein the pattern projected is a plurality of stripes.

17. A system for constructing a surface model of a three-dimensional surface, the system comprising:

a projector configured to output light in a pattern;

one or more cameras configured to capture the pattern of light output by the projector as superimposed on an object; and

a computer processor, configured to;

mark an ending part on the pattern projected onto the object;

perform a disparity propagation routine in a first direction and a second direction, respectively, wherein;

the disparity propagation routine is halted in the first direction or the second direction based on encountering the ending part; and

construct the surface model of the three-dimensional surface using the performed disparity propagation routine.

18. The system for constructing the surface model of the three-dimensional surface of claim 17, wherein the computer processor is further configured to:

perform a mismatch detection using the constructed surface model of the three-dimensional surface.

19. The system for constructing the surface model of the three-dimensional surface of claim 18, wherein the mismatch detection identifies a mismatch if a distance between a first three-dimensional point of a first pixel and a second three-dimensional point of a neighboring second pixel is greater than a threshold value.

20. The system for constructing the surface model of the three-dimensional surface of claim 17, wherein the ending part is an implicit ending part, wherein:

the implicit ending part is adjacent to an explicit ending part of another portion of the pattern.

21. The system for constructing the surface model of the three-dimensional surface of claim 20, wherein:

the explicit ending part is not adjacent to another portion of the pattern; and

implicit ending parts and explicit ending parts indicate existence of depth discontinuities.

22. The system for constructing the surface model of the three-dimensional surface of claim 17, wherein the first direction is horizontal and the second direction is vertical.

23. The system for constructing the surface model of the three-dimensional surface of claim 17, wherein the computer processor being configured to perform the disparity propagation routine in the first direction and the second direction, respectively, further comprises the computer processor being configured to perform the disparity propagation routine in the first direction again after the second direction.

24. The system for constructing the surface model of the three-dimensional surface of claim 17, wherein the pattern projected by the projector is a plurality of stripes using infrared light.

25. The system for constructing the surface model of the three-dimensional surface of claim 17, wherein the object is a human face.