ENHANCED OBJECT RECONSTRUCTION

• US 20110038530A1
• Filed: 10/26/2010
• Published: 02/17/2011
• Est. Priority Date: 01/07/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Gu, Jin
• Granted Patent: US 7,953,271 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Processing images includes projecting an infra-red pattern onto a three-dimensional object and producing a first image, a second image, and a third image of the three-dimensional object while the pattern is projected on the three-dimensional object. The first image and the second image include the three-dimensional object and the pattern. The first image and the second image are produced by capturing at a first camera and a second camera, respectively, light filtered through an infra-red filter. The third image includes the three-dimensional object but not the pattern. Processing the images also includes establishing a first-pair correspondence between a portion of pixels in the first image and a portion of pixels in the second image. Processing the images further includes constructing, based on the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

Claims

1. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

mark explicit and implicit ending parts on a stripe projected onto an object;

perform a disparity propagation routine in a horizontal direction, a vertical direction, and the horizontal direction, respectively, further comprising halting the disparity propagation routine in the horizontal or vertical direction based on encountering the explicit or implicit ending parts;

construct a continuous three dimensional surface based on the performed disparity propagation; and

perform a mismatch detection using the constructed three dimensional surface if a distance between matched pixels is greater than a predefined threshold.

2. The computer program product of claim 1,

wherein the explicit ending parts correspond to an end of the stripe or a break in the stripe, and

wherein the implicit ending parts include a pixel on the stripe adjacent to one of the explicit ending parts of a second stripe, resulting when the stripe and the second stripe join each other due to a depth discontinuity.

3. The computer program product of claim 1, wherein the stripe is projected onto the object using infrared light.

4. The computer program product of claim 1, further comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

establish a correspondence between an edge stripe pixel in a first image and a corresponding edge stripe pixel in a second image,

wherein the disparity propagation routine is performed based on establishing the correspondence.

5. The computer program product of claim 1, wherein:

the object is a user'"'"'s body; and

an explicit ending part is marked between a jaw and a shoulder of the user'"'"'s body.

6. The computer program product of claim 1, wherein performing the disparity routine in the horizontal direction further comprises propagating a disparity along a raster line.

7. The computer program product of claim 1, wherein performing the disparity routine in the horizontal direction further comprises propagating a disparity across the stripe in both left and right directions.

8. The computer program product of claim 1, wherein the disparity routine is performed in the horizontal direction, the vertical direction, and the horizontal direction, respectively.

9. A computer-implemented method comprising:

marking explicit and implicit ending parts on a stripe projected onto an object;

performing a disparity propagation routine in a horizontal direction, a vertical direction, and the horizontal direction, respectively, further comprising halting the disparity propagation routine in the horizontal or vertical direction based on encountering the explicit or implicit ending parts;

constructing a continuous three dimensional surface based on the performed disparity propagation; and

performing a mismatch detection using the constructed three dimensional surface if a distance between matched pixels is greater than a predefined threshold.

10. A device comprising a processor configured to:

mark explicit and implicit ending parts on a stripe projected onto an object;

perform a disparity propagation routine in a horizontal direction, a vertical direction, and the horizontal direction, respectively, further comprising halting the disparity propagation routine in the horizontal or vertical direction based on encountering the explicit or implicit ending parts;

construct a continuous three dimensional surface based on the performed disparity propagation; and

perform a mismatch detection using the constructed three dimensional surface if a distance between matched pixels is greater than a predefined threshold.