ENHANCED OBJECT RECONSTRUCTION

• US 20090003686A1
• Filed: 06/24/2008
• Published: 01/01/2009
• Est. Priority Date: 01/07/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Gu, Jin
• Granted Patent: US 7,822,267 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Processing images includes projecting an infra-red pattern onto a three-dimensional object and producing a first image, a second image, and a third image of the three-dimensional object while the pattern is projected on the three-dimensional object. The first image and the second image include the three-dimensional object and the pattern. The first image and the second image are produced by capturing at a first camera and a second camera, respectively, light filtered through an infra-red filter. The third image includes the three-dimensional object but not the pattern. Processing the images also includes establishing a first-pair correspondence between a portion of pixels in the first image and a portion of pixels in the second image. Processing the images further includes constructing, based on the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

Claims

1. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

marking explicit and implicit ending parts on a stripe projected onto an object;

performing a disparity propagation routine in a horizontal direction, a vertical direction, and the horizontal direction, respectively, further comprising halting the disparity propagation routine in the horizontal or vertical direction based on encountering the explicit or implicit ending parts;

constructing a continuous three dimensional surface based on the performed disparity propagation; and

performing a mismatch detection using the constructed three dimensional surface if a distance between matched pixels is greater than a predefined threshold.

2. The computer-readable medium of claim 1,

wherein the explicit ending parts correspond to an end of the stripe or a break in the stripe, and

wherein the implicit ending parts include a pixel on the stripe adjacent to one of the explicit ending parts of a second stripe, resulting when the stripe and the second stripe join each other due to a depth discontinuity.

3. The computer-readable medium of claim 1, wherein the stripe is projected onto the object using infrared light.

4. The computer-readable medium of claim 1, further comprising instructions that, when executed, operate to cause the computer to perform operations comprising:

establishing a correspondence between an edge stripe pixel in a first image and a corresponding edge stripe pixel in a second image,

wherein the disparity propagation routine is performed based on establishing the correspondence.

5. The computer-readable medium of claim 1, wherein:

the object is a user'"'"'s body; and

an explicit ending part is marked between a jaw and a shoulder of the user'"'"'s body.

6. The computer-readable medium of claim 1, wherein performing the disparity routine in the horizontal direction further comprises propagating a disparity along a raster line.

7. The computer-readable medium of claim 1, wherein performing the disparity routine in the horizontal direction further comprises propagating a disparity across the stripe in both left and right directions.

8. The computer-readable medium of claim 1, wherein the disparity routine is performed in the horizontal direction, the vertical direction, and the horizontal direction, respectively.

9. A computer-implemented method comprising:

marking explicit and implicit ending parts on a stripe projected onto an object;

performing a disparity propagation routine in a horizontal direction, a vertical direction, and the horizontal direction, respectively, further comprising halting the disparity propagation routine in the horizontal or vertical direction based on encountering the explicit or implicit ending parts;

constructing a continuous three dimensional surface based on the performed disparity propagation; and

performing a mismatch detection using the constructed three dimensional surface if a distance between matched pixels is greater than a predefined threshold.

10. A device comprising a processor configured to:

mark explicit and implicit ending parts on a stripe projected onto an object;

perform a disparity propagation routine in a horizontal direction, a vertical direction, and the horizontal direction, respectively, further comprising halting the disparity propagation routine in the horizontal or vertical direction based on encountering the explicit or implicit ending parts;

construct a continuous three dimensional surface based on the performed disparity propagation; and

perform a mismatch detection using the constructed three dimensional surface if a distance between matched pixels is greater than a predefined threshold.

11. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

detecting first and second stripes projected onto an object, in both of first and second images;

extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second pixels located on the raster line associated with the first and second stripes, respectively;

determining a local minimum in the intensity profile amongst the first and second pixels;

designating the first or second stripe as a reference stripe based on the determined local minimum;

matching the reference stripe in the first and second images; and

constructing a three dimensional surface based on matching the reference stripe in the first and second images.

12. The computer-readable medium of claim 11, wherein the first and second stripes are projected onto the object using infrared light.

13. The computer-readable medium of claim 11, further comprising instructions that, when execute, operate to cause the computer to perform operations comprising:

controlling a disparity propagation along the reference stripe,

wherein the three dimensional surface is constructed further based on controlling the disparity propagation.

14. A computer-implemented method comprising:

detecting first and second stripes projected onto an object, in both of first and second images;

extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second pixels located on the raster line associated with the first and second stripes, respectively;

determining a local minimum in the intensity profile amongst the first and second pixels;

designating the first or second stripe as a reference stripe based on the determined local minimum;

matching the reference stripe in the first and second images; and

construct a three dimensional surface based on matching the reference stripe in the first and second images.

15. The method of claim 14, wherein the first and second stripes are projected onto the object using infrared light.

16. The method of claim 14, further comprising:

controlling a disparity propagation along the reference stripe,

wherein the three dimensional surface is constructed further based on controlling the disparity propagation.

17. A device comprising a processor configured to:

detect first and second stripes projected onto an object, in both of first and second images;

extract, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second pixels located on the raster line associated with the first and second stripes, respectively;

determine a local minimum in the intensity profile amongst the first and second pixels;

designate the first or second stripe as a reference stripe based on the determined local minimum;

match the reference stripe in the first and second images; and

construct a three dimensional surface based on matching the reference stripe in the first and second images.

18. The device of claim 17, wherein the first and second stripes are projected onto the object using infrared light.

19. The device of claim 17, wherein the processor is further configured to:

control a disparity propagation along the reference stripe,

wherein the three dimensional surface is constructed further based on controlling the disparity propagation.

20. The device of claim 17, further comprising a camera configured to capture the first and second images.