Enhanced object reconstruction

• US 7,822,267 B2
• Filed: 06/24/2008
• Issued: 10/26/2010
• Est. Priority Date: 01/07/2005
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Gu, Jin
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US12/145,194
• Publication Number:
  US 20090003686A1
• Time in Patent Office: 854 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Processing images includes projecting an infra-red pattern onto a three-dimensional object and producing a first image, a second image, and a third image of the three-dimensional object while the pattern is projected on the three-dimensional object. The first image and the second image include the three-dimensional object and the pattern. The first image and the second image are produced by capturing at a first camera and a second camera, respectively, light filtered through an infra-red filter. The third image includes the three-dimensional object but not the pattern. Processing the images also includes establishing a first-pair correspondence between a portion of pixels in the first image and a portion of pixels in the second image. Processing the images further includes constructing, based on the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

Claims

1. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

detect first and second stripes projected onto an object, in both of first and second images;

extract, for a raster profile and from the first or second images, an intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively;

determine a local minimum in the intensity profile amongst the first and second segments;

designate the first or second stripe as a reference stripe based on the determined local minimum;

match the reference stripe in the first and second images; and

construct a three dimensional surface based on matching the reference stripe in the first and second images.

2. The computer program product of claim 1, wherein the first and second stripes are projected onto the object using infrared light.

3. The computer program product of claim 1, further comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

control a disparity propagation along the reference stripe,

wherein the three dimensional surface is constructed further based on controlling the disparity propagation.

17. The computer program product of claim 1, wherein the instructions operate to cause data processing apparatus to extract, for the raster profile and from the first or second images, the intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, by extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second segments of the raster line associated with the first and second stripes, respectively.

18. The computer program product of claim 1, wherein the instructions operate to cause data processing apparatus to extract, for the raster profile and from the first or second images, the intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, by extracting, for a raster profile and from the first or second images, an intensity profile of first and second pixels located on the raster line associated with the first and second stripes, respectively.

19. The computer program product of claim 1, wherein the instructions operate to cause data processing apparatus to extract, for the raster profile and from the first or second images, the intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, by extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second pixels located on the raster line associated with the first and second stripes, respectively.

4. A computer-implemented method comprising:

detecting first and second stripes projected onto an object, in both of first and second images;

extracting, for a raster profile and from the first or second images, an intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively;

determining a local minimum in the intensity profile amongst the first and second segments;

designating the first or second stripe as a reference stripe based on the determined local minimum;

matching the reference stripe in the first and second images; and

construct a three dimensional surface based on matching the reference stripe in the first and second images.

5. The method of claim 4, wherein the first and second stripes are projected onto the object using infrared light.

6. The method of claim 4, further comprising:

controlling a disparity propagation along the reference stripe,

wherein the three dimensional surface is constructed further based on controlling the disparity propagation.

14. The method of claim 4, wherein extracting, for a raster profile and from the first or second images, an intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, comprises extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second segments of the raster line associated with the first and second stripes, respectively.

15. The method of claim 4, wherein extracting, for a raster profile and from the first or second images, an intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, comprises extracting, for a raster profile and from the first or second images, an intensity profile of first and second pixels located on the raster line associated with the first and second stripes, respectively.

16. The method of claim 4, wherein extracting, for a raster profile and from the first or second images, an intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, comprises extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second pixels located on the raster line associated with the first and second stripes, respectively.

7. A device comprising a processor configured to:

detect first and second stripes projected onto an object, in both of first and second images;

extract, for a raster profile and from the first or second images, an intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively;

determine a local minimum in the intensity profile amongst the first and second segments;

designate the first or second stripe as a reference stripe based on the determined local minimum;

match the reference stripe in the first and second images; and

construct a three dimensional surface based on matching the reference stripe in the first and second images.

8. The device of claim 7, wherein the first and second stripes are projected onto the object using infrared light.

9. The device of claim 7, wherein the processor is further configured to:

control a disparity propagation along the reference stripe,

wherein the three dimensional surface is constructed further based on controlling the disparity propagation.

10. The device of claim 7, further comprising a camera configured to capture the first and second images.

11. The device of claim 7, wherein the processor is configured to extract, for the raster profile and from the first or second images, the intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, by extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second segments of the raster line associated with the first and second stripes, respectively.

12. The device of claim 7, wherein the processor is configured to extract, for the raster profile and from the first or second images, the intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, by extracting, for a raster profile and from the first or second images, an intensity profile of first and second pixels located on the raster line associated with the first and second stripes, respectively.

13. The device of claim 7, wherein the processor is configured to extract, for the raster profile and from the first or second images, the intensity profile of first and second segments of the raster line associated with the first and second stripes, respectively, by extracting, for a raster profile and from the first or second images, an intensity profile including a curve representing intensity values of first and second pixels located on the raster line associated with the first and second stripes, respectively.