Creating 3D images of objects by illuminating with infrared patterns

• US 7,570,805 B2
• Filed: 04/23/2008
• Issued: 08/04/2009
• Est. Priority Date: 01/07/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: Gu, Jin
• Primary Examiner(s): Carter; Aaron W
• Application Number:
  US12/108,154
• Publication Number:
  US 20080199071A1
• Time in Patent Office: 468 Days
• Field of Search: 382/154, 382/285, 345419-427, 356 12- 14, 348 42- 60, 359462-477, 352 57- 65, 33/204, 353 7- 9, 378 41- 42, 396324-331
• US Class Current: 382/154
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

According to a general aspect, processing images includes projecting an infra-red pattern onto a three-dimensional object and producing a first image, a second image, and a third image of the three-dimensional object while the pattern is projected on the three-dimensional object. The first image and the second image include the three-dimensional object and the pattern. The first image and the second image are produced by capturing at a first camera and a second camera, respectively, light filtered through an infra-red filter. The third image includes the three-dimensional object but not the pattern. Processing the images also includes establishing a first-pair correspondence between a portion of pixels in the first image and a portion of pixels in the second image. Processing the images further includes constructing, based on the first-pair correspondence and the third image, a two-dimensional image that depicts a three-dimensional construction of the three-dimensional object.

Claims

1. A computer-implemented method comprising:

using at least one processor to perform operations comprising;

generating first and second images of an object illuminated with a pattern;

establishing a first-pair correspondence between initial matched pixels of the first and second image based on detecting the illuminated pattern in the first and second images;

identifying a depth discontinuity associated with the object based on a deformation of the illuminated pattern;

controlling a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity; and

generating a three-dimensional reconstruction of the object based on the controlled disparity propagation.

2. The method of claim 1, wherein the object is illuminated using infrared light.

3. The method of claim 1, wherein the pattern comprises a stripe.

4. The method of claim 1, wherein identifying the depth discontinuity further comprises identifying a break in the pattern.

5. A device comprising:

a camera configured to generate first and second images of an object illuminated with a pattern; and

a processor configured to;

establish a first-pair correspondence between initial matched pixels of the first and second image based on detecting the illuminated pattern in the first and second images,

identify a depth discontinuity associated with the object based on a deformation of the illuminated pattern,

control a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity, and

generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.

6. A computer-implemented method comprising:

using at least one processor to perform operations comprising;

receiving first and second pairs of images in a sequence of paired images of an object illuminated with a pattern; and

for each of the first and second pairs of images;

establishing a first-pair correspondence between initial matched pixels of the paired images based on detecting the illuminated pattern in the paired images,

identifying a depth discontinuity associated with the object based on a deformation of the illuminated pattern,

controlling a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity, and

generating a three-dimensional reconstruction of the object based on the controlled disparity propagation.

7. The method of claim 6, wherein the object is illuminated using infrared light.

8. The method of claim 6, wherein the pattern comprises a stripe.

9. The method of claim 6, wherein, for each of the first and second pairs of images, identifying the depth discontinuity further comprises identifying a break in the pattern.

10. A device comprising:

a camera configured to generate first and second pairs of images in a sequence of paired images of an object illuminated with a pattern; and

a processor configured, for each of the first and second pairs of images, to;

establish a first-pair correspondence between initial matched pixels of the paired images based on detecting the illuminated pattern in the paired images,

identify a depth discontinuity associated with the object based on a deformation of the illuminated pattern,

control a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity, and

generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.

11. A computer-implemented method comprising:

using at least one processor to perform operations comprising;

receiving first and second images of an object illuminated with a pattern of uncoded stripes;

establishing a first-pair correspondence between initial matched pixels of the paired images based on detecting the illuminated pattern in the paired images;

identifying a depth discontinuity associated with the object based on a deformation of the illuminated pattern;

controlling a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity; and

generating a three-dimensional reconstruction of the object based on the controlled disparity propagation.

12. The method of claim 11, wherein the object is illuminated using infrared light.

13. The method of claim 11, wherein identifying the depth discontinuity further comprises identifying a break in the pattern.

14. The method of claim 11, wherein identifying the depth discontinuity further comprises identifying wherein first and second uncoded stripes join each other.

15. A device comprising:

a camera configured to generate first and second images of an object illuminated with a pattern of uncoded stripes; and

a processor configured to;

establish a first-pair correspondence between initial matched pixels of the paired images based on detecting the illuminated pattern in the paired images,

identify a depth discontinuity associated with the object based on a deformation of the illuminated pattern,

control a disparity propagation of the initial matched pixels in two directions based on the identified depth discontinuity, and

generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.

16. A readable computer-readable storage medium, encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to:

identify a depth discontinuity associated with an object illuminated with a pattern, based on a deformation of the illuminated pattern;

control a disparity propagation in two directions based on the identified depth discontinuity; and

generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.

17. The computer-readable storage medium of claim 16, wherein the object is illuminated using infrared light.

18. The computer-readable storage medium of claim 16, wherein the pattern comprises a stripe.

19. A computer-implemented method comprising:

using at least one processor to perform operations comprising;

identifying a depth discontinuity associated with an object illuminated with a pattern, based on a deformation of the illuminated pattern;

controlling a disparity propagation in two directions based on the identified depth discontinuity; and

generating a three-dimensional reconstruction of the object based on the controlled disparity propagation.

20. A device comprising a processor configured to:

identify a depth discontinuity associated with an object illuminated with a pattern, based on a deformation of the illuminated pattern;

control a disparity propagation in two directions based on the identified depth discontinuity; and

generate a three-dimensional reconstruction of the object based on the controlled disparity propagation.