Three-dimensional mapping and imaging

• US 8,717,417 B2
• Filed: 04/12/2010
• Issued: 05/06/2014
• Est. Priority Date: 04/16/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: PrimeSense Ltd (Apple Inc.)
• Inventors: Sali, Erez, Avraham, Assaf
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): BELAI, NAOD W
• Application Number:
  US12/758,047
• Publication Number:
  US 20100265316A1
• Time in Patent Office: 1,485 Days
• Field of Search: 348/46, 348/266, 345/8, 356/603, 382/118, 250/208
• US Class Current: 348/46
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects

Abstract

Imaging apparatus includes an illumination subassembly, which is configured to project onto an object a pattern of monochromatic optical radiation in a given wavelength band. An imaging subassembly includes an image sensor, which is configured both to capture a first, monochromatic image of the pattern on the object by receiving the monochromatic optical radiation reflected from the object and to capture a second, color image of the object by receiving polychromatic optical radiation, and to output first and second image signals responsively to the first and second images, respectively. A processor is configured to process the first and second signals so as to generate and output a depth map of the object in registration with the color image.

Claims

1. Imaging apparatus, comprising:

an illumination subassembly, which is configured to project onto an object a pattern of monochromatic optical radiation in an infrared (IR) wavelength band;

an imaging subassembly comprising;

an image sensor, which comprises an array of detector elements and is configured to capture;

a first, monochromatic image of the pattern on the object by receiving the monochromatic optical radiation reflected from the object;

a second, color image of the object by receiving polychromatic optical radiation; and

a third image representative of an ambient background radiation level in the IR wavelength band exclusive of the projected pattern, and

to output first, second and third image signals responsively to the first, second and third images, respectively; and

a mosaic filter comprising a pattern of filter elements overlaid respectively on the detector elements, wherein the filter elements comprise second filter elements configured to pass different, respective visible wavelength bands, and first filter elements configured to pass the IR band of the monochromatic optical radiation, and

wherein the first and third image signals are generated by the detector elements that are overlaid by the first filter elements, while the second image signal is generated by the detector elements that are overlaid by the second filter elements; and

a processor, which is configured to process the first, second and third signals so as to generate and output a depth map of the object in registration with the color image,

wherein the depth map is generated by subtracting the third image signals from the first image signals to produce a difference signal, and processing the difference signal in order to generate the depth map.

2. The apparatus according to claim 1, wherein the IR wavelength band has a bandwidth that is no greater than 20 nm.

3. The apparatus according to claim 1, wherein the first filter elements comprise red, green and blue filter elements, and the second filter elements comprise infrared (IR) filter elements, which are interleaved with the first filter elements in a rectilinear pattern.

4. A method for imaging, comprising:

projecting onto an object a pattern of monochromatic optical radiation in an infrared (IR) wavelength band;

capturing, using an image sensor, which comprises an array of detector elements, both a first, monochromatic image of the pattern on the object by receiving the monochromatic optical radiation reflected from the object and a second, color image of the object by receiving polychromatic optical radiation,

wherein capturing the first and second images comprises filtering the optical radiation impinging on the image sensor using a mosaic filter, which comprises a pattern of filter elements overlaid respectively on the detector elements,

wherein the filter elements comprise second filter elements configured to pass different, respective visible wavelength bands, and first filter elements configured to pass the IR wavelength band of the monochromatic optical radiation;

capturing, using the image sensor, a third image representative of a background radiation level in the IR wavelength band exclusive of the projected pattern, and outputting a third image signal responsively to the third image;

outputting from the image sensor first, second and third image signals responsively to the first, second and third images, respectively,

wherein the first and third image signals are generated by the detector elements that are overlaid by the first filter elements, while the second image signal is generated by the detector elements that are overlaid by the second filter elements; and

processing the first, second and third signals so as to generate a depth map of the object in registration with the color image,

wherein the depth map is generated by subtracting the third image signals from the first image signals to produce a difference signal, and processing the difference signal in order to generate the depth map.

5. The method according to claim 4, wherein the IR wavelength band has a bandwidth that is no greater than 20 nm.

6. The method according to claim 4, wherein the first filter elements comprise red, green and blue filter elements, and the second filter elements comprise infrared (IR) filter elements, which are interleaved with the first filter elements in a rectilinear pattern.