Single-sensor system for extracting depth information from image blur

• US 9,530,213 B2
• Filed: 12/30/2013
• Issued: 12/27/2016
• Est. Priority Date: 01/02/2013
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Gharib, Morteza, Jeon, David
• Primary Examiner(s): Bailey, Frederick
• Assistant Examiner(s): Nawaz, Talha M
• Application Number:
  US14/143,832
• Publication Number:
  US 20140184748A1
• Time in Patent Office: 1,093 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/236 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Hardware and software methodology are described for three-dimensional imaging in connection with a single sensor. A plurality of images is captured at different degrees of focus without focus change of an objective lens between such images. Depth information is extracted by comparing image blur between the images captured on the single sensor.

Claims

1. A three-dimensional imaging system comprising:

a single image sensor;

optics adapted to allow the single image sensor to capture images of an object in a scene along a common optical axis, wherein the optics are adapted to allow the single image sensor to capture the images such that they are focused to blur in different degrees without a change of objective lens focus or position of the object, wherein the optics comprise a beam splitter and an optical delay, the optical delay adapted to lengthen a beam path to the single image sensor;

a computer processor connected with the single image sensor; and

a non-transitory computer readable medium having instructions stored thereon, which, when executed cause the computer processor to;

match elements from the images; and

determine depth position for elements of the images by a comparison of differences in blurring for the elements.

2. The system of claim 1, wherein the optics comprise a moving element selected from a reflective element and a refractive element.

3. The system of claim 1, wherein the delay is based on an optical switchback.

4. The system of claim 1, wherein the delay is based on material refractive index.

5. The system of claim 1, wherein the optics comprise an array of micro-lenses.

6. The system of claim 5, wherein portions of the single image sensor are uncovered by the array.

7. The system of claim 5, wherein the array includes micro-lenses of at least two focal lengths.

8. The system of claim 1, further comprising a single objective lens assembly.

14. The method of claim 9, employing a system as described in any of claims 1, 2, and 5-9.

9. A method of three-dimensional imaging comprising:

capturing, with a single image sensor, a plurality of images of an object in a scene along a common optical axis, wherein the plurality of images are captured at different focuses without moving a camera objective lens focus or a position of the object, wherein light incident on the single image sensor passes through a beam splitter and an optical delay, the optical delay adapted to lengthen a beam path to the single image sensor; and

using a computer processor, matching elements from the images and determining depth position for the elements of the images by comparing differences in size for the elements.

10. The method of claim 9, wherein the images are captured in a time-separated fashion on overlapping portions of the image sensor.

11. The method of claim 10, further comprising moving a refractive or reflective element to allow the images to be captured in a time-separated fashion.

12. The method of claim 9, wherein the images are captured simultaneously on different portions of the image sensor without overlap.

13. The method of claim 12, wherein the different focuses are captured at different-distance image planes on the image sensor.