Method and device for high-resolution imaging which obtains camera pose using defocusing

• US 9,247,235 B2
• Filed: 07/15/2013
• Issued: 01/26/2016
• Est. Priority Date: 08/27/2008
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Gharib, Morteza, Lu, Jian, Hsieh, Scott, Pereira, Francisco, Harvard, Alexei
• Primary Examiner(s): Pollack, Melvin H
• Application Number:
  US13/942,576
• Publication Number:
  US 20140022350A1
• Time in Patent Office: 925 Days
• Field of Search: 348/46
• US Class Current: 1/1
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/236 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects

Abstract

A method and device for high-resolution three-dimensional (3-D) imaging which obtains camera pose using defocusing is disclosed. The device comprises a lens obstructed by a mask having two sets of apertures. The first set of apertures produces a plurality of defocused images of the object, which are used to obtain camera pose. The second set of optical filters produces a plurality of defocused images of a projected pattern of markers on the object. The images produced by the second set of apertures are differentiable from the images used to determine pose, and are used to construct a detailed 3-D image of the object. Using the known change in camera pose between captured images, the 3-D images produced can be overlaid to produce a high-resolution 3-D image of the object.

Claims

1. An imaging device for producing a high-resolution three-dimensional (3-D) image of an object, comprising;

a lens obstructed by a mask having a first set and a second set of off-axis apertures where;

the first set of off-axis apertures comprises a plurality of apertures fitted with filter separators for capturing a plurality of defocused images of an object simultaneously; and

the second set of off-axis apertures comprises a plurality of apertures fitted with filter separators differentiable from those of the first set of off-axis apertures for capturing a plurality of defocused images,

a projector for projecting a pattern of markers on the surface of the object, the projected pattern of markers being of a wavelength corresponding to the filter separators of the second set of off-axis apertures;

a sensor configured to capture the defocused images produced; and

a data processing system, the data processing system having one or more processors configured to, by defocusing without stereo imaging, use the images captured on the sensor through the first set of off-axis apertures to determine camera pose, and use the images captured on the sensor through the second set of off-axis apertures to construct a high-resolution three-dimensional image of the object.

2. The device of claim 1, wherein the data processing system is further configured to:

capture, at an initial time, an initial plurality of defocused images of an object simultaneously with a sensor, from an initial sensor pose;

extract three-dimensional (3-D) locations of three or more object features from the relative locations of the object features in the initial plurality of defocused images on the sensor;

capture, at a subsequent time, a subsequent plurality of defocused images of the object simultaneously, from a subsequent sensor pose;

extract three-dimensional (3-D) locations of three or more object features from the relative locations of the object features in the subsequent plurality of defocused images on the sensor;

match the object features from the initial plurality of defocused images with corresponding object features from the subsequent plurality of defocused images using a technique selected from the group consisting of;

a feature matching algorithm; and

an error minimization method; and

calculate a change in pose of the sensor between the initial and subsequent times using the 3-D locations extracted from the initial and subsequent pluralities of defocused images, whereby the change in pose of the moving sensor is determined.

3. The device of claim 2, wherein the data processing system is further configured to:

project a pattern of markers on the object;

capture a plurality of defocused images of the projected pattern of markers with the sensor, the defocused images being differentiable from the initial and subsequent pluralities of defocused images of the object used for determining sensor pose; and

construct a 3-D image of a surface of the object based on relative positions of the plurality of defocused images of the projected pattern of markers on the sensor.

4. The device of claim 3, wherein the data processing system is further configured to calculate an absolute pose of the sensor relative to an environment by:

capturing a plurality of defocused images of three or more fixed features of known position in the environment with the sensor;

extracting 3-D locations of the three or more fixed features in the environment from the relative locations of the fixed features in the plurality of defocused images on the sensor; and

calculating the absolute pose of the sensor using the 3-D locations extracted from the plurality of defocused images.

5. The device of claim 4, wherein each of the plurality of apertures comprising the first set of off-axis apertures is fitted with a different type of filter separator, such that the defocused images produced through said apertures are differentiable from one another.

6. The device of claim 2, wherein the data processing system is further configured to calculate an absolute pose of the sensor relative to an environment by:

capturing a plurality of defocused images of three or more fixed features of known position in the environment with the sensor;

extracting 3-D locations of the three or more fixed features in the environment from the relative locations of the fixed features in the plurality of defocused images on the sensor; and

calculating the absolute pose of the sensor using the 3-D locations extracted from the plurality of defocused images.

7. The device of claim 1, wherein each of the plurality of apertures comprising the first set of off-axis apertures is fitted with a different type of filter separator, such that the defocused images produced through said apertures are differentiable from one another.