Accurate 3D object reconstruction using a handheld device with a projected light pattern

• US 8,773,514 B2
• Filed: 08/27/2010
• Issued: 07/08/2014
• Est. Priority Date: 08/27/2009
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Gharib, Morteza, Lu, Jian, Harvard, Alexei, Hsieh, Scott
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): AYNALEM, NATHNAEL B
• Application Number:
  US12/869,919
• Publication Number:
  US 20110074932A1
• Time in Patent Office: 1,411 Days
• Field of Search: 348 36- 39, 348/42, 348/49, 348/50, 348/51, 382/154
• US Class Current: 348/50
• CPC Class Codes: G01B 11/16 for measuring the deformation in a solid, e.g. optical strain gauge
• CPC Class Codes: G01B 11/167 by projecting a pattern on the object
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

A camera has a lens and aperture device for determining 3D information. A projector projects an optical pattern toward a surface. The camera has at least two off-axis apertures thereon, arranged to obtain an image of the projected pattern including defocused information. The camera is movable between different positions to image the surface from said different positions, and the projector is at a specified angle of at least 5° relative to said camera. A processor carries out a first operation using information received through the apertures to determine a pose of said camera, and to determine three dimensional information about the object based on a degree of deformation of said optical pattern on said surface indicative of a three dimensional surface. An embodiment projects a grid of laser dots and uses laser-dot defocusing for approximate Z and thus grid correspondence, which can greatly increase the working depth of the system.

Claims

1. A system, comprising:

a projector that projects an optical pattern toward a surface at an angle of projection;

a camera, including a first plate with at least two off-axis apertures thereon, arranged to obtain an image of the optical pattern that has been projected, said camera being movable between different positions to image the surface from said different positions,

said projector and said angle of projection being at a specified angle of at least 30°

relative to said camera; and

a processor that receives information from said camera, said information indicative of imaged scenes at different times, wherein said processor uses information received through said apertures to determine a relationship, based on the angle of projection of the optical pattern, between different amounts of deformation of the projected pattern and a feature of the surface, to determine a pose of said camera, and to determine three dimensional information about said surface based on a degree of deformation of said optical pattern on said surface, said pose and said three dimensional information obtained from multiple different poses of said camera, and wherein said processor combines together said three dimensional information from said multiple poses using each of a plurality of poses.

2. A system as in claim 1, wherein said projector projects a grid of laser dots.

3. A system as in claim 1, wherein said projector projects a random pattern of dots.

4. A system as in claim 1, wherein said projector projects a grid of optical dots, and said processor uses defocused information from said optical dots to determine a correspondence of the optical dots to the grid and to determine an approximate Z axis correspondence based on said defocused information.

5. A system as in claim 4, wherein said processor averages centroids of images within the defocused information.

6. A system as in claim 1, wherein said angle is at least 45°

.

7. A method, comprising:

projecting an optical pattern toward a surface;

receiving an image which has passed through a plate with at least two off-axis apertures in a camera, said camera being movable between different positions to image the surface from said different positions,

wherein said projecting is from a specified angle of at least 30°

relative to an axis of said camera;

receiving information from said camera, said information being indicative of imaged scenes at different times;

using information received through said apertures to determine a relationship, based on the specified angle of projection of the pattern, between different amounts of deformation of the projected pattern and a feature of said surface, to determine a pose of said camera, and to determine three dimensional information about said surface based on a degree of deformation of said optical pattern on said surface, and

combining together said three dimensional information from said pose with three dimensional information determined from another pose to generate a three dimensional image.

8. A method as in claim 7, wherein said projecting comprises projecting a grid of optical dots.

9. A method as in claim 7, wherein said projecting comprises projecting a random pattern.

10. A method as in claim 8, wherein said projecting comprises projecting laser dots.

11. A method as in claim 8, further comprising using defocused information from said optical pattern to determine a correspondence of the optical pattern to the grid and to determine an approximate Z axis correspondence based on said defocused information.

12. A method as in claim 11, further comprising averaging centroids of the defocused information.

15. A method as in claim 7, wherein said angle is at least 45°

.

13. A system, comprising:

a laser dot projector that projects a grid of laser dots toward a surface to be imaged;

a camera including a first plate with at least two off-axis apertures thereon, said camera arranged to obtain an image of the projected laser dots from said surface,

said projector projecting a beam at a specified angle of at least 30°

relative to an axis of said camera; and

a processor that is programmed to receive information from said camera, and to use defocused information from said image of said laser dots to determine a relationship, based on the specified angle of the beam, between different amounts of deformation of the projected laser dots and a feature of said surface, to determine a correspondence of said image of said laser dots to the grid, to determine an approximate Z axis correspondence based on said defocused information, and to determine three dimensional information about said surface based on a degree of deformation of said laser grid on said surface, including the determination of centroids of said defocused information from said image of said laser dots,

wherein said camera is movable between different positions to image said surface from said different positions, and said processor is programmed to use information received through said apertures to determine a pose of said camera, and is programmed to combine together said three dimensional information from said multiple poses of said camera, to determine an overall three dimensional information obtained from multiple different poses of said camera.

14. A system as in claim 13, wherein said angle is at least 45°

.