Accurate 3D Object Reconstruction Using a Handheld Device with a Projected Light Pattern

• US 20110074932A1
• Filed: 08/27/2010
• Published: 03/31/2011
• Est. Priority Date: 08/27/2009
• Status: Active Grant

• Current Assignee: California Institute of Technology
• Original Assignee: California Institute of Technology
• Inventors: Lu, Jian, Harvard, Alexei, Gharib, Morteza, Hsieh, Scott
• Granted Patent: US 8,773,514 B2
• Time in Patent Office: Days
• US Class Current: 348/50
• CPC Class Codes: G01B 11/16 for measuring the deformation in a solid, e.g. optical strain gauge
• CPC Class Codes: G01B 11/167 by projecting a pattern on the object
• CPC Class Codes: G01B 11/22 for measuring depth
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01B 11/2545 with one projection direction and several detection directions, e.g. stereo
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

A lens and aperture device for determining 3D information. A projector projects an optical pattern toward a surface. The camera has at least two off-axis apertures thereon, arranged to obtain an image of the projected pattern including defocused information. The camera is movable between different positions to image the surface from said different positions, and the projector is at a specified angle of at least 5° relative to said camera. A processor carries out a first operation using information received through the apertures to determine a pose of said camera, and to determine three dimensional information about the object based on a degree of deformation of said optical pattern on said surface indicative of a three dimensional surface. An embodiment projects a grid of laser dots and uses laser-dot defocusing for approximate Z and thus grid correspondence, which can greatly increase the working depth of the system.

Claims

7. A method, comprising:

projecting an optical pattern toward a surface;

coupling an image of said pattern via a first plate with at least two off-axis apertures thereon;

receiving an which has passed through said apertures in a camera, said camera being movable between different positions to image the surface from said different positions,

wherein said projecting is from a specified angle of at least 5°

relative to an axis of said camera;

receiving information from said camera, said information being indicative of imaged scenes at different times;

carrying out a first operation using information received through said apertures to determine a pose of said camera, and to determine three dimensional information about said surface based on a degree of deformation of said optical pattern on said surface indicative of a three dimensional surface, said pose and said three dimensional information being obtained from multiple different poses of said camera,

and combining together said three dimensional information from said multiple poses using each of a plurality of poses to combine 3D information obtained from multiple different poses of the camera.

1. A system, comprising:

a projector, that projects an optical pattern toward a surface;

a camera, including a first plate with at least two off-axis apertures thereon, arranged to obtain an image of the optical pattern that has been projected, said camera being movable between different positions to image the surface from said different positions,

said projector being at a specified angle of at least 5°

relative to said camera; and

a processor, receiving information from said camera, said information indicative of imaged scenes at different times, and said processor carrying out a first operation using information received through said apertures to determine a pose of said camera, and to determine three dimensional information about said surface based on a degree of deformation of said optical pattern on said surface indicative of a three dimensional surface, said pose and said three dimensional information being obtained from multiple different poses of said camera, and combining together said three dimensional information from said multiple poses using each of a plurality of poses to combine 3D information obtained from multiple different poses of the camera.

2. A system as in claim 1, wherein said projector projects a grid of laser dots.

3. A system as in claim 1, wherein said projector projects a random pattern of dots.

4. A system as in claim 1, wherein said projector projects a grid of optical dots, and said processor uses defocused information from said optical dots to determines a correspondence of the optical dots to the grid and to determine an approximate Z axis correspondence based on said defocused information.

5. A system as in claim 3, wherein said processor averages centroids of images within the defocused information.

6. A system as in claim 1, wherein said projector is at a specified angle of at least 30°

relative to said camera.

8. A method as in claim 7, wherein said projecting comprises projecting a grid of optical dots.

9. A method as in claim 7, wherein said projecting comprises projecting a random pattern.

10. A method as in claim 8, wherein said projecting comprises projecting laser dots.

11. A method as in claim 8, wherein said first operation comprises using defocused information from said optical pattern to determine a correspondence of the optical pattern dots to the grid and to determine an approximate Z axis correspondence based on said defocused information.

12. A method as in claim 11, wherein said first operation further comprises averaging centroids of the defocused images.

12-1. A method as in claim 7, wherein said projector being at a specified angle of at least 45°

relative to said camera.

13. A system, comprising:

a laser dot projector, that projects a grid of laser dots toward a surface to be imaged;

a camera, including a first plate with at least two off-axis apertures thereon, arranged to obtain an image of the projected pattern,

said projector projecting its beam at a specified angle of at least 5°

relative to an axis of said camera; and

a processor, receiving information from said camera, and using defocused information from said laser dots to determines a correspondence of the optical dots to the grid and to determine an approximate Z axis correspondence based on said defocusing information,

and to determine three dimensional information about said object based on a degree of deformation of said laser grid on said surface, including determining centroids of said defocused information from said laser dots.

14. A system as in claim 13, wherein said camera is movable between different positions to image the surface from said different positions, and said processor carrying out a first operation using information received through said apertures to determine a pose of said camera, and combining together said three dimensional information from said multiple poses of the camera, to determine an overall three dimensional information being obtained from multiple different poses of said camera.

15. A system as in claim 13, wherein said projector is at a specified angle of at least 30°

relative to said camera.