3-D scanning and positioning system

• US 9,816,809 B2
• Filed: 06/26/2013
• Issued: 11/14/2017
• Est. Priority Date: 07/04/2012
• Status: Active Grant

• Current Assignee: Creaform, Inc. (Ametek Incorporated)
• Original Assignee: Creaform, Inc. (Ametek Incorporated)
• Inventors: Hebert, Patrick, Rochette, Flix
• Primary Examiner(s): Vo, Tung
• Application Number:
  US14/405,428
• Publication Number:
  US 20150138349A1
• Time in Patent Office: 1,602 Days
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: H04N 23/56 provided with illuminating means

Abstract

A system for obtaining three-dimensional information about a surface, the system comprising: a sensing device having: a pattern projector for providing a projected pattern on the surface; and a camera for acquiring a 2D image of the surface from a viewpoint, wherein at least one target of a set of reference targets and at least a portion of the projected pattern is apparent on the 2D image; a storage for calibration data; an image processor for extracting 2D image coordinates of surface points of the projected pattern and target contours from the 2D image; a 3D surface point calculator for calculating 3D coordinates of the surface points from the 2D image coordinates of the surface points; a 3D target calculator for calculating a 3D position and/or an actual orientation for the reference target.

Claims

1. A system for obtaining three-dimensional information about a surface having a set of at least one reference target, each reference target being provided at a fixed position one of on an environment of said surface and on said surface, said system comprising:

a sensing device having;

a pattern projector for providing a projected pattern on said surface;

a camera for acquiring a 2D image of said surface from a viewpoint, said camera cooperating with said pattern projector, wherein at least one target of said set of reference targets and at least a portion of said projected pattern is apparent on said 2D image;

a storage for calibration data, said calibration data including extrinsic and intrinsic parameters of said camera and said pattern projector;

an image processor for extracting 2D image coordinates of surface points of the projected pattern and target contours from said 2D image;

a 3D surface point calculator for calculating 3D coordinates of the surface points from said 2D image coordinates of said surface points using said calibration data;

a 3D target calculator for calculating, from said 2D image coordinates of said target contours, using said 3D coordinates of the surface points and said calibration data, at least one of a 3D position and an actual orientation for said reference target;

said system being adapted to perform the steps of;

identifying neighboring surface points located around said target;

calculating 3D coordinates of the neighboring surface points;

fitting a surface section model on said 3D coordinates of said neighboring surface points; and

at least one of;

refining an estimated 3D position into the 3D position; and

identifying the actual orientation of said target.

2. The system as claimed in claim 1, wherein said reference target is a retro-reflective target, wherein said sensing device further comprises a light source for illuminating said surface, said camera cooperating with said light source.

3. The system as claimed in claim 2, wherein said light source is a ring light source surrounding said camera.

4. The system as claimed in claim 3, wherein said ring light source has at least two Light Emitting Diodes (LED).

5. The system as claimed in claim 1, wherein said pattern projector is one of a laser pattern projector and a wideband pattern projector.

6. The system as claimed in claim 1, wherein said pattern projector and said camera are synchronized to simultaneously provide said projected pattern and acquire said 2D image by said camera.

7. The system as claimed in claim 1, further comprising:

a target model storage for storing a model of targets of said surface;

a target matcher for matching said target with a same target in said model of targets and generating a rigid transformation mapping a coordinate system of the sensing device to a coordinate system of said target model.

8. The system as claimed in claim 7, further comprising:

a target transformer for transforming said 3D position for said reference target into a transformed 3D position for said reference target using said rigid transformation;

a target model builder for updating said model of targets in said target model storage using said transformed 3D position for said reference target.

9. The system as claimed in claim 7, further comprising:

a surface point transformer for transforming said 3D coordinates of said surface points into transformed 3D coordinates of said surface points using said rigid transformation;

a surface reconstructor for producing a surface model of said surface using said transformed 3D coordinates of said surface points.

10. The system as claimed in claim 9, further comprising:

a user interface display for displaying at least one of said surface model of said surface and said model of targets.

11. The system as claimed in claim 1, wherein said sensing device is adapted to be displaced to a plurality of distinct viewpoints and wherein said camera acquires said 2D image at each viewpoint of said plurality of viewpoints and wherein said 3D surface point calculator and said 3D target calculator use 2D image coordinates from at least two of said 2D images.

12. The system as claimed in claim 1, wherein said camera acquires at least two of said 2D image at said viewpoint and wherein said 3D surface point calculator and said 3D target calculator use 2D image coordinates from at least two of said 2D images.

13. The system as claimed in claim 1, wherein said sensing device is adapted to be displaced to a plurality of distinct viewpoints and wherein said camera acquires at least two of said 2D image at each viewpoint of said plurality of viewpoints and wherein said 3D surface point calculator and said 3D target calculator use 2D image coordinates from at least two of said 2D image.

14. The system as claimed in claim 1, wherein said camera is a first camera and wherein said 2D image is a first 2D image;

and wherein said sensing device further comprises a second camera for acquiring a second 2D image of said surface, said second camera cooperating with said first camera, a spatial relationship between said first camera and said second camera being known, wherein said calibration data includes extrinsic and intrinsic parameters of said second camera.

15. The system as claimed in claim 14, wherein said sensing device further comprises a synchronizing unit in communication with said first camera, said second camera and said pattern projector, said synchronizing unit controlling said first camera and said pattern projector to capture said first 2D image during a first time interval and controlling said second camera and said pattern projector to capture said second 2D image during a second time interval;

wherein said first time interval is distinct from said second time interval.

16. The system as claimed in claim 15, wherein said synchronizing unit controls said pattern projector to provide said projected pattern during said first time interval and prevents said pattern projector from providing said projected pattern during said second time interval.

17. The system as claimed in claim 14, wherein said second camera is a texture camera, said second 2D image being a 2D texture image of said surface, said system further comprising a texture extractor for extracting a texture of said surface at said 2D image coordinates of surface points to obtain texture information for said surface point.

20. The system as claimed in claim 1, wherein said 3D target calculator is adapted to perform the step of refining the estimated 3D position using a target thickness.

18. A method for extracting a 3D position for a target affixed on a surface in a sensor coordinate system, a pattern being projected on said surface, comprising:

retrieving target information, said target information including a target size;

retrieving calibration data, said calibration data including extrinsic and intrinsic parameters;

providing a 2D image of said surface, wherein said target and at least a portion of said projected pattern being apparent on said 2D image;

extracting 2D image coordinates of target contours of said target from said 2D image;

calculating at least one possible orientation for said target using said 2D image coordinates of target contours and said calibration data;

estimating an estimated 3D position for said target in said sensor coordinate system using said 2D image coordinates of said target contours, said calibration data and said target size;

extracting 2D image coordinates of surface points of the projected pattern from said 2D image;

identifying neighboring surface points located around said target using said 2D image coordinates of said target contours;

calculating 3D coordinates of the neighboring surface points in said sensor coordinate system from said 2D image coordinates of said surface points using said calibration data;

fitting a surface section model on said 3D coordinates of said neighboring surface points in said sensor coordinate system;

using said fitted surface section model and said at least one possible orientation, at least one of refining said estimated 3D position into a refined 3D position in said sensor coordinate system and identifying an actual orientation for said target.

19. The method as claimed in claim 18, wherein

said target information includes a target thickness;

said refining said estimated 3D position includes using said target thickness for said refining.