Six degree-of-freedom triangulation scanner and camera for augmented reality

• US 10,021,379 B2
• Filed: 08/07/2015
• Issued: 07/10/2018
• Est. Priority Date: 06/12/2014
• Status: Active Grant

• Current Assignee: Faro Technologies Incorporated
• Original Assignee: Faro Technologies Incorporated
• Inventors: Bridges, Robert E.
• Primary Examiner(s): Luo, Kate H
• Application Number:
  US14/820,665
• Publication Number:
  US 20150373321A1
• Time in Patent Office: 1,068 Days
• Field of Search: None
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G01B 5/004 for measuring coordinates of points
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G01S 17/48 Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4813 Housing arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/006 Pseudo-stereoscopic systems, i.e. systems wherein a stereoscopic effect is obtained without sending different images to the viewer's eyes

Abstract

A 3D coordinate measuring system includes a six-DOF unit having a unit frame of reference and including a structure, a retroreflector, a triangulation scanner, and an augmented reality (AR) color camera. The retroreflector, scanner and AR camera are attached to the structure. The scanner includes a first camera configured to form a first image of the pattern of light projected onto the object by a projector. The first camera and projector configured to cooperate to determine first 3D coordinates of a point on the object in the unit frame of reference, the determination based at least in part on the projected pattern of light and the first image. The system also includes a coordinate measuring device having a device frame of reference and configured to measure a pose of the retroreflector in the device frame of reference, the measured pose including measurements of six degrees-of-freedom of the retroreflector.

Claims

1. A three-dimensional (3D) coordinate measuring system including:

a six degree-of-freedom (six-DOF) unit having a unit frame of reference, the unit including a structure, a retroreflector, a triangulation scanner, and an augmented reality (AR) camera, the retroreflector, the triangulation scanner, and the AR camera attached to the structure, the triangulation scanner including a first camera and a projector, the projector configured to project a pattern of light onto an object, the first camera configured to form a first image of the pattern of light on the object, the projector and the first camera configured to cooperate to determine first 3D coordinates of a point on the object in the unit frame of reference, the determination based at least in part on the projected pattern of light and the first image, the AR camera being a color camera configured to obtain a color two-dimensional image in the unit frame of reference;

a laser tracker device having a device frame of reference, the device including a means to measure a pose of the retroreflector in the device frame of reference, wherein the measured pose includes measurements of six degrees-of-freedom of the retroreflector; and

one or more processors configured in a first instance to receive a first AR image from the AR camera and to receive from the device a first measured pose, the one or more processors further configured in a second instance to receive a second AR image from the AR camera and to receive from the device a second measured pose, the one or more processors further configured to determine a first cardinal point common to the first AR image and the second AR image, the one or more processors further configured to determine second 3D coordinates of the first cardinal point in the device frame of reference, wherein the one or more processors are further configured to combine the first 3D coordinates, the second 3D coordinates, the first AR image, and the second AR image in a common frame of reference into a composite 3D image based at least in part on the first measured pose of the retroreflector and the second measured pose of the retroreflector.

2. The 3D coordinate measuring system of claim 1, wherein the laser tracker device includes a light source and a distance meter, the light source configured to send a beam of light to the retroreflector and to receive a reflected beam of light in response, the distance meter configured to measure a distance to the retroreflector based at least in part on the reflected beam of light.

3. The 3D coordinate measuring system of claim 2, wherein the AR camera has a wider field of view than the first camera.

4. The 3D coordinate measuring system of claim 1, wherein the one or more processors are further configured to combine the composite 3D image with the first 3D coordinates of the point measured by the triangulation scanner.

5. A three-dimensional (3D) measurement method, comprising:

sending a first beam of first light from a coordinate measuring device to a retroreflector in a six degree-of-freedom (six-DOF) unit and receiving a reflected first beam of reflected first light in response, the coordinate measuring device having a device frame of reference, the six-DOF unit having a unit frame of reference, the six-DOF unit including a structure, the retroreflector, a triangulation scanner, and an AR camera, the retroreflector, the triangulation scanner, and the AR camera attached to the structure, the triangulation scanner including a first camera and a projector, the AR camera being a color camera configured to obtain a two-dimensional color image in the unit frame of reference;

determining by the coordinate measuring device a first pose of the retroreflector based at least in part on the reflected first beam, the determined first pose being measurements of six degrees-of-freedom of the retroreflector in the device frame of reference;

projecting from the projector a first pattern of light onto an object;

forming with the first camera a first image of a reflection of the first pattern of light from the object;

determining with one or more processors first 3D coordinates of a first point on the object in the device frame of reference based at least in part on the first pattern of light, the first image, and the determined first pose of the retroreflector;

capturing, in a first instance, with the AR camera a first color image in the unit frame of reference;

combining with the one or more processors the first 3D coordinates and the first color image in a common frame of reference based at least in part on the determined first pose of the retroreflector; and

sending a second beam of the first light from the coordinate measuring device to the retroreflector and receiving a second reflected beam in response;

determining by the coordinate measuring device a second pose of the retroreflector based at least in part on the reflected second beam;

projecting from the projector a second pattern of light onto the object;

forming with the first camera a second image of a reflection of the second pattern of light from the object;

determining with the one or more processors second 3D coordinates of a second point on the object in the device frame of reference based at least in part on the second pattern of light, the second image, and the determined second pose of the retroreflector;

capturing, in a second instance, with the AR camera a second color image in the unit frame of reference; and

combining with the one or more processors the first 3D coordinates, the second 3D coordinates, the first color image, and the second color image in a common frame of reference into a composite 3D image based at least in part on the determined first pose of the retroreflector and the determined second pose of the retroreflector.