Triangulation scanner and camera for augmented reality

• US 9,506,744 B2
• Filed: 12/07/2015
• Issued: 11/29/2016
• Est. Priority Date: 12/16/2014
• Status: Expired due to Fees

• Current Assignee: Faro Technologies Incorporated
• Original Assignee: Faro Technologies Incorporated
• Inventors: Bridges, Robert E.
• Primary Examiner(s): Poon, King
• Assistant Examiner(s): Siddo, Ibrahim
• Application Number:
  US14/960,626
• Publication Number:
  US 20160171776A1
• Time in Patent Office: 358 Days
• US Class Current: 1/1
• CPC Class Codes: G01B 11/005 coordinate measuring machines
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 3/786 the desired condition being maintained automatically
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

A method of combining 2D images into a 3D image includes providing a coordinate measurement device and a triangulation scanner having an integral camera associated therewith, the scanner being separate from the coordinate measurement device. In a first instance, the coordinate measurement device determines the position and orientation of the scanner and the integral camera captures a first 2D image. In a second instance, the scanner is moved, the coordinate measurement device determines the position and orientation of the scanner, and the integral camera captures a second 2D image. A cardinal point common to the first and second images is found and is used, together with the first and second images and the positions and orientations of the scanner in the first and second instances, to create the 3D image.

Claims

1. A method of combining a plurality of two-dimensional (2D) images into a three-dimensional (3D) image, the method comprising steps of:

providing a six-degree of freedom (six-DOF) triangulation scanner, the triangulation scanner including a retroreflector and an integral augmented reality (AR) camera;

providing a coordinate measurement device having a device frame of reference the device being separate from the triangulation scanner, the device including a first motor, a second motor, a first angle measuring transducer, a second angle measuring transducer, a distance meter, a position detector, a control system, and a processor, the first motor and the second motor configured together to direct a first beam of light to a first direction, the first direction determined by a first angle of rotation about a first axis and a second angle of rotation about a second axis, the first angle of rotation produced by the first motor and the second angle of rotation produced by the second motor, the first angle measuring transducer configured to measure the first angle of rotation and the second angle measuring transducer configured to measure the second angle of rotation, the distance meter configured to measure a distance from the device to the retroreflector based at least in part on a first part of the first beam of light reflected by the retroreflector and received by a first optical detector and on a speed of light in air, the position detector configured to receive a second part of the first beam of light reflected by the retroreflector and to produce a first signal in response, the control system configured to send a second signal to the first motor and a third signal to the second motor, the second signal and the third signal based at least in part on the first signal, the control system configured to adjust the first direction of the first beam of light to a position in space of the retroreflector, the processor configured to determine, in the device frame of reference, 3D translational coordinates of the triangulation scanner;

in a first instance;

with the device, measuring a third angle with the first angle measuring transducer, measuring a fourth angle with the second angle measuring transducer, measuring with the distance meter a first distance;

forming a first 2D image with the camera;

in a second instance;

moving the triangulation scanner;

with the device, measuring a fifth angle with the first angle measuring transducer, measuring a sixth angle with the second angle measuring transducer, measuring with the distance meter a second distance;

forming a second 2D image with the camera;

determining a first cardinal point in common between the first and second 2D images, the first cardinal point having a first location on the first 2D image and a second location on the second 2D image;

determining 3D coordinates of the first cardinal point in a first frame of reference based at least in part on the third angle, the fourth angle, the fifth angle, the sixth angle, the first distance, the second distance, the first location, and the second location;

creating the 3D image as a first composite 3D image from the first 2D image and the second 2D image based at least in part on the first 2D image, the second 2D image, and the 3D coordinates of the first cardinal point in the first frame of reference; and

storing the first composite 3D image.

3. The method of claim 1, further including a step of measuring a first plurality of points on a surface of the object.

4. The method of claim 3, further including a step of forming a first graphical representation, the first graphical representation including the first composite 3D image and the first plurality of points on the surface of the object, the first plurality of points viewed from a first observer perspective relative to the first composite 3D image.

5. The method of claim 4, further including a step of forming a second graphical representation, the second graphical representation including the first composite 3D image and the first plurality of points on the surface of the object, the first plurality of points viewed form a second observer perspective relative to the first composite 3D image, the second observer perspective being different than the first observer perspective.

6. The method of claim 1, wherein:

in the step of forming a first 2D image with the camera, the first 2D image is a color image;

in the step of forming a second 2D image with the camera, the second 2D image is a color image; and

in the step of creating a first composite 3D image, the first composite 3D image is a color image.

7. The method of claim 1, wherein, in the step of forming a first 2D image with the camera, the first 2D image includes a first area of the object surface.

8. The method of claim 7, further including:

a step of providing a drawing; and

a step of superimposing the first composite 3D image onto the drawing.

9. The method of claim 1, further including:

providing a first graphical element; and

superimposing the first graphical element on the first composite 3D image.

10. The method of claim 9, wherein, in the step of providing a first graphical element, the first graphical element is a mechanical extension of the object.

11. The method of claim 9, wherein, in the step of providing a first graphical element, the first graphical element is placed within a volume represented by the first composite 3D image.

12. The method of claim 9, wherein, in the step of providing a first graphical element, the first graphical element is downloaded from a network connection.

13. The method of claim 7, further includes:

a step of forming a third 2D image with the camera, the third 2D image including a second area of the object surface, the second area including a region of the object not visible in the first composite 3D image; and

a step of forming a second composite 3D image, the second composite 3D image including the third 2D image and at least a portion of the first 2D image or a portion of the second 2D image.

2. The method of step 1, wherein the triangulation scanner comprises one of a laser line probe and a structured area scanner.