Triangulation scanner and camera for augmented reality

• US 10,244,222 B2
• Filed: 12/04/2017
• Issued: 03/26/2019
• Est. Priority Date: 12/16/2014
• Status: Active Grant

• Current Assignee: Faro Technologies Incorporated
• Original Assignee: Faro Technologies Incorporated
• Inventors: Bridges, Robert E.
• Primary Examiner(s): Siddo, Ibrahim
• Application Number:
  US15/830,494
• Publication Number:
  US 20180091792A1
• Time in Patent Office: 477 Days
• CPC Class Codes: G01B 11/005 coordinate measuring machines
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 3/786 the desired condition being maintained automatically
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

A method and system of combining 2D images into a 3D image. The method includes providing a coordinate measurement device and a triangulation scanner having an integral camera associated therewith, the scanner being separate from the coordinate measurement device. In a first instance, the coordinate measurement device determines the position and orientation of the scanner and the integral camera captures a first 2D image. In a second instance, the scanner is moved, the coordinate measurement device determines the position and orientation of the scanner, and the integral camera captures a second 2D image. A common feature point in the first and second images is found and is used, together with the first and second images and the positions and orientations of the scanner in the first and second instances, to create the 3D image.

Claims

1. A method of combining a plurality of two-dimensional (2D) images into a three-dimensional (3D) image of an object, the method comprising steps of:

providing a six-degree of freedom triangulation scanner having a retroreflector and a camera;

providing a laser tracker device having a distance meter, the device operable to direct a beam of light in a direction and measure a distance from the device to the retroreflector, the device further configured to determine 3D coordinates of the retroreflector;

determining with the device a first 3D coordinates of the retroreflector at a first position;

forming a first 2D image with the camera at the first position;

determining with the device a second 3D coordinates of the retroreflector at a second position;

forming a second 2D image with the camera at the second position;

determining a first common feature point in the first 2D image and the second 2D image;

determining 3D coordinates of the first common feature point based at least in part on the first 3D coordinates and the second 3D coordinates;

generating a first composite 3D image from the first 2D image and the second 2D image based at least in part on the 3D coordinates of the first common feature point; and

storing in a memory the first composite 3D image.

2. The method of claim 1, wherein the second 2D image at least partially overlapping the first 2D image, and the first common feature point is located in an overlapping portion.

3. The method of claim 1, wherein the determining of the first common feature point is performed using interest point detection.

4. The method of claim 3, wherein the first common feature point is a corner point.

5. The method of claim 1, wherein the determining of the common feature point is performed using one of edge detection, blob detection or ridge detection.

6. The method of claim 1, further comprising:

determining a plurality of common feature points;

wherein the first 2D image, the second 2D image, and the first composite 3D image are color images.

7. The method of claim 6, further comprising obtaining texture and color information from visible regions between the plurality of common feature points.

8. The method of claim 7, wherein the obtaining of texture and color information is performed using interpolation.

9. A method of combining a plurality of two-dimensional (2D) images into a three-dimensional (3D) image of an object, the method comprising:

providing a six-degree of freedom triangulation scanner having a retroreflector and a camera;

providing a laser tracker device having a distance meter, the device operable to direct a beam of light in a direction and measure a distance from the device to the retroreflector, the device further configured to determine 3D coordinates of the retroreflector;

determining with the device a first 3D coordinates of the retroreflector at a first position;

forming a first 2D image with the camera at the first position;

determining with the device a second 3D coordinates of the retroreflector at a second position;

forming a second 2D image with the camera at the second position;

determining a first common feature point in the first 2D image and the second 2D image;

determining 3D coordinates of the first common feature point based at least in part on the first 3D coordinates and the second 3D coordinates;

generating a first composite 3D image from the first 2D image and the second 2D image based at least in part on the 3D coordinates of the first common feature point

storing in a memory the first composite 3D image; and

determining a plurality of 3D coordinates of the object with the triangulation scanner; and

determining a scale of the composite 3D image based at least partially on the plurality of 3D coordinates.

10. A system for combining a plurality of two-dimensional (2D) images into a three-dimensional (3D) image of an object, the system comprising:

a six-degree of freedom triangulation scanner having a retroreflector and a camera;

a laser tracker device having a distance meter, the device operable to direct a beam of light to a direction and measure a distance from the device to the retroreflector, the device further operable to determine a 3D coordinates of the triangulation scanner; and

one or more processors responsive to nontransitory executable computer instructions, the one or more processors electrically coupled to the triangulation scanner and the device, the nontransitory executable computer instructions comprising;

determining with the device a first 3D coordinates of the retroreflector at a first position;

forming a first 2D image with the camera at the first position;

determining with the device a second 3D coordinates of the triangulation scanner at a second position;

forming a second 2D image with the camera at the second position;

determining a first common feature point in the first 2D image and the second 2D image;

determining 3D coordinates of the first common feature point based at least in part on the first set of 3D coordinates and the second set of 3D coordinates;

creating a first composite 3D image from the first 2D image and the second 2D image based at least in part on the 3D coordinates of the first common feature point in the first frame of reference; and

storing in a memory the first composite 3D image.

11. The system of claim 10, wherein the second 2D image at least partially overlapping the first 2D image, and the first common feature point is located in an overlapping portion.

12. The system of claim 11, wherein the second 2D image at least partially overlapping the first 2D image, and the first common feature point is located in an overlapping portion.

13. The system of claim 10, wherein the determining of the first common feature point is performed using interest point detection.

14. The system of claim 13, wherein the first common feature point is a corner point.

15. The system of claim 10, wherein the determining of the common feature point is performed using one of edge detection, blob detection or ridge detection.

16. The system of claim 10, wherein the nontransitory executable computer instructions further comprise:

determining a plurality of common feature points; and

wherein the first 2D image, the second 2D image and the first composite 3D image are color images.

17. The system of claim 16, wherein the nontransitory executable computer instructions further comprise obtaining texture and color information from visible regions between the plurality of common feature points.

18. The system of claim 17, wherein the obtaining of texture and color information is performed using interpolation.

19. The system of claim 10, wherein the nontransitory executable computer instructions further comprise:

determining a plurality of 3D coordinates of the object with the triangulation scanner; and

determining a scale of the composite 3D image based at least partially on the plurality of 3D coordinates.