Coordinate measuring device with a six degree-of-freedom handheld probe and integrated camera for augmented reality

• US 10,089,789 B2
• Filed: 07/20/2016
• Issued: 10/02/2018
• Est. Priority Date: 06/12/2014
• Status: Active Grant

• Current Assignee: Faro Technologies Incorporated
• Original Assignee: Faro Technologies Incorporated
• Inventors: Tohme, Yazid, Bridges, Robert E.
• Primary Examiner(s): Luo, Kate
• Application Number:
  US15/215,031
• Publication Number:
  US 20160335803A1
• Time in Patent Office: 804 Days
• Field of Search: None
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01S 17/66 Tracking systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 7/4813 Housing arrangements
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

A method of combining 2D images into a 3D image includes providing a coordinate measurement device and a six-DOF probe having an integral camera associated therewith, the six-DOF probe being separate from the coordinate measurement device. In a first instance, the coordinate measurement device determines the position and orientation of the six-DOF probe and the integral camera captures a first 2D image. In a second instance, the six-DOF probe is moved, the coordinate measurement device determines the position and orientation of the six-DOF probe, and the integral camera captures a second 2D image. A cardinal point common to the first and second image is found and is used, together with the first and second images and the positions and orientations of the six-DOF probe in the first and second instances, to create the 3D image.

Claims

1. A system comprising:

a six-degree of freedom (six-DOF) probe assembly, the six-DOF probe assembly including a probe head having a retroreflector and an integral camera;

a coordinate measurement device having a device frame of reference, the coordinate measurement device including a distance meter, an orientation sensor, and a control system electrically coupled to a processor, the coordinate measurement device being operable to direct a first beam of light to a first direction, the first direction determined at least by a first angle of rotation about a first axis and a second angle of rotation about a second axis;

wherein the processor is operable to execute non-transient computer readable instructions, the computer readable instructions comprising;

in a first instance;

measuring a first distance to the retroreflector with the distance meter and a first set of three orientational degrees of freedom with the orientation sensor;

forming a first 2D image of the object with the camera;

in a second instance;

in response to the six-DOF probe assembly being moved, measuring a second distance to the retroreflector with the distance meter and a second set of three orientational degrees of freedom with the orientation sensor;

forming a second 2D image of the object with the camera;

determining a first cardinal point in common between the first 2D image and the second 2D image, the first cardinal point being positioned a first location on the first 2D image and a second location on the second 2D image;

determining 3D coordinates of the first cardinal point in a first frame of reference;

creating a first composite 3D image from the first 2D image and the second 2D image based at least in part on the first 2D image, the second 2D image, and the 3D coordinates of the first cardinal point in the first frame of reference; and

storing the first composite 3D image.

2. The system of claim 1, wherein the six-DOF probe assembly further includes a probe stylus having a spherical tip portion.

3. The system of claim 2, wherein the computer readable instructions further comprising, in the first instance, identifying a first point on a surface in response the tip portion touching the surface, the first point being visible in the first 2D image and the second 2D image.

4. The system of claim 3, wherein the computer readable instructions further comprising, in the first instance, measuring with the coordinate measurement device a first 3D coordinates of the first point.

5. The system of claim 4, wherein the computer readable instructions further comprising, in the second instance, identifying a second point on the surface in response to touching the tip portion to the surface, the second point being visible in the first 2D image and the second 2D image.

6. The system of claim 5, wherein the computer readable instructions further comprising, in the second instance, measuring with the coordinate measurement device a second 3D coordinates of the second point.

7. The system of claim 6, wherein the computer readable instructions further comprises changing a scale of the first composite 3D image based at least in part on the first 2D image, the second 2D image, the first 3D coordinates and the second 3D coordinates.

8. The system of claim 2, wherein the computer readable instructions further comprise:

measuring a first plurality of points on an object with the coordinate measurement device in response to the contacting of the tip portion to the object; and

forming a first graphical representation from a first observer perspective that includes the first composite 3D image and the first plurality of points.

9. The system of claim 8, further comprising forming a second graphical representation from a second observer perspective that includes the first composite 3D image and the first plurality of point, the second observer perspective being different that the first observer perspective.

10. A method comprising:

providing a six-degree of freedom (six-DOF) probe assembly, the six-DOF probe assembly including a probe head having a retroreflector and an integral camera;

providing a coordinate measurement device having a device frame of reference, the coordinate measurement device including a distance meter, an orientation sensor, and a control system electrically coupled to a processor, the coordinate measurement device being operable to direct a first beam of light to a first direction, the first direction determined at least by a first angle of rotation about a first axis and a second angle of rotation about a second axis;

in a first instance;

measuring a first distance to the retroreflector with the distance meter and a first set of three orientational degrees of freedom with the orientation sensor;

forming a first 2D image of the object with the camera;

in a second instance;

moving the six-DOF probe assembly;

measuring a second distance to the retroreflector with the distance meter and a second set of three orientational degrees of freedom with the orientation sensor;

forming a second 2D image of the object with the camera;

determining a first cardinal point in common between the first 2D image and the second 2D image, the first cardinal point being positioned a first location on the first 2D image and a second location on the second 2D image;

determining 3D coordinates of the first cardinal point in a first frame of reference;

creating a first composite 3D image from the first 2D image and the second 2D image based at least in part on the first 2D image, the second 2D image, and the 3D coordinates of the first cardinal point in the first frame of reference; and

storing the first composite 3D image.

11. The method of claim 10, wherein the six-DOF probe assembly further includes a probe stylus having a spherical tip portion.

12. The method of claim 11, wherein:

in the first instance further includes touching the tip portion to a first point on a surface; and

the first point being visible in the first 2D image and the second 2D image.

13. The method of claim 12, further comprising in the first instance measuring with the coordinate measurement device a first 3D coordinates of the first point.

14. The method of claim 13, wherein:

in the first instance further includes touching the tip portion to a second point on the surface; and

the second point being visible in the first 2D image and the second 2D image.

15. The method of claim 14, further comprising in the second instance measuring with the coordinate measurement device a second 3D coordinates of the second point.

16. The method of claim 15, further comprising changing a scale of the first composite 3D image based at least in part on the first 2D image, the second 2D image, the first 3D coordinates and the second 3D coordinates.

17. The method of claim 11, further comprising:

measuring a first plurality of points on an object with the coordinate measurement device by contacting the tip portion to the object; and

forming a first graphical representation from a first observer perspective that includes the first composite 3D image and the first plurality of points.

18. The method of claim 17, further comprising forming a second graphical representation from a second observer perspective that includes the first composite 3D image and the first plurality of point, the second observer perspective being different that the first observer perspective.

19. The method of claim 17, further comprising:

providing a drawing; and

superimposing the first composite 3D image onto the drawing.

20. The method of claim 17 further comprising:

providing a first graphical element, the first graphical element being a mechanical extension of the object; and

superimposing the first graphical element on the first composite 3D image.