System and method of scanning an environment and generating two dimensional images of the environment

• US 10,824,773 B2
• Filed: 09/25/2017
• Issued: 11/03/2020
• Est. Priority Date: 03/28/2017
• Status: Active Grant

• Current Assignee: Faro Technologies Incorporated
• Original Assignee: Faro Technologies Incorporated
• Inventors: Santos, Joao, Frank, Aleksej, Ramadneh, Ahmad, Raab, Simon, Zweigle, Oliver, White, Derik J.
• Primary Examiner(s): Johnson, Cedric
• Application Number:
  US15/713,931
• Publication Number:
  US 20180285482A1
• Time in Patent Office: 1,135 Days
• Field of Search: None
• CPC Class Codes: G01B 11/2518 Projection by scanning of the object
• CPC Class Codes: G01C 15/008 combined with inclination sensor
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09B 25/04 of buildings

Abstract

A system and method of generating a two-dimensional (2D) image of an environment is provided. The system includes a housing having a body and a handle. A 2D scanner is disposed in the body and has a light source, an image sensor and a controller, the light source steers a beam of light within a first plane to illuminate object points in the environment. The image sensor is arranged to receive light reflected from the object points and the controller determines a distance value to at least one of the object points. An inertial measurement unit is provided having a 3D accelerometer and a 3D gyroscope. One or more processors are responsive to executable instructions for generating a 2D image of the environment in response to an activation signal from an operator and based at least in part on the distance values and the signal.

Claims

1. A system of generating a two-dimensional (2D) image of an environment, the system comprising:

a housing having a body and a handle, the housing being sized to be carried by a single person during operation, the body having a first plane extending therethrough;

a 2D scanner disposed in the body and having a light source, an image sensor, a first position indicator capable of engaging a second position indicator, and a controller, the light source steers a beam of light within the first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points, the controller being operable to determine a distance value to at least one of the object points;

an inertial measurement unit disposed in the housing and having a three-dimensional accelerometer and a three-dimension gyroscope, the inertial measurement unit generating a signal in response a change in position or orientation of the housing; and

one or more processors operably coupled to the inertial measurement unit and the 2D scanner, the one or more processors being responsive to executable instructions for generating a 2D image of the environment in response to an activation signal from an operator and based at least in part on the distance value and the signal;

wherein the body includes a planar bottom wall, the one or more processors being disposed adjacent the bottom wall,

a plurality of intake vents extending through the bottom wall, each of the plurality of intake vents having a first opening extending through a bottom surface of the body adjacent the 2D scanner and a second opening adjacent the one or more processors, and

a plurality of exhaust vents extending through a side of the body on an opposite side of the one or more processors from the second opening.

3. The system of claim 1, further comprising a laser projector coupled to the body, the laser projector having a second light source that emits in operation a visible beam of light in a second plane parallel to the first plane.

4. The system of claim 1, further comprising a portable computing device having a second image sensor, the portable computing device being coupled for communication to the one or more processors, wherein the one or more processors are responsive to merge a first image from the portable computing device into the 2D image of the environment in response to the image being acquired by the second image sensor, a first position of the first image on the 2D image corresponding a second position in the environment where the first image is acquired.

5. The system of claim 1, further comprising a three-dimensional camera disposed in the body, the three-dimensional camera being operably coupled to the one or more processors.

2. A system of generating a two-dimensional (2D) image of an environment, the system comprising:

a housing having a body and a handle, the housing being sized to be carried by a single person during operation, the body having a first plane extending therethrough;

a 2D scanner disposed in the body and having a light source, an image sensor, a first position indicator capable of engaging a second position indicator, and a controller, the light source steers a beam of light within the first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points, the controller being operable to determine a distance value to at least one of the object points;

an inertial measurement unit disposed in the housing and having a three-dimensional accelerometer and a three-dimension gyroscope, the inertial measurement unit generating a signal in response a change in position or orientation of the housing;

one or more processors operably coupled to the inertial measurement unit and the 2D scanner, the one or more processors being responsive to executable instructions for generating a 2D image of the environment in response to an activation signal from an operator and based at least in part on the distance value and the signal; and

a haptic feedback device disposed within the handle and operably coupled to the one or more processors, wherein the one or more processors are responsive for activating the haptic feedback device in response to the signal indicating the orientation has crossed a variance threshold.

6. A method for generating a two-dimensional (2D) image of an environment, the method comprising:

moving a measurement device to a plurality of registration positions in the environment, the measurement device having a 2D scanner, a first position indicator having a near field communications tag, an inertial measurement unit and is sized and weighted to be carried by a single operator, the 2D scanner being configured to sweep a beam of light in a horizontal plane, the inertial measurement unit being configured to determine movement and orientation of the measurement device, the plurality of registration positions including a first registration position and a second registration position;

emitting a visible laser light beam from a laser projector onto a point on an object, the laser projector being coupled to the measurement device, the measurement device further having one or more processors;

obtaining by the 2D scanner a first plurality of 2D scan sets while the measurement device moves from the first registration position to the second registration position, each of the first plurality of 2D scan sets being a set of 2D coordinates of points in the environment, each of the first plurality of 2D scan sets being collected by the 2D scanner at a different position relative to the first registration position;

determining for the measurement device a first translation value corresponding to a first translation direction, a second translation value corresponding to a second translation direction, a first rotation value corresponding to a first orientational axis, and a second rotation value corresponding to a second orientational axis, wherein the first translation value, the second translation value, the first rotation value and the second rotation value are determined based at least in part on measurements of movement and orientation by the inertial measurement unit;

generating a 2D image of the environment based at least in part on the plurality of 2D scans, the first translation value, the first translation direction, the second translation value, the second translation direction, the first rotation value, and the second rotation value;

engaging the first position indicator with a second position indicator to register, determine, and record the position of the 2D scanner;

measuring a distance from the measurement device to the point on the object using a three-dimensional camera coupled to the measurement device; and

annotating the 2D image, with the one or more processors, to include a position of the point on the object based at least in part on the distance.

7. The method of claim 6, further comprising:

coupling for communication a portable computing device to the measurement device, the portable computing device having an image sensor;

acquiring a first image with the image sensor;

transmitting the first image to the measurement device; and

annotating the 2D image, with the one or more processors, to include the first image, wherein a position of the first image in the 2D image is based at least in part on a device position of the measurement device when the first image is acquired by the image sensor.

8. The method of claim 7, wherein the object includes a target member and the visible laser light beam is emitted onto a target.

9. The method of claim 8, further comprising:

annotating the 2D image, with the one or more processors, with a location of the target;

generating a three-dimensional point cloud of at least a portion of the environment, the three-dimensional point cloud including a scan of the target;

aligning the target in the three-dimensional point cloud with the position of the target in the 2D image; and

merging the three-dimensional point cloud into the 2D image.

10. The method of claim 6, further comprising:

scanning an area about the point on the object with the three-dimensional camera;

determining a scanning position and orientation of the measurement device when the area is being scanned;

generating a three-dimensional point cloud of the area; and

annotating the 2D image to include a linkage to the three-dimensional point cloud of the area, the annotation being positioned in the 2D image in a position corresponding to the position of the object in the area.

11. The method of claim 7, wherein the annotating of the 2D image further includes an annotation selected from a group comprising photos, videos, sounds, text, and hyperlinks.

12. The method of claim 11, wherein the annotation is acquired by the portable computing device.

13. The method of claim 7, wherein the first image includes a spot of light on the object from the visible laser light beam.

14. The method of claim 6, wherein the generating the 2D image includes generating a grid map that includes a plurality of cells, each of the plurality of cells indicating a probability of a cell area being occupied.

15. A method for generating a two-dimensional (2D) image of an environment, the method comprising

moving a measurement device to a plurality of registration positions in the environment, the measurement device having a 2D scanner, an inertial measurement unit and is sized and weighted to be carried by a single operator, the 2D scanner being configured to sweep a beam of light in a horizontal plane, the inertial measurement unit being configured to determine movement and orientation of the measurement device, the plurality of registration positions including a first registration position and a second registration position;

obtaining by the 2D scanner a first plurality of 2D scan sets while the measurement device moves from the first registration position to the second registration position, each of the first plurality of 2D scan sets being a set of 2D coordinates of points in the environment, each of the first plurality of 2D scan sets being collected by the 2D scanner at a different position relative to the first registration position;

determining for the measurement device a first translation value corresponding to a first translation direction, a second translation value corresponding to a second translation direction, a first rotation value corresponding to a first orientational axis, and a second rotation value corresponding to a second orientational axis, wherein the first translation value, the second translation value, the first rotation value and the second rotation value are determined based at least in part on measurements of movement and orientation by the inertial measurement unit;

generating a 2D image of the environment based at least in part on the plurality of 2D scans, the first translation value, the first translation direction, the second translation value, the second translation direction, the first rotation value, and the second rotation value; and

actuating a haptic feedback device in the measurement device in response to the second rotation value crossing a predetermined threshold.

16. A system of generating a two-dimensional (2D) image of an environment, the system comprising:

one or more processors;

a 2D scanner sized and weighted to be carried by a single person, having a first light source, an image sensor, an inertial measurement unit and a first position indicator, the first light source steers a beam of light within a first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points;

a 3D scanner having a second light source, a beam steering unit, a light receiver, and a second position indicator, the 3D scanner being operable to cooperate with the one or more processors to determine 3D coordinates of a first object point based on emitting a light with the second light source and receiving a reflected light with the light receiver, the 3D scanner being movable from a first position to a second position;

wherein the one or more processors are responsive to executable instructions which when executed by the one or more processors to;

cause the 3D scanner at the first position to determine 3D coordinates of a first collection of points on an object surface;

cause the 3D scanner at the second position to determine 3D coordinates of a second collection of points on the object surface;

identify a correspondence between registration targets in the first collection of points and the second collection of points;

transfer 3D coordinate data in a 2D scanner coordinate frame of reference from the 2D scanner to the 3D scanner in response to the first position indicator engaging the second position indicator; and

transforming the 3D coordinate data from the 2D scanner coordinate frame of reference to a 3D scanner coordinate frame of reference.

17. The system of claim 16, wherein:

the first position indicator includes a near field communications tag; and

the second position indicator includes a near field communications transmitter and a near field communications receiver.

18. The system of claim 16, wherein:

the first position indicator includes a magnetically activated switch; and

the second position indicator includes a magnet.

19. The system of claim 16, wherein:

the first position indicator includes a first feature on a body of the 2D scanner;

the second position indicator includes a second feature on a housing of the 3D scanner; and

the one or more processors are further responsive to transferring the 3D coordinate data in response to an operator actuating an actuator on one of the 2D scanner or the 3D scanner.

20. The system of claim 16, wherein:

the first position indicator includes a camera operably coupled to the one or more processors;

the second position indicator includes a machine readable indicia on a housing of the 3D scanner; and

the engaging of the first position indicator and the second position indicator includes acquiring an image of the machine readable indicia.

21. The system of claim 20, wherein the machine readable indicia is a QR-code label.

22. The system of claim 20, wherein the machine readable indicia is a bar-code.

23. A method of generating a two-dimensional (2D) image of an environment, the method comprising:

performing a first scan of the environment with a 3D scanner and determining 3D coordinates of first collection of points in a 3D scanner coordinate frame of reference, the 3D scanner being in a first position, the 3D scanner having a first light source, a beam steering unit, a light receiver, and a first position indicator, the 3D coordinates of a first object point being determined based on emitting a light with the first light source and receiving a reflected light with the light receiver;

engaging a second position indicator on a 2D scanner with the first position indicator while the 3D scanner is in the first position, the 2D scanner further having a second light source, an image sensor, and an inertial measurement unit, the second light source steers a beam of light within a first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points;

performing a 2D scan of the environment with the 2D scanner to acquire 2D scan data in 2D scanner coordinate frame of reference;

moving the 2D scanner from the first position to a second position while scanning the environment;

moving the 3D scanner from the first position to the second position;

engaging the second position indicator with the first position indicator with the 3D scanner in the second position;

performing a second scan of the environment with a 3D scanner and determining 3D coordinates of second collection of points in a 3D scanner coordinate frame of reference, the second scan being performed with the 3D scanner in the second position;

identifying with the one or more processors a correspondence between registration targets in the first collection of points and the second collection of points;

determining the 3D coordinates of a registered 3D collection of points based on a correspondence among registration targets, the 3D coordinates of the first collection of points, and the 3D coordinates of the second collection of points;

transferring the 2D scan data from the 2D scanner to the 3D scanner; and

transforming the 2D scan data from the 2D scanner coordinate frame of reference to the 3D scanner coordinate frame of reference in response to engage.

24. The method of claim 23, wherein the engaging of the first position indicator and the second position indicator includes transmitting an identification data from the second position indicator to the first position indicator.

25. The method of claim 24, wherein the first position indicator includes a near field communications transmitter and a near field communications receiver.

26. The method of claim 23, wherein the engaging of the first position indicator and the second position indicator includes closing a magnetically actuated switch within the 2D scanner and transmitting an identification data from the 2D scanner to the 3D scanner.

27. The method of claim 23, wherein the engaging of the first position indicator and the second position indicator includes:

contacting a first feature on a body of the 2D scanner with a second feature on a housing of the 3D scanner; and

actuating an actuator on one of the 2D scanner or the 3D scanner when the first feature is contacting the second feature.

28. The method of claim 23, wherein the engaging of the first position indicator and the second position indicator includes:

positioning the 2D scanner adjacent a machine readable indicia on a housing of the 3D scanner;

acquiring an image of the machine readable indicia with a camera coupled to the 2D scanner; and

transmitting an identification data from the 2D scanner to the 3D scanner.