Apparatus, systems and methods for point cloud generation and constantly tracking position

• US 10,481,265 B2
• Filed: 01/12/2017
• Issued: 11/19/2019
• Est. Priority Date: 12/21/2011
• Status: Active Grant

• Current Assignee: Robotic Paradigm Systems LLC
• Original Assignee: Robotic Paradigm Systems LLC
• Inventors: Coddington, Timothy, Gilbert, Lynn Coddington
• Primary Examiner(s): Hulka, James R
• Application Number:
  US15/405,304
• Publication Number:
  US 20170123066A1
• Time in Patent Office: 1,041 Days
• Field of Search: 356 401, 356 501
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/50 Systems of measurement based on relative movement of target
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/875 for determining attitude
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4817 relating to scanning

Abstract

A system having a range-finding laser device (RFLD) is configured to be coupled to an operator performs scans producing range and angle data points on surrounding structures. An attitude inertial measuring unit (IMU) attached to the RFLD measures pitch and roll of the RFLD and at least one zero-velocity update (zupt) IMU coupled to the operator is used to estimate the position, velocity and yaw of the operator. The system has computer logic that transforms data points from sensor frames of reference to a global frame of reference and merges transformed data points in a point cloud that can be used to generate images of scanned environments on a display.

Claims

1. A system for creating an image of a layout of one or more structures in an interior of a building, said system comprising:

a mobile range-finding laser device (RFLD) configured to be coupled to an operator and to perform scans, each scan measuring a plurality of data points comprising range and angle values relative to a current location of the range-finding laser device and said one or more structures in a field of view of the RFLD;

an attitude inertial measurement unit (IMU) that is affixed to the range-finding laser device that measures an attitude of the RFLD;

a first zero-velocity update (zupt) IMU configured to be coupled to the operator and to calculate a position, velocity, and yaw of the operator;

an image display device;

a computer in communication with the RFLD, the IMU, the first zupt and the image display device; and

a power supply providing power to one or more of the computer, the RFLD, the IMU, the first zupt, and the image display device

wherein said computer comprises logic configured to;

a) convert the plurality of data points for each scan to Cartesian data points,

b) compare the Cartesian data points for a current scan with Cartesian data points for a preceding scan to derive a first estimated change in position of the RFLD from a position of the preceding scan to a position of the current scan,

c) using data from the attitude IMU, calculate a change in pitch and roll of the RFLD for the current attitude relative to a pitch and roll of the RFLD for a preceding scan,

d) using the estimated position, velocity, and yaw data from the first zupt for the current scan and a predetermined offset between a center of the operator and a center of the RFLD, calculate a second estimated change in the position and attitude of the RFLD,

e) fuse the first estimated change in position, the second estimated change in position and the change in attitude to obtain-a fused change in position and attitude of the RFLD relative to a position and attitude of the RFLD for a preceding scan,

f) calculate a current global pose for the RFLD based upon the fused change in position and attitude RFLD in step e),

g) transform the plurality of data points from a sensor frame of reference to a global frame of reference for the current scan to obtain current transformed data points,

h) merge the current transformed data points with a point cloud comprising prior transformed data points from previous scans, and

i) display the current and prior transformed data points on the image display device to produce an image of said one or more structures in the field of view of the range-finding laser device.

2. The system of claim 1, wherein the logic is further configured to eliminate transformed data from the current scan redundant to prior-transformed data points from previous scans.

3. The system of claim 1, wherein the logic is further configured to eliminate current transformed data points that the logic determines to be statistical outliers.

4. The system of claim 1, further comprising a second zupt IMU in communication with the computer and wherein the first zupt IMU is configured to be coupled to a first foot of the operator and the second zupt IMU is configured to be coupled to a second foot of the operator and the first and second zupt IMUS are configured to estimate position, velocity, and yaw of the operator'"'"'s feet.

5. The system of claim 1, further comprising a backpack apparatus, which is configured to carry the range-finding laser device and the IMU and to be worn by the operator.

6. The system of claim 5, wherein the power supply is attached to the backpack apparatus.

7. The system of claim 5, wherein-the computer is a mobile computing device coupled to the backpack apparatus.

8. The system of claim 5, wherein the display device is coupled to the backpack apparatus via an arm comprising at least one pivot to enable the operator to manually position the display device in a plurality of positions.

9. The system of claim 1, further comprising a computing device communicatively coupled to the mobile computing device for receiving point cloud data.

10. The system of claim 1, wherein the RFLD is coupled to an extendable pole configured to be held by the operator.

11. The system of claim 1, wherein the range-finding laser device comprises a motor which, when actuated, rotates the range-finding laser device to change the pitch at which the range-finding laser device operates.

12. The system of claim 1, further comprising a camera and wherein the logic is configured to capture video via the camera and correlate the captured video with the display of transformed data points.

13. The system of claim 1, wherein said display device is a wrist display device configured to be worn by the operator for displaying an image of the transformed data points.

14. A method, for creating an image of a layout of a structure, said method comprising:

a) performing a current scan measuring a plurality of current range and angle value data points relative to a current location of a range-finding laser device (RFLD) and a surrounding structure in a field of view of the RFLD;

b) measuring a current pitch and roll of the RFLD via an attitude inertial measurement unit (IMU) that is affixed to the RFLD to determine a current attitude of the RFLD;

c) estimating position, velocity, and yaw of an operator carrying the RFLD via a first zero-velocity update (zupt) IMU coupled to the operator;

d) converting each of the plurality of current range and angle value data points to Cartesian data points, thereby generating current scan data;

e) comparing the current scan data with scan data from a preceding scan to derive a first estimated change in position of the RFLD from the position of the RFLD during the preceding scan;

f) calculating a change in the attitude of the RFLD relative to an attitude of the RFLD during the preceding scan,

g) converting the estimated change in position-of the operator carrying the laser range-finding device to a second estimated change in the position of the RFLD relative to the preceding scan;

h) fusing the first estimated change in position, estimated change in attitude and the second estimated change in position of the RFLD to obtain a fused change in position and attitude of the RFLD relative to the preceding scan;

i) calculating a current global pose based upon the fused change in position and attitude of the RFLD relative to the preceding scan;

j) transforming the Cartesian coordinates of the current scan data points from a sensor frame of reference to a global frame of reference to obtain current transformed data points;

k) merging the current transformed data points with a point cloud comprising prior transformed data points resulting from previous scans;

l) displaying the current transformed data points on a visual display and

m) moving the RFLD and repeating a)-I) to produce an image of said layout of said structure.

15. The method of claim 14, further comprising eliminating redundant range-finding laser device scan data from the current transformed data points resulting from slow range-finding laser device movement between scans.

16. The method of claim 14, further comprising eliminating data points in the current transformed data points determined to be statistical outliers.

17. The method of claim 14, further comprising attaching the first zupt IMU to a first foot of the operator and attaching a second zupt IMU to a second foot of the operator and, in step g, estimating position, velocity, and yaw of the operator'"'"'s feet using data from the first and second zupt IMUS.

18. The method of claim 14, further comprising transmitting point cloud data to a remote computing device.

19. The method of claim 14, further comprising capturing video via a camera and correlating the captured video with the displayed transformed data points.

20. The method of claim 14, wherein displaying the current transformed data points on a visual display comprises rendering an image in real time for each current scan, together with the transformed data points of prior scans, wherein the displayed image represents combined scans during operation.