Device and method for merging lidar data

• US 10,663,590 B2
• Filed: 05/01/2017
• Issued: 05/26/2020
• Est. Priority Date: 05/01/2017
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Inventors: Rzeszutek, Richard Jeffrey, Scott, Bradley M.
• Primary Examiner(s): Hulka, James R
• Application Number:
  US15/583,680
• Publication Number:
  US 20180313956A1
• Time in Patent Office: 1,121 Days
• Field of Search: 356 501
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/003 Transmission of data between radar, sonar or lidar systems and remote stations
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/4813 Housing arrangements

Abstract

A device and method for merging lidar data is provided. Point cloud data is combined, via a lidar imaging controller, into a common point cloud data set, each set of point cloud data representing respective angular lidar scans of a region as at least two lidar devices are moved relative to the region of a shelf. The respective angular lidar scans from each lidar device occur at a non-zero and non-perpendicular angle to a movement direction. Common point cloud data set points are binned into a plane perpendicular to the movement direction of a mobile automation apparatus and extending from a virtual lidar position. The lidar imaging controller combines points among multiple planes.

Claims

1. A device comprising:

a lidar imaging controller and a communication interface, the controller configured to;

receive, via the communication interface, point cloud data representing respective angular lidar scans of a region as at least two lidar devices are moved relative the region, the respective angular lidar scans occurring at a non-zero and non-perpendicular angle to a movement direction;

combine the point cloud data into a common point cloud data set;

bin points from the common point cloud data set into a plane perpendicular to the movement direction and extending from a virtual lidar position relative to the common point cloud data set;

while ignoring outlier points located behind closer points to the virtual lidar position, perform curve fitting on the closer points to smooth out noise from the closer points and produce curve fit points in the plane; and

combine the curve fit points with other curve points from other planes extending from other virtual lidar positions along the movement direction into a virtual lidar scan of the region along the movement direction.

2. The device of claim 1, wherein the controller is further configured to combine the point cloud data into the common point cloud data set using navigational data of an apparatus moving the at least two lidar devices in the movement direction.

3. The device of claim 1, wherein the controller is further configured to bin the points from the common point cloud data set into the plane perpendicular to the movement direction using a threshold distance.

4. The device of claim 1, wherein the outlier points each comprise a point that is further away from the virtual lidar position than a respective closer point along a common line extending from the virtual lidar position to the point and the respective closer point.

5. The device of claim 1, wherein the virtual lidar position is located at a position between the at least two lidar devices relative to the common point cloud data set.

6. The device of claim 1, wherein the virtual lidar position is located at a position of a camera relative to the common point cloud data set.

7. The device of claim 6, wherein the position of the camera is between the at least two lidar devices relative to the common point cloud data set.

8. The device of claim 6, wherein the controller is further configured to compare the virtual lidar scan with camera data from the camera to determine positions of features in the region, the camera data comprising one or more images of the region acquired by the camera.

9. The device of claim 1, wherein each point of the plane is represented in polar coordinates.

10. A method comprising:

at a device comprising a lidar imaging controller and a communication interface, receiving, via the communication interface, point cloud data representing respective angular lidar scans of a region as at least two lidar devices are moved relative the region, the respective angular lidar scans occurring at a non-zero and non-perpendicular angle to a movement direction;

combining, using the controller, the point cloud data into a common point cloud data set;

binning, using the controller, points from the common point cloud data set into a plane perpendicular to the movement direction and extending from a virtual lidar position relative to the common point cloud data set;

while ignoring outlier points located behind closer points to the virtual lidar position, performing, using the controller, curve fitting on the closer points to smooth out noise from the closer points and produce curve fit points in the plane; and

combining, using the controller, the curve fit points with other curve points from other planes extending from other virtual lidar positions along the movement direction into a virtual lidar scan of the region along the movement direction.

11. The method of claim 10, further comprising combining the point cloud data into the common point cloud data set using navigational data of an apparatus moving the at least two lidar devices in the movement direction.

12. The method of claim 10, further comprising binning the points from the common point cloud data set into the plane perpendicular to the movement direction using a threshold distance.

13. The method of claim 10, wherein the outlier points each comprise a point that is further away from the virtual lidar position than a respective closer point along a common line extending from the virtual lidar position to the point and the respective closer point.

14. The method of claim 10, wherein the virtual lidar position is located at a position between the at least two lidar devices relative to the common point cloud data set.

15. The method of claim 10, wherein the virtual lidar position is located at a position of a camera relative to the common point cloud data set.

16. The method of claim 15, wherein the position of the camera is between the at least two lidar devices relative to the common point cloud data set.

17. The method of claim 15, further comprising comparing the virtual lidar scan with camera data from the camera to determine positions of features in the region, the camera data comprising one or more images of the region acquired by the camera.

18. The method of claim 10, wherein each point of the plane is represented in polar coordinates.

19. A non-transitory computer-readable medium storing a computer program, wherein execution of the computer program is for:

at a device comprising a lidar imaging controller and a communication interface, receiving, via the communication interface, point cloud data representing respective angular lidar scans of a region as at least two lidar devices are moved relative the region, the respective angular lidar scans occurring at a non-zero and non-perpendicular angle to a movement direction;

combining, using the controller, the point cloud data into a common point cloud data set;

binning, using the controller, points from the common point cloud data set into a plane perpendicular to the movement direction and extending from a virtual lidar position relative to the common point cloud data set;

while ignoring outlier points located behind closer points to the virtual lidar position, performing, using the controller, curve fitting on the closer points to smooth out noise from the closer points and produce curve fit points in the plane; and

combining, using the controller, the curve fit points with other curve points from other planes extending from other virtual lidar positions along the movement direction into a virtual lidar scan of the region along the movement direction.