Ladar point cloud compression

• US 10,386,464 B2
• Filed: 08/14/2015
• Issued: 08/20/2019
• Est. Priority Date: 08/15/2014
• Status: Active Grant

• Current Assignee: AEye, Inc.
• Original Assignee: AEye, Inc.
• Inventors: Dussan, Luis Carlos
• Primary Examiner(s): Hellner, Mark
• Application Number:
  US14/827,202
• Publication Number:
  US 20160047903A1
• Time in Patent Office: 1,467 Days
• Field of Search: 3561411
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4814 of transmitters alone
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/4861 Circuits for detection, sampling, integration or read-out
• CPC Class Codes: G01S 7/499 using polarisation effects

Abstract

Various embodiments are disclosed ladar point cloud compression. For example, a processor can be used to analyze data representative of an environmental scene to intelligently select a subset of range points within a frame to target with ladar pulses via a scanning ladar transmission system. In another example embodiment, a processor can perform ladar point cloud compression by (1) processing data representative of an environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud.

Claims

1. A method comprising:

a processor receiving frame data representative of an environmental scene that serves as a scan area for a scanning ladar transmission system, the frame data comprising a plurality of points that define the scan area;

the processor identifying a plurality of frame data segments corresponding to different segments of the environmental scene;

the processor performing corner detection on the frame data to identify a corner feature within the environmental scene; and

the processor selecting a subset of the points to use as range points for targeting by ladar pulses transmitted by the scanning ladar transmission system, wherein the subset includes a plurality of points corresponding to the identified corner feature within the environmental scene, wherein the selecting step comprises the processor selecting a relatively higher concentration of range points within an area of the environmental scene corresponding to the identified corner feature than within another area of the environmental scene, and wherein the selecting step further comprises the processor selecting the subset of range points on a segment-by-segment basis; and

the scanning ladar transmission system transmitting a plurality of ladar pulses toward the range points of the selected subset.

13. The method of claim 1 wherein the transmitting step comprises the scanning ladar transmission system transmitting the ladar pulses toward the range points of the selected subset according to a dynamic scan pattern.

14. The method of claim 1 further comprising:

the processor repeating the receiving, performing, and selecting step for each of a plurality of frames representative of the environmental scene; and

the scanning ladar transmission system transmitting the ladar pulses toward the range points of the selected subset for each frame.

15. The method of claim 1 further comprising the processor controlling the segmentation according to a size parameter and a threshold parameter.

16. The method of claim 1 wherein the performing step comprises the processor performing the corner detection separately for each of a plurality of the identified frame data segments.

17. The method of claim 16 further comprising:

the processor performing the segment-by-segment selecting according to at least one member of the group consisting of (1) a distance parameter, (2) an angle parameter, and (3) a density parameter.

18. The method of claim 16 wherein the selecting step further comprises:

the processor selecting a plurality of points within the frame data as range point candidates for retention in the subset; and

the processor refining the range point candidates for retention in the subset based on a defined point budget parameter.

19. The method of claim 18 wherein the selecting step further comprises:

the processor defining a fill pattern for the environmental scene; and

the processor selecting a plurality of range points candidates in accordance with the defined fill pattern.

20. The method of claim 19 wherein the selecting step further comprises the processor refining the range point candidates for retention in the subset based on a distance parameter to eliminate a plurality of range point candidates that are deemed too close to each other in view of the distance parameter.

21. The method of claim 1 wherein the frame data comprises image data representative of the environmental scene.

22. The method of claim 1 wherein the frame data is derived from a ladar point cloud representative of the environmental scene.

66. The apparatus of claim 21 further comprising:

an environmental sensing system comprising a camera, the camera configured to generate image data of the environmental scene, and wherein the environmental sensing system is configured to communicate the image data to the processor for processing thereby.

2. A method comprising:

a processor receiving frame data representative of an environmental scene that serves as a scan area for a scanning ladar transmission system, the frame data comprising a plurality of points that define the scan area;

the processor identifying a plurality of frame data segments corresponding to different segments of the environmental scene;

the processor controlling the segmentation according to a size parameter and a threshold parameter;

the processor analyzing each of a plurality of the identified frame data segments separately to identify high contrast areas within such frame data segments; and

the processor selecting a subset of the points to use as range points for targeting by ladar pulses transmitted by the scanning ladar transmission system, wherein the subset includes a relatively higher concentration of range points within an area of the environmental scene corresponding to identified high contrast areas than within another area of the environmental scene; and

the scanning ladar transmission system transmitting a plurality of ladar pulses toward the range points of the selected subset.

23. The method of claim 2 wherein the analyzing step comprises the processor performing a member of the group consisting of edge detection and corner detection.

24. The method of claim 2 wherein the frame data comprises image data representative of the environmental scene.

25. The method of claim 2 wherein the frame data is derived from a ladar point cloud representative of the environmental scene.

3. An apparatus comprising:

a scanning ladar transmission system; and

a processor configured to receive frame data representative of an environmental scene that serves as a scan area for the scanning ladar transmission system, the frame data comprising a plurality of points that define the scan area, (2) identify a plurality of frame data segments corresponding to different segments of the environmental scene, (3) perform corner detection on the frame data to identify a corner feature within the environmental scene, and (4) select a subset of the points to use as range points for targeting by ladar pulses transmitted by the scanning ladar transmission system, wherein the subset includes a plurality of points corresponding to the identified corner feature within the environmental scene;

wherein the processor is further configured to select the subset such that a relatively higher concentration of range points within an area of the environmental scene corresponding to the identified corner feature are selected than within another area of the environmental scene;

wherein the processor is further configured to select the range points for inclusion in the subset on a segment-by-segment basis; and

wherein the scanning ladar transmission system is configured to transmit a plurality of ladar pulses toward the range points of the selected subset.

26. The apparatus of claim 3 wherein the scanning ladar transmission system is further configured to transmit the ladar pulses toward the range points of the selected subset according to a dynamic scan pattern.

27. The apparatus of claim 3 wherein the processor is further configured to repeat the receive operation, the corner detection performance operation, and the subset selection operation for each of a plurality of frames representative of the environmental scene;

and

wherein the scanning ladar transmission system is further configured to transmit the ladar pulses toward the range points of the selected subset for each frame.

28. The apparatus of claim 3 wherein the processor is further configured to control the segmentation according to a size parameter and a threshold parameter.

29. The apparatus of claim 3 wherein the processor is further configured to perform the corner detection separately for each of a plurality of the identified frame data segments.

30. The apparatus of claim 29 wherein the processor is further configured to perform the segment-by-segment selection of range points for inclusion in the subset according to at least one member of the group consisting of (1) a distance parameter, (2) an angle parameter, and (3) a density parameter.

31. The apparatus of claim 29 wherein the processor is further configured to (1) select a plurality of points within the frame data as range point candidates for retention in the subset, and (2) refine the range point candidates for retention in the subset based on a defined point budget parameter.

32. The apparatus of claim 31 wherein the processor is further configured to (1) define a fill pattern for the environmental scene, and (2) select a plurality of range points candidates in accordance with the defined fill pattern.

33. The apparatus of claim 32 wherein the processor is further configured to refine the range point candidates for retention in the subset based on a distance parameter to eliminate a plurality of range point candidates that are deemed too close to each other in view of the distance parameter.

34. The apparatus of claim 3 wherein the frame data comprises image data representative of the environmental scene.

35. The apparatus of claim 3 wherein the frame data is derived from a ladar point cloud representative of the environmental scene.

4. An apparatus comprising:

a scanning ladar transmission system; and

a processor configured to (1) receive frame data representative of an environmental scene that serves as a scan area for the scanning ladar transmission system, the frame data comprising a plurality of points that define the scan area, (2) identify a plurality of frame data segments corresponding to different segments of the environmental scene, (3) control the segmentation according to a size parameter and a threshold parameter, (4) analyze each of a plurality of the identified frame data segments separately to identify high contrast areas within such frame data segments, and (5) select a subset of the points to use as range points for targeting by ladar pulses transmitted by the scanning ladar transmission system, wherein the subset includes a relatively higher concentration of range points within an area of the environmental scene corresponding to identified high contrast areas than within another area of the environmental scene; and

wherein the scanning ladar transmission system is configured to transmit a plurality of ladar pulses toward the range points of the selected subset.

36. The apparatus of claim 4 wherein the processor is further configured to perform a member of the group consisting of edge detection and corner detection as part of the analysis operation.

37. The apparatus of claim 4 wherein the frame data comprises image data representative of the environmental scene.

38. The apparatus of claim 4 wherein the frame data is derived from a ladar point cloud representative of the environmental scene.

5. A method comprising:

generating a 3D point cloud via ladar, the 3D point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor compressing the 3D point cloud, wherein the compressing step comprises the processor (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the 3D point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated 3D point cloud; and

wherein the compressing step further comprises the processor flattening the 3D point cloud into a 2D intensity map from a selected perspective from among a plurality of perspectives, wherein the 2D intensity map serves as the environmental scene data.

39. The method of claim 5 wherein the ladar is scanning ladar.

40. The method of claim 5 wherein the ladar is flash ladar.

43. The method of claim 5 wherein the processing step comprises the processor analyzing the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and wherein the selecting step comprises the processor selecting a relatively higher concentration of range points within the identified area than within another area of the environmental scene.

6. A method comprising:

generating a point cloud via ladar, the point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor compressing the point cloud, wherein the compressing step comprises the processor (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud;

wherein the processing step comprises the processor analyzing the data representative of the environmental scene to identify areas of contrast within the environmental scene that exceed a threshold, and wherein the selecting step comprises the processor selecting a relatively higher concentration of range points within the identified area than within another area of the environmental scene.

41. The method of claim 6 wherein the point cloud comprises a 3D point cloud.

42. The method of claim 6 wherein the processing step further comprises the processor identifying a fill portion of the environmental scene, and wherein the selecting step comprises the processor (1) defining a fill pattern for the fill portion, and (2) selecting a plurality of range points within the fill portion in accordance with the defined fill pattern.

7. A method comprising:

generating a point cloud via ladar, the point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor compressing the point cloud, wherein the compressing step comprises the processor (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud;

wherein the processing step comprises the processor analyzing the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and wherein the selecting step comprises the processor selecting a relatively higher concentration of range points within the identified area than within another area of the environmental scene;

wherein the processing step further comprises the processor identifying a fill portion of the environmental scene, and wherein the selecting step comprises the processor (1) defining a fill pattern for the fill portion, and (2) selecting a plurality of range points within the fill portion in accordance with the defined fill pattern.

8. A method comprising:

generating a point cloud via ladar, the point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor compressing the point cloud, wherein the compressing step comprises the processor (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud;

wherein the processing step comprises;

the processor pre-filtering the data representative of the environmental scene to reduce noise according to a filter;

the processor analyzing the pre-filtered data to identify a plurality of segments of the environmental scene; and

wherein the selecting step comprises the processor selecting the range points on a segment-by-segment basis.

44. The method of claim 8 wherein the processing step further comprises the processor selecting the filter for the pre-filtering step based on a filter select parameter, wherein the selected filter is configured to perform an edge detection operation.

45. The method of claim 8 wherein the step of analyzing the pre-filtered data to identify the segments comprises the processor controlling the segmentation according to a size parameter and a threshold parameter.

46. The method of claim 8 wherein the selecting step further comprises:

for each of a plurality of the identified segments, the processor analyzing an identified segment to identify a plurality of range points that correspond to at least one member of the group consisting of an edge, a face, and a corner within the environmental scene.

47. The method of claim 46 wherein the step of analyzing the identified segments comprises the processor controlling the segment analysis according to at least one member of the group consisting of (1) a distance parameter, (2) an angle parameter, and (3) a density parameter.

48. The method of claim 46 wherein the selecting step further comprises:

the processor selecting the identified range points as range point candidates for retention in the compressed point cloud; and

the processor refining the range point candidates for retention in the compressed point cloud based on a defined point budget parameter.

49. The method of claim 48 wherein the selecting step further comprises:

the processor defining a fill pattern for the environmental scene; and

the processor selecting a plurality of range points candidates in accordance with the defined fill pattern.

50. The method of claim 49 wherein the selecting step further comprises the processor refining the range point candidates for retention in the compressed point cloud based on a distance parameter to eliminate a plurality of range point candidates that are deemed too close to each other in view of the distance parameter.

51. The method of claim 8 further comprising electronically communicating the compressed point cloud over a channel to a destination.

52. The method of claim 8 wherein the processed data comprises an image of the environmental scene.

9. An apparatus comprising:

a ladar system, the ladar system configured to generate a 3D point cloud via ladar, the 3D point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor configured to compress the 3D point cloud by (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the 3D point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated 3D point cloud; and

wherein the processor is further configured to flatten the 3D point cloud into a 2D intensity map from a selected perspective from among a plurality of perspectives, wherein the 2D intensity map serves as the environmental scene data.

53. The apparatus of claim 9 wherein the ladar system is a scanning ladar system.

54. The apparatus of claim 9 wherein the ladar system is a flash ladar system.

57. The apparatus of claim 9 wherein the processor is further configured to (1) analyze the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and (2) select a relatively higher concentration of range points within the identified area than within another area of the environmental scene.

67. The apparatus of claim 9 wherein the processor is further configured to communicate the compressed point cloud over a channel to a destination.

68. The apparatus of claim 9 further comprising:

an environmental sensing system comprising a camera, the camera configured to generate image data of the environmental scene, and wherein the environmental sensing system is configured to communicate the image data to the processor for processing thereby.

10. An apparatus comprising:

a ladar system, the ladar system configured to generate a point cloud via ladar, the point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor configured to compress the point cloud by (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud; and

wherein the processor is further configured to (1) analyze the data representative of the environmental scene to identify areas of contrast within the environmental scene that exceed a threshold, and (2) select a relatively higher concentration of range points within the identified area than within another area of the environmental scene.

55. The apparatus of claim 10 wherein the point cloud comprises a 3D point cloud.

56. The apparatus of claim 10 wherein the processor is further configured to (1) identify a fill portion of the environmental scene, (2) define a fill pattern for the fill portion, and (3) select a plurality of range points within the fill portion in accordance with the defined fill pattern.

69. The apparatus of claim 10 wherein the ladar system is a scanning ladar system.

70. The apparatus of claim 10 wherein the ladar system is a flash ladar system.

71. The apparatus of claim 10 wherein the processor is further configured to (1) analyze the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and (2) select a relatively higher concentration of range points within the identified area than within another area of the environmental scene.

72. The apparatus of claim 10 wherein the processor is further configured to communicate the compressed point cloud over a channel to a destination.

73. The apparatus of claim 10 further comprising:

an environmental sensing system comprising a camera, the camera configured to generate image data of the environmental scene, and wherein the environmental sensing system is configured to communicate the image data to the processor for processing thereby.

11. An apparatus comprising:

a ladar system, the ladar system configured to generate a point cloud via ladar, the point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor configured to compress the point cloud by (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud;

wherein the processor is further configured to (1) analyze the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and (2) select a relatively higher concentration of range points within the identified area than within another area of the environmental scene; and

wherein the processor is further configured to (1) identify a fill portion of the environmental scene, (2) define a fill pattern for the fill portion, and a) select a plurality of range points within the fill portion in accordance with the defined fill pattern.

74. The apparatus of claim 11 wherein the ladar system is a scanning ladar system.

75. The apparatus of claim 11 wherein the ladar system is a flash ladar system.

76. The apparatus of claim 11 wherein the point cloud comprises a 3D point cloud.

77. The apparatus of claim 11 wherein the processor is further configured to (1) analyze the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and (2) select a relatively higher concentration of range points within the identified area than within another area of the environmental scene.

78. The apparatus of claim 11 wherein the processor is further configured to communicate the compressed point cloud over a channel to a destination.

79. The apparatus of claim 11 further comprising:

an environmental sensing system comprising a camera, the camera configured to generate image data of the environmental scene, and wherein the environmental sensing system is configured to communicate the image data to the processor for processing thereby.

12. An apparatus comprising:

a ladar system, the ladar system configured to generate a point cloud via ladar, the point cloud being representative of an environmental scene and comprising a plurality of range points; and

a processor configured to compress the point cloud by (1) processing data representative of the environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud; and

wherein the processor is further configured to;

pre-filter the data representative of the environmental scene to reduce noise according to a filter;

analyze the pre-filtered data to identify a plurality of segments of the environmental scene; and

select the range points on a segment-by-segment basis.

58. The apparatus of claim 12 wherein the processor is further configured to select the filter for the pre-filtering operation based on a filter select parameter, and wherein the selected filter is configured to perform an edge detection operation.

59. The apparatus of claim 12 wherein the processor is further configured to control the segmentation according to a size parameter and a threshold parameter.

60. The apparatus of claim 12 wherein processor is further configured to, for each of a plurality of the identified segments, analyze an identified segment to identify a plurality of range points that correspond to at least one member of the group consisting of an edge, a face, and a corner within the environmental scene.

61. The apparatus of claim 60 wherein the processor is further configured to control the segment analysis according to at least one member of the group consisting of (1) a distance parameter, (2) an angle parameter, and (3) a density parameter.

62. The apparatus of claim 60 wherein the processor is further configured to (1) select the identified range points as range point candidates for retention in the compressed point cloud, and (2) refine the range point candidates for retention in the compressed point cloud based on a defined point budget parameter.

63. The apparatus of claim 62 wherein the processor is further configured to (1) define a fill pattern for the environmental scene, and (2) select a plurality of range points candidates in accordance with the defined fill pattern.

64. The apparatus of claim 63 wherein the processor is further configured to refine the range point candidates for retention in the compressed point cloud based on a distance parameter to eliminate a plurality of range point candidates that are deemed too close to each other in view of the distance parameter.

65. The apparatus of claim 12 wherein the processor is further configured to communicate the compressed point cloud over a channel to a destination.

80. The apparatus of claim 12 wherein the ladar system is a scanning ladar system.

81. The apparatus of claim 12 wherein the ladar system is a flash ladar system.

82. The apparatus of claim 12 wherein the point cloud comprises a 3D point cloud.

83. The apparatus of claim 12 wherein the processor is further configured to (1) analyze the data representative of the environmental scene to identify areas within the environmental scene corresponding to a line or corner, and (2) select a relatively higher concentration of range points within the identified area than within another area of the environmental scene.