System and method for synthetically filling ladar frames based on prior ladar return data

• US 10,670,718 B1
• Filed: 03/18/2019
• Issued: 06/02/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: AEye, Inc.
• Original Assignee: AEye, Inc.
• Inventors: Dussan, Luis Carlos, Steinhardt, Allan, Presutti, Federico, Benscoter, Joel David
• Primary Examiner(s): Bolda, Eric L
• Application Number:
  US16/356,116
• Publication Number:
  US 20200200878A1
• Time in Patent Office: 442 Days
• CPC Class Codes: G01S 17/006 Theoretical aspects
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/93 for anti-collision purposes
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/4814 of transmitters alone
• CPC Class Codes: G01S 7/4816 of receivers alone
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/484 Transmitters
• CPC Class Codes: G01S 7/4861 Circuits for detection, sampling, integration or read-out
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates

Abstract

Systems and methods are disclosed where a ladar system synthetically fills a ladar frame. A ladar transmitter can employ compressive sensing to interrogate a subset of range points in a field of view. Returns from this subset of range points correspond to a sparse ladar frame, and interpolation can be performed on these returns to synthetically fill the ladar frame.

Claims

1. A ladar system comprising:

a ladar transmitter configured to transmit a plurality of ladar pulse shots into a coordinate space toward a plurality of range points such that less than a full set of range points within a ladar frame are targeted with the ladar pulse shots;

a ladar receiver configured to receive incident light and detect ladar returns from the ladar pulse shots based on the received incident light; and

a processor configured to synthetically fill a ladar frame based on the detected ladar returns such that the ladar frame includes synthetic return data for range points in the ladar frame that were not targeted with ladar pulse shots; and

wherein the ladar transmitter is further configured to employ a rolling scan of the ladar frame to support generation of a plurality of the synthetically-filled ladar frames.

2. The system of claim 1 wherein the ladar transmitter is further configured to employ compressive sensing within the ladar frame to transmit ladar pulse shots into the coordinate space toward a plurality of range points such that less than the full set of range points within the ladar frame are targeted with the ladar pulse shots.

3. The system of claim 1 wherein the processor is further configured to compute the synthetic return data based on a plurality of interpolations from the detected ladar returns.

4. The system of claim 1 wherein the processor is further configured to compute, for each of a plurality of non-targeted range points in the ladar frame, the synthetic return data for that non-targeted range point based on the detected return data for a plurality of range points in a defined vicinity of that non-targeted range point.

5. The system of claim 4 further comprising:

a memory configured to store data about the detected ladar returns from prior ladar pulse shots in a spatial index, the spatial index associating the detected ladar return data with a plurality of locations in the coordinate space; and

wherein the processor is further configured to retrieve the detected return data in the defined vicinities for use in the synthetic return data computations from the spatial index based on the locations of the non-targeted range points in the coordinate space.

6. The system of claim 1 wherein the detected ladar return data comprises intensity data for the targeted range points, and wherein the synthetic return data comprises synthetic intensity data for the non-targeted range points.

7. The system of claim 1 wherein the detected ladar return data comprises range data for the targeted range points, and wherein the synthetic return data comprises synthetic range data for the non-targeted range points.

8. The system of claim 1 further comprising:

an image classification application for execution by a processor, the image classification application configured to process the synthetically-filled ladar frame to classify an object that is depicted in the synthetically-filled ladar frame.

9. The system of claim 1 wherein the ladar transmitter is further configured to employ the rolling scan of the ladar frame using compressive sensing to support the generation of synthetically-filled ladar frames.

10. The system of claim 9 wherein the rolling scan skips two lines in a field of view for the ladar frame in ascending and descending shot schedules.

11. The system of claim 9 wherein the processor is further configure to estimate a velocity of an object depicted in the synthetically-filled ladar frames based on comparative positions of the object in the synthetically-filled ladar frames.

12. The system of claim 11 wherein a synthetically-filled ladar frame corresponding to an ascending scan of a field of view and a synthetically-filled ladar frame corresponding to a descending scan of a field of view are used by the processor for the velocity estimation.

13. The system of claim 11 wherein the object exhibits a size that is at least as large as a skip length for the rolling scan.

14. The system of claim 1 wherein the ladar transmitter includes a plurality of scanable mirrors;

wherein the processor is further configured to dynamically schedule a plurality of ladar pulse shots for the ladar transmitter to target a plurality of range points in the coordinate space;

wherein the ladar transmitter is further configured to (1) controllably scan the scanable mirrors to target the ladar transmitter at the targeted range points in accordance with the scheduled ladar pulse shots, and (2) transmit a plurality of ladar pulse shots toward the targeted range points via the controllably scanned mirrors.

15. The system of claim 14 wherein the processor is further configured to perform the dynamic scheduling on a shot-by-shot basis.

16. A method comprising:

a ladar transmitter transmitting a plurality of ladar pulse shots into a coordinate space toward a plurality of range points such that less than a full set of range points within a ladar frame are targeted with the ladar pulse shots;

a ladar receiver receiving incident light;

the ladar receiver detecting ladar returns from the ladar pulse shots based on the received incident light; and

a processor synthetically filling a ladar frame based on the detected ladar returns such that the ladar frame includes synthetic return data for range points in the ladar frame that were not targeted with ladar pulse shots; and

wherein the transmitting step includes employing a rolling scan of the ladar frame to support generation of a plurality of the synthetically-filled ladar frames.

17. The method of claim 16 wherein the transmitting step comprises the ladar transmitter compressively sensing within the ladar frame by transmitting ladar pulse shots into the coordinate space toward a plurality of range points such that less than the full set of range points within the ladar frame are targeted with the ladar pulse shots.

18. The method of claim 16 wherein the synthetically filling step comprises the processor computing the synthetic return data based on a plurality of interpolations from the detected ladar returns.

19. The method of claim 16 wherein the synthetically filling step comprises the processor computing, for each of a plurality of non-targeted range points in the ladar frame, the synthetic return data for that non-targeted range point based on the detected return data for a plurality of range points in a defined vicinity of that non-targeted range point.

20. The method of claim 19 further comprising:

storing data about the detected ladar returns from prior ladar pulse shots in a spatial index, the spatial index associating the detected ladar return data with a plurality of locations in the coordinate space; and

the processor retrieving the detected return data in the defined vicinities for use in the synthetic return data computations from the spatial index based on the locations of the non-targeted range points in the coordinate space.

21. The method of claim 16 wherein the detected ladar return data comprises intensity data for the targeted range points, and wherein the synthetic return data comprises synthetic intensity data for the non-targeted range points.

22. The method of claim 16 wherein the detected ladar return data comprises range data for the targeted range points, and wherein the synthetic return data comprises synthetic range data for the non-targeted range points.

23. The method of claim 16 further comprising:

an image classification application processing the synthetically-filled ladar frame to classify an object that is depicted in the synthetically-filled ladar frame.

24. The method of claim 16 wherein the rolling scan skips two lines in a field of view for the ladar frame in ascending and descending shot schedules.

25. The method of claim 16 further comprising:

the processor estimating a velocity of an object depicted in the synthetically-filled ladar frames based on comparative positions of the object in the synthetically-filled ladar frames.

26. The method of claim 25 wherein a synthetically-filled ladar frame corresponding to an ascending scan of a field of view and a synthetically-filled ladar frame corresponding to a descending scan of a field of view are used by the processor for the velocity estimation.

27. The method of claim 25 wherein the object exhibits a size that is at least as large as a skip length for the rolling scan.

28. The method of claim 16 wherein the ladar system includes a ladar transmitter that transmits ladar pulse shots into the coordinate space via a plurality of scanning mirrors, and wherein the method further comprises:

a processor dynamically scheduling a plurality of ladar pulse shots for the ladar transmitter to target a plurality of range points in the coordinate space;

the ladar transmitter controllably scanning the mirrors to target the ladar transmitter at the targeted range points in accordance with the scheduled ladar pulse shots; and

the ladar transmitter transmitting a plurality of ladar pulse shots toward the targeted range points via the scanning mirrors.

29. The method of claim 28 wherein the dynamically scheduling step comprises a processor performing the dynamic scheduling on a shot-by-shot basis.