Method and apparatus for an adaptive ladar receiver

• US 9,933,513 B2
• Filed: 05/09/2017
• Issued: 04/03/2018
• Est. Priority Date: 02/18/2016
• Status: Active Grant

• Current Assignee: AEye, Inc.
• Original Assignee: AEye, Inc.
• Inventors: Dussan, Luis Carlos, Steinhardt, Allan, Cook, David
• Primary Examiner(s): Bolda, Eric L
• Application Number:
  US15/590,491
• Publication Number:
  US 20170242108A1
• Time in Patent Office: 329 Days
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: G01S 7/4865 Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak peak detection in noise, signal conditioning G01S7/487
• CPC Class Codes: G01S 7/4868 Controlling received signal intensity or exposure of sensor
• CPC Class Codes: G01S 7/487 Extracting wanted echo signals , e.g. pulse detection
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating

Abstract

Disclosed herein are various embodiments of an adaptive ladar receiver and associated method whereby the active pixels in a photodetector array used for reception of ladar pulse returns can be adaptively controlled based at least in part on where the ladar pulses were targeted. Additional embodiments disclose improved imaging optics for use by the receiver and further adaptive control techniques for selecting which pixels of the photodetector array are used for sensing incident light.

Claims

1. A ladar receiver apparatus comprising:

an array comprising a plurality of light sensors, each light sensor configured to sense light that is indicative of a plurality of ladar pulses reflected from a plurality of range points and generate a signal indicative of the sensed light; and

a circuit in communication with the array, the circuit configured to selectively define a plurality of subsets of the light sensors for read out over time, wherein the circuit comprises;

a signal processing circuit; and

a multiplexer in communication with the array and the signal processing circuit, wherein the multiplexer is configured to selectively connect each defined subset of the light sensors with the signal processing circuit in response to a control signal such that the multiplexer provides the signal generated by each selectively connected light sensor to the signal processing circuit; and

wherein the signal processing circuit is configured to amplify and convert the provided signals into a plurality of digital samples for processing to compute range information with respect to the range points.

2. The apparatus of claim 1 wherein the circuit further comprises:

a control circuit in communication with the multiplexer, the control circuit configured to generate the control signal such that the control signal selects subsets of light sensors within the array based on locations of the range points in a field of view of a scanning ladar transmitter that transmits ladar pulses toward the range points.

3. The apparatus of claim 2 wherein the control circuit is further configured to generate the control signal such that the control signal selects subsets of light sensors within the array in a sequence that follows a shot list for the scanning ladar transmitter.

4. The apparatus of claim 2 wherein the shot list selectively targets a subset of range points within a field of view of the ladar transmitter.

5. The apparatus of claim 2 wherein the circuit further comprises:

a feedback circuit that provides a plurality of feedback signals to the light sensors for adjusting outputs from the light sensors in a controlled feedback loop.

6. The apparatus of claim 1 wherein the subset comprises a plurality of the light sensors, wherein circuits are positioned to combine signals from a plurality of light sensors to each of a plurality of multiplexer input lines to thereby define composite pixels for readout to the signal processing circuit.

7. The apparatus of claim 1 further comprising a photodetector that is positioned optically downstream from the scanning ladar transmitter, wherein the photodetector comprises a region that is positioned to receive (1) the ladar pulses from the scanning ladar transmitter, and (2) light from the scanning ladar transmitter that is at a different frequency than the ladar pulses, and wherein the region is configured to pass the ladar pulses but absorb and convert the light at the different frequency into an output signal;

and

wherein the circuit is further configured to track where the scanning ladar transmitter is targeted based on the output signal from the photodetector.

8. The apparatus of claim 7 wherein the circuit is further configured to selectively define the subsets of light sensors based on the tracking of where the scanning ladar transmitter is targeted when ladar pulses are transmitted.

9. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, and wherein the circuit is configured to adapt which pixels are included in the defined subsets based on information derived from the sensed light.

29. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, and wherein the circuit is further configured to adapt which pixels are included in the defined subsets based on a detection of malfunctioning pixels such that pixels detected as malfunctioning are not included in the defined subsets.

30. The apparatus of claim 29 wherein the circuit is further configured to detect pixels that are malfunctioning.

31. The apparatus of claim 29 wherein the malfunctioning pixels comprise dark pixels.

32. The apparatus of claim 29 wherein the malfunctioning pixels comprise white pixels.

33. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, and wherein the circuit is further configured to adapt which pixels are included in the defined subsets based on a detection of an oversaturation condition to adjust a dynamic range for the produced signal in a manner that reduces the oversaturation condition.

34. The apparatus of claim 33 wherein the circuit is further configured to detect whether there is an oversaturation condition present with respect to the light sensed by the subset of pixels.

35. The apparatus of claim 33 wherein the oversaturation condition corresponds to an oversaturated pixel in the subset.

36. The apparatus of claim 33 wherein the subset comprises a plurality of pixels, and wherein the oversaturation condition corresponds to an aggregation of pixels in the produced signal.

37. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, wherein the subset comprises a plurality of pixels, and wherein the circuit is further configured to adapt which pixels are included in the defined subsets based on a detection of a pixel that is overly saturated with interfering light such that the overly saturated pixel is not included in the defined subsets while it is overly saturated with noise from interfering light.

38. The apparatus of claim 37 wherein the circuit is further configured to detect whether any pixels in the subset are overly saturated with noise from interfering light.

39. The apparatus of claim 37 wherein the interfering light corresponds to a ladar pulse from a different ladar transmitter.

40. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, and wherein the circuit is further configured to adapt which pixels are included in the defined subsets based on a detection of a scattered ladar pulse reflection.

41. The apparatus of claim 1 wherein the ladar receiver apparatus is part of a ladar system, the ladar system further comprising the scanning ladar transmitter;

and

wherein the scanning ladar transmitter is configured to adjust a shot list of range points based on feedback from the ladar receiver apparatus.

42. The apparatus of claim 41 wherein the adjusted shot list avoids range points that correspond to overly saturated pixels in the array.

43. The apparatus of claim 41 wherein the adjusted shot list avoids range points that correspond to a region of the field of view that includes too much interfering light.

44. The apparatus of claim 1 wherein the ladar receiver apparatus is part of a ladar system, the ladar system further comprising the scanning ladar transmitter;

and

wherein the scanning ladar transmitter is configured to adjust how much power is included in the ladar pulses based on feedback from the ladar receiver apparatus.

45. The apparatus of claim 1 wherein the signal processing circuit further comprises (1) an amplifier configured to amplify the provided signals, and (2) an analog-to-digital converter (ADC) circuit downstream from the amplifier, the ADC circuit configured to convert the provided signals into the digital samples.

46. The apparatus of claim 1 wherein the signal processing circuit further comprises (1) an amplifier configured to amplify the provided signals, and (2) a time-to-digital converter (TDC) circuit downstream from the amplifier, the TDC circuit configured to convert the provided signals into the digital samples.

47. The apparatus of claim 1 wherein the signal processing circuit further comprises a field programmable gate array (FPGA), the FPGA configured to compute the range information based on the digital samples.

48. The apparatus of claim 47 wherein the FPGA is further configured to perform interpolation on the digital samples to compute the range information.

49. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, and wherein the subsets of selected light sensors change over time with respect to how many pixels are included in the subsets.

50. The apparatus of claim 1 wherein the light sensors of the array correspond to a plurality of pixels, and wherein a plurality of the pixels exhibit a hexagonal shape.

51. The apparatus of claim 50 wherein a plurality of the pixels exhibit different sizes.

52. The apparatus of claim 1 wherein a plurality of the pixels exhibit different sizes.

10. A ladar receiver apparatus comprising:

an array comprising a plurality of light sensors, each light sensor configured to sense light that is indicative of a plurality of ladar pulses reflected from a plurality of range points and generate a signal indicative of the sensed light; and

a circuit in communication with the array, the circuit configured to selectively define a plurality of subsets of the light sensors for read out over time to produce a signal representative of the sensed light, the produced signal for use in computing range information with respect to the range points;

the circuit comprising a plurality of amplifiers, each amplifier corresponding to a sensor in the array and configured to amplify an output from its corresponding sensor; and

wherein the circuit is further configured to selectively control which of the amplifiers are in a quiescent state and which are powered for operation based on the selected subsets.

53. The apparatus of claim 10 wherein the circuit further comprises:

a multiplexer in communication with the array via the amplifiers, wherein the multiplexer is configured to select each defined subset of the light sensors for read out in response to a control signal such that the multiplexer passes the amplified sensor outputs generated by the amplifiers corresponding to the sensors in each defined subset.

54. The apparatus of claim 53 wherein the circuit further comprises:

a control circuit in communication with the multiplexer, the control circuit configured to generate the control signal such that the control signal selects subsets of light sensors within the array based on locations of the range points in a field of view of a scanning ladar transmitter that transmits ladar pulses toward the range points.

11. A ladar receiver apparatus comprising:

an array comprising a plurality of light sensors, each light sensor configured to sense light that is indicative of a plurality of ladar pulses reflected from a plurality of range points and generate a signal indicative of the sensed light; and

a circuit in communication with the array, the circuit configured to selectively define a plurality of subsets of the light sensors for read out over time to produce a signal representative of the sensed light, the produced signal for use in computing range information with respect to the range points;

wherein the circuit includes a plurality of channels for light sensor readout, and wherein the circuit is further configured to select different subsets of the light sensors for the different channels at a given time to simultaneously produce a plurality of different channel-specific signals representative of the sensed light.

55. The apparatus of claim 11 wherein each channel includes:

a multiplexer in communication with the array, wherein the multiplexer is configured to select each defined subset of the light sensors for read out in that channel in response to a control signal such that the multiplexer passes the amplified sensor signals corresponding to the sensors in each defined subset for that channel.

56. The apparatus of claim 55 wherein the circuit further includes:

a control circuit in communication with the multiplexers, the control circuit configured to generate the control signals such that the control signals select subsets of light sensors within the array based on locations of the range points in a field of view of a scanning ladar transmitter that transmits ladar pulses toward the range points.

12. A method comprising:

transmitting a ladar pulse toward a targeted range point in a field of view;

sensing light via an array of light sensors, wherein the sensed light includes a reflection of the transmitted ladar pulse;

selecting a subset of the light sensors in the array based on a location of the targeted range point in the field of view;

generating a signal representative of the light sensed by the selected subset;

processing the generated signal, wherein the processing step includes converting the generated signal into a plurality of digital samples that are representative of the generated signal;

computing range information attributable to the targeted range point based on the processing; and

repeating the transmitting, sensing, selecting, generating, processing, and computing steps for a plurality of different targeted range points in the field of view.

13. The method of claim 12 further comprising:

selecting a subset of range points within the field of view for targeting via compressive sensing; and

wherein the transmitting and repeating steps comprise transmitting a plurality of ladar pulses toward the selected subset of range points.

14. The method of claim 12 wherein the generating step comprises:

amplifying a plurality of read signals from the light sensors using a controlled feedback loop; and

generating the sensed light signal based on the amplified read signals.

15. The method of claim 14 wherein the amplifying step is performed by a plurality of amplifiers that are positioned to receive a signal output from the array, wherein each amplifier corresponds to a sensor in the array such that each amplifier is positioned to amplify a signal output from its corresponding sensor, the method further comprising:

selectively controlling which of the amplifiers are in a quiescent state and which are powered for operation based on the selected subsets.

16. The method of claim 12 wherein the selecting and repeating steps comprise adaptively adjusting which light sensors are included in the selected subsets based on information derived from the sensed light.

17. The method of claim 12 wherein the repeating step comprises adjusting which range points are targeted by the transmitted ladar pulses based on information derived from the sensed light.

18. The method of claim 12 wherein the repeating step comprises transmitting ladar pulses at a reduced power level based on information derived from the sensed light.

19. The method of claim 12 wherein the transmitting and repeating steps comprise (1) transmitting the ladar pulses toward the targeted range points via a plurality of scanning mirrors, and (2) transmitting a reference light at a frequency different than the transmitted ladar pulses, the method further comprising:

a photodetector that is optically downstream from the scanning mirrors receiving the transmitted ladar pulses and the transmitted reference light;

the photodetector passing the received transmitted ladar pulses toward the range points;

the photodetector absorbing the received transmitted reference light and converting the absorbed reference light into a tracking signal; and

tracking where the scanning mirrors are targeted when the ladar pulses were transmitted based on the tracking signal.

20. The method of claim 12 further comprising:

directing a reference light onto the array via an optical path that is distinct from a path taken by (1) the transmitted ladar pulse to the targeted range point and (2) the ladar pulse reflection back to the array such that the sensed light includes a portion attributable to the reflection of the transmitted ladar pulse and a portion attributable to the reference light;

wherein the processing step comprises determining a timing relationship between the reflection of the transmitted ladar pulse and the reference light based on the generated signal.

21. The method of claim 20 wherein the computing step comprises computing the range information attributable to the targeted range point based on the determined timing relationship.

22. The method of claim 20 wherein the reference light and the transmitted ladar pulse share a common pulse shape, and wherein the step of determining the timing relationship comprises correlating a delayed version of the generated signal against the generated signal to determine the timing relationship.

23. The method of claim 22 wherein the distinct optical path comprises a direct optical path between (1) a transmitter that transmits the ladar pulse and the reference light and (2) the array.

24. The method of claim 12 wherein the selecting step comprises selecting a plurality of the light sensors for inclusion in the subset.

25. The method of claim 12 wherein each of a plurality of the selected subsets of light sensors includes different numbers of light sensors.

57. The method of claim 12 wherein the selecting step comprises selecting the subset of the light sensors using a multiplexer.

58. The method of claim 16 further comprising:

detecting a malfunctioning light sensor in the array; and

wherein the adaptively adjusting step comprises selecting the subsets of light sensors such that the subsets do not include the light sensor detected to be malfunctioning.

59. The method of claim 16 further comprising:

detecting an oversaturation condition with respect to at least one of a light sensor in the selected subset and the generated signal; and

wherein the adaptively adjusting step comprises selecting the subsets of light sensors to mitigate the detected oversaturation condition.

60. The method of claim 59 wherein the oversaturation condition corresponds to a light sensor that is oversaturated with sensed light.

61. The method of claim 59 wherein the oversaturation condition corresponds to a generated signal that exceeds a defined threshold.

62. The method of claim 16 wherein the adaptively adjusting step comprises adjusting which light sensors are included in the selected subsets to control a dynamic range for the generated signal.

63. The method of claim 16 further comprising:

detecting a light sensor in the selected subset for which its sensed light is corrupted by an interfering ladar pulse; and

wherein the adaptively adjusting step comprises selecting the subsets of light sensors such that the subsets do not include the light sensor corrupted by the interfering ladar pulse.

64. The method of claim 16 further comprising:

detecting a region in the field of view that is a source of noise;

determining a plurality of light sensors in the array that correspond to the detected region; and

wherein the adaptively adjusting step comprises selecting the subsets of light sensors such that the subsets do not include the determined light sensors.

65. The method of claim 16 further comprising:

detecting a light sensor in the selected subset for which its sensed light is corrupted by a scattering ladar pulse; and

wherein the adaptively adjusting step comprises selecting the subsets of light sensors such that the subsets do not include the light sensor corrupted by the scattering ladar pulse.

66. The method of claim 12 wherein the repeating step comprises transmitting ladar pulses at a reduced power level based on information derived from the sensed light.

67. The method of claim 12 wherein the light sensors in the array correspond to a plurality of pixels, wherein a plurality of the pixels exhibit a hexagonal shape.

68. The method of claim 12 wherein the light sensors in the array correspond to a plurality of pixels, wherein a plurality of the pixels exhibit different sizes.

69. The method of claim 12 wherein at least one of selected subsets of light sensors is a single light sensor in the array.

70. The method of claim 12 wherein the selecting step comprises selecting a plurality of the light sensors for inclusion in the subset.

71. The method of claim 12 wherein each of a plurality of the selected subsets of light sensors includes different numbers of light sensors.

26. A method comprising:

transmitting a ladar pulse toward a targeted range point in a field of view;

sensing light via an array of light sensors, wherein the sensed light includes a reflection of the transmitted ladar pulse;

providing a plurality of channels for signal readout from the array;

selecting a plurality of subsets of the light sensors in the array based on a location of the targeted range point in the field of view, wherein the selecting step comprises selecting different subsets for each of a plurality of the channels;

for each of a plurality of the channels, generating a signal representative of the light sensed by the selected subset for that channel;

for each of a plurality of the channels, processing the generated signal for that channel;

computing range information attributable to the targeted range point based on the processing; and

repeating the transmitting, sensing, selecting, generating, processing, and computing steps for a plurality of different targeted range points in the field of view.

27. The method of claim 26 further comprising:

selecting a subset of range points within the field of view for targeting via compressive sensing; and

wherein the transmitting and repeating steps comprise transmitting a plurality of ladar pulses toward the selected subset of range points.

28. The method of claim 26 wherein the selecting step comprises selecting the subsets of the light sensors using a multiplexer.