Trajectory detection devices and methods

• US 10,600,187 B2
• Filed: 03/20/2019
• Issued: 03/24/2020
• Est. Priority Date: 03/20/2018
• Status: Active Grant

• Current Assignee: Owl Autonomous Imaging, Inc.
• Original Assignee: Owl Autonomous Imaging, Inc.
• Inventors: Petilli, Eugene M.
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US16/358,873
• Publication Number:
  US 20190295264A1
• Time in Patent Office: 370 Days
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10036 Multispectral image; Hyperspectral image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: H01L 27/14618 Containers
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H01L 27/14636 Interconnect structures
• CPC Class Codes: H01L 27/14669 Infrared imagers
• CPC Class Codes: H01L 31/024 Arrangements for cooling, heating, ventilating or temperature compensation for photovoltaic devices H01L31/052
• CPC Class Codes: H01L 31/035218 the quantum structure being quantum dots
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules

Abstract

A trajectory detection device includes a lens configured to receive an image of a field of view. An array of microlenses is configured to create an array of light field images based on the image. A detector array includes a plurality of photon sensitive photodetectors. The detector array is configured to generate output signals from each photodetector based on the array of light field images. A controller is configured to integrate the output signals over an integration period. At least a portion of the output signals are modulated at a modulating frequency having a modulating frequency cycle time that is smaller than the integration period. A three-dimensional image of motion in the field of view is generated based on the integration of the modulated output signals.

Claims

1. A trajectory detection device comprising:

a lens configured to receive an image of a field of view of the lens;

an array of microlenses optically coupled to the lens and configured to create an array of light field images based on the image;

a detector array comprising a plurality of photon sensitive photodetectors, the detector array being optically coupled to the array of microlenses and configured to generate output signals from each photodetector based on the array of light field images; and

a controller communicatively coupled to the detector array and configured to;

receive the output signals from the detector array;

integrate the output signals over an integration period, wherein at least a portion of the output signals are modulated at a modulating frequency having a modulating frequency cycle time that is smaller than the integration period; and

generate a three-dimensional image of motion within the field of view based on the integration of the modulated output signals.

2. The device of claim 1, wherein the output signals are modulated such that, for substantially a first half of the modulating frequency cycle time, the output signals generated from each photodetector are integrated to added to a total integrated signal output generated from each photodetector during the modulating frequency cycle time, and for substantially a second half of the modulating frequency cycle time, the output signals generated from each photodetector are integrated to subtracted from the total integrated signal output generated from each photodetector during the modulating frequency cycle time.

3. The device of claim 1, wherein the detector array comprises a plurality of quantum dots.

4. The device of claim 1, wherein each microlens in the array of microlenses has one of an infrared pass coating or an infrared block coating thereon.

5. The device of claim 1, wherein the detector array comprises a plurality of photovoltaic photodetectors.

6. The device of claim 1, wherein the controller is configured to process the three-dimensional image to provide a three-dimensional vector pointing to a starting point of the object in motion.

7. The detection device of claim 1, wherein the image of an object in motion is further processed to extract motion vectors.

8. The device of claim 1, wherein controller is configured to:

determine whether an object in motion is captured in the three-dimensional image; and

delete the three-dimensional image when the object in motion is not captured in the three dimensional image.

9. The device of claim 1, wherein the modulating frequency has a modulating frequency cycle time that is at least 10 times smaller than the integration period.

10. The device of claim 1, wherein the detector array comprises a plurality of non-silicon based photodetectors.

11. The device of claim 1, wherein the detector array is cooled to a temperature range of between 77 Kelvin and 180 Kelvin.

12. The device of claim 1, wherein the detector array comprises a plurality of photodetectors sensitive to light bands in one of the SWIR range, MWIR range and LWIR range.

13. A method of determining a three-dimensional trajectory of motion, the method comprising:

receiving, through a lens, an image of a field of view of the lens;

creating an array of light field images based on the image, from an array of microlenses optically coupled to the lens;

generating output signals, from photodetectors of a detector array optically coupled to the array of microlenses, based on the array of light field images;

integrating, through a controller coupled to the detector array, the output signals over an integration period wherein at least a portion of the output signals are modulated at a modulating frequency having a modulating frequency cycle time that is smaller than the integration period; and

generating an image of motion in the field of view based on the integration of the modulated output signals, wherein the image displays a three dimensional trajectory of the motion.

14. The method of claim 13 comprising:

modulating the output signals such that, for substantially a first half of the modulating frequency cycle time, the output signals generated from each photodetector are integrated to added to a total integrated signal output generated from each photodetector during the modulating frequency cycle time, and for substantially a second half of the modulating frequency cycle time, the output signals generated from each photodetector are integrated to subtracted from the total integrated signal output generated from each photodetector during the modulating frequency cycle time.

15. The method of claim 13 comprising:

providing the detector array such that the photodetectors comprise a plurality of quantum dots.

16. The method of claim 13 comprising:

providing the detector array such that the photodetectors comprise a plurality of photovoltaic photodetectors.

17. The method of claim 13 comprising:

providing the detector array such that the photodetectors comprise a plurality of non-silicon based photodetectors.

18. The method of claim 13 comprising:

cooling the detector array to a temperature range of between 77 Kelvin and 180 Kelvin.

19. The method of claim 13 wherein the modulating frequency has a modulating frequency cycle time that is at least 10 times smaller than the integration period.

20. The method of claim 13 wherein the detector array comprises a plurality of photodetectors sensitive to light bands in one of the SWIR range, MWIR range and LWIR range.