METHODS AND APPARATUS FOR AN ACTIVE PULSED 4D CAMERA FOR IMAGE ACQUISITION AND ANALYSIS

• US 20180295344A1
• Filed: 12/22/2017
• Published: 10/11/2018
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: 4D Intellectual Properties, LLC
• Original Assignee: 4D Intellectual Properties, LLC
• Inventors: Retterath, James E.
• Granted Patent: US 10,298,908 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format
• CPC Class Codes: H04N 23/671 in combination with active ranging signals, e.g. using light or sound signals emitted toward objects

Abstract

An active-pulsed four-dimensional camera system that utilizes a precisely-controlled light source produces spatial information and human-viewed or computer-analyzed images. The acquisition of four-dimensional optical information is performed at a sufficient rate to provide accurate image and spatial information for in-motion applications where the camera is in motion and/or objects being imaged, detected and classified are in motion. Embodiments allow for the reduction or removal of image-blocking conditions like fog, snow, rain, sleet, and dust from the processed images. Embodiments provide for operation in daytime or nighttime conditions and can be utilized for day or night full-motion video capture with features like shadow removal. Multi-angle image analysis is taught as a method for classifying and identifying objects and surface features based on their optical reflective characteristics.

Claims

1. (canceled)

2-26. -26. (canceled)

27. An active camera system configured to generate and present views of low-visibility objects on a vehicle display system inside a vehicle, comprising:

at least one emitter configured to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

an array of detectors configured to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

control circuitry operably coupled to the at least one emitter and the array of detectors and configured to;

control a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration; and

produce at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame; and

a processing system operably coupled to the control circuitry and the one or more frame buffers and configured to;

correlate pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

display a plurality of images of the scene on the vehicle display device inside the vehicle such that one or more low-visibility objects are displayed based on the intensity values of image pixels in the scene corresponding to the one or more low-visibility objects.

28. The system of claim 27, wherein the vehicle display device is configured to project the images on a vehicle windshield inside the vehicle.

29. The system of claim 27, wherein the vehicle display device is configured to project the images on a heads-up display inside the vehicle.

30. The system of claim 27, wherein the vehicle display device is configured to project the images on a computer video display inside the vehicle.

31. The system of claim 27, wherein the intensity values from the images are communicated to an autonomous vehicle control system of the vehicle.

32. The system of claim 27, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

33. The system of claim 27, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.

34. The system of claim 27, wherein each detector in the array of detectors has an individual field of view that is a different subset of the field of view of the array of detectors.

36. The method of claim 34, wherein the vehicle display device is configured to display the image of the scene by projecting the images on a vehicle windshield inside the vehicle.

37. The method of claim 34, wherein the vehicle display device is configured to display the images of the scene on a heads-up display inside the vehicle.

38. The method of claim 34, wherein the vehicle display device is configured to display the images of the scene on a computer video display inside the vehicle.

39. The method of claim 34, further comprising delivering the intensity values from the images to an autonomous vehicle control system of the vehicle.

40. The method of claim 34, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

41. The method of claim 34, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.

43. The method of claim 41, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

44. The method of claim 41, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.

45. The system of claim 41, wherein each detector in the array of detectors has an individual field of view that is a different subset of the field of view of the array of detectors

47. The method of claim 41, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

48. The method of claim 41, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.

35. A method of generating and displaying images of low-visibility objects on a vehicle display system inside a vehicle, comprising:

causing at least one emitter to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

causing an array of detectors to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

controlling a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration;

producing at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame; and

utilizing a processing system operably coupled to the one or more frame buffers to;

correlate pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

display a plurality of images of the scene on the vehicle display device inside the vehicle such that one or more low-visibility objects are displayed based on the intensity values of image pixels in the scene corresponding to the one or more low-visibility objects.

42. An active camera system configured to generate image data representing low-visibility objects viewed from a vehicle, comprising:

at least one emitter configured to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

an array of detectors configured to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

means for controlling a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration;

means for producing at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame;

means for correlating pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

means for delivering the image data of the scene to an autonomous vehicle control system within the vehicle.

46. An active camera system configured to generate image data representing low-visibility objects viewed from a vehicle, comprising:

at least one emitter configured to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

an array of detectors configured to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

control circuitry operably coupled to the at least one emitter and the array of detectors and configured to;

control a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration; and

produce at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame; and

a processing system operably coupled to the control circuitry and the one or more frame buffers and configured to;

correlate pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

deliver the image data to an autonomous vehicle control system within the vehicle such that one or more low-visibility objects are computer-analyzed based on the intensity values of image pixels in the scene corresponding to the one or more low-visibility objects.