METHODS AND APPARATUS FOR AN ACTIVE PULSED 4D CAMERA FOR IMAGE ACQUISITION AND ANALYSIS

• US 20190058867A1
• Filed: 10/22/2018
• Published: 02/21/2019
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: 4D Intellectual Properties, LLC
• Original Assignee: 4D Intellectual Properties, LLC
• Inventors: Retterath, James E.
• Granted Patent: US 10,382,742 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format
• CPC Class Codes: H04N 23/671 in combination with active ranging signals, e.g. using light or sound signals emitted toward objects

Abstract

An active-pulsed four-dimensional camera system that utilizes a precisely-controlled light source produces spatial information and human-viewed or computer-analyzed images. The acquisition of four-dimensional optical information is performed at a sufficient rate to provide accurate image and spatial information for in-motion applications where the camera is in motion and/or objects being imaged, detected and classified are in motion. Embodiments allow for the reduction or removal of image-blocking conditions like fog, snow, rain, sleet, and dust from the processed images. Embodiments provide for operation in daytime or nighttime conditions and can be utilized for day or night full-motion video capture with features like shadow removal. Multi-angle image analysis is taught as a method for classifying and identifying objects and surface features based on their optical reflective characteristics.

Claims

1-25. -25. (canceled)

26. An active-pulsed four-dimensional (4D) camera system for acquiring information from a scene comprising:

at least one emitter configured to emit K successive cycles of a plurality of active light pulses throughout the scene, wherein a time between a start of one of the K successive cycles and a start of a next one of the K successive cycles is varied throughout the K successive cycles;

an array of detectors configured to receive reflected light related to the active light pulses emitted in each of the K successive cycles, wherein a time between the start of emitting the light pulses and a start of receiving reflected light is varied in each of the K successive cycles; and

a plurality of buffers configured to store digital information related to the reflected light received by the array of detectors; and

a processing system operably coupled to the plurality of buffers and configured to;

analyze the digital information to determine distance information and color information for the scene based at least in part on the time between the start of one of the K successive cycles and the start of the next one of the K successive cycles being varied throughout the K successive cycles, and

construct a digital representation of the scene based on the determined distance information and the determined color information.

27. An active camera system configured to generate a lighting-invariant image of a scene comprising:

at least one emitter configured to emit an active light pulse toward the scene;

an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, each detector in the array of detectors configured to produce a linearized response to a number of incident photons of light;

control circuitry operably coupled to the at least one emitter and the array of detectors and configured to store at least two successive frames of data as frame pixels in one or more frame buffers, wherein each frame pixel is based on the linearized response from a corresponding one or more of the detectors in the array of detectors, wherein a duration of a capture cycle of the array of detectors is constant for the at least two successive frames but an intensity of the active light pulse emitted during each capture cycle is different for the at least two successive frames; and

a processing system operably coupled to the control circuitry and the one or more frame buffers to generate the lighting-invariant image of the scene, the processing system configured to;

analyze the at least two successive frames of data to determine a minimum intensity value due to ambient light in the scene and a maximum intensity value for frame pixels; and

generate the lighting-invariant image based on a difference between the minimum intensity values and the maximum intensity values of the frame pixels such that the image is based on a reflected portion of the frame data resulting from the active light pulse.

28. An image processing system that analyzes a series of images of a scene to assess an optical property of an object in the scene comprising:

an active camera system configured to capture and store at least two successive frames of data of the scene to be used to generate each image of the series of images, wherein each frame is captured during a capture cycle in one or more frame buffers;

a processing system operably coupled to the active camera system to generate the series of images of the scene, the processing system configured to;

analyze the at least two successive frames of data to determine a minimum intensity value due to ambient light in the scene and a maximum intensity value of image pixels corresponding to the scene and generate an image of the series of images based on a difference between the minimum intensity values and the maximum intensity values of the image pixels;

identify at least one object in the series of images that has a surface that is differentiable from other objects such that a common point on the surface of the object can be identified and analyzed in the series of images captured from different distances of the active camera system to the object and different angular orientations of the active camera system relative to the object;

analyze the series of images that include the common point to determine a set of characteristic values of the image pixels corresponding to the common point, including;

a reflected intensity value that is based on the maximum intensity value relative to the minimum intensity value;

a distance from the active camera system to the common point;

a normal vector for the common point; and

an angular orientation between an optical path of the image pixel and the normal vector;

create an optical profile for the common point based on the set of characteristic values of the image pixels corresponding to the common point;

compare the optical profile for the common point to a set of predefined characteristic profiles of optical properties of objects; and

identify the object based on the characteristic profile that best matches the optical profile for at least one common point of the object.

29. An active camera system configured to generate an image of a scene through a signal-attenuating environment comprising:

at least one emitter configured to emit an active light pulse toward the scene;

an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, each detector in the array of detectors configured to produce a linearized response to a number of incident photons of light;

control circuitry operably coupled to the at least one emitter and the array of detectors and configured to store at least five successive frames of data as frame pixels in one or more frame buffers, wherein each frame pixel is based on the linearized response from a corresponding one or more of the detectors in the array of detectors, wherein a start of the active light pulse is before a start of a capture cycle for each frame, wherein a duration between the start of the active light pulse and the start of the capture cycle is different for each of the at least five successive frames; and

a processing system operably coupled to the control circuitry and the one or more frame buffers to generate the image of the scene, the processing system configured to;

analyze the at least five successive frames of data to determine a minimum intensity value and a maximum intensity value corresponding to each image pixel of the image; and

generate an intensity value for each image pixel based on a difference between the minimum intensity value and the maximum intensity value of the at least five frame pixels corresponding to the image pixel to reduce an impact of the signal-attenuating environment on the image.