Vehicle display system for low visibility objects and adverse environmental conditions

• US 10,298,908 B2
• Filed: 12/22/2017
• Issued: 05/21/2019
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: 4D Intellectual Properties, LLC
• Original Assignee: 4D Intellectual Properties, LLC
• Inventors: Retterath, James E.
• Primary Examiner(s): Patel, Jayanti K
• Assistant Examiner(s): Habib, Irfan
• Application Number:
  US15/853,222
• Publication Number:
  US 20180295344A1
• Time in Patent Office: 515 Days
• CPC Class Codes: G01S 17/18 wherein range gates are used
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4863 Detector arrays, e.g. charge-transfer gates
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format
• CPC Class Codes: H04N 23/671 in combination with active ranging signals, e.g. using light or sound signals emitted toward objects

Abstract

A display system for use in a vehicle is disclosed including a camera configured to capture frames corresponding to a field of view. The camera is in communication with a processing unit configured to receive data representative of objects within the captured frames from the camera. A display is in communication with the processing unit which is configured to display objects based on the data representative of the captured frames received by the processing unit. Multi-frame capture is utilized with differential inter-frame illumination to provide low-visibility object detection and display. Adverse environmental conditions whereby image display is enhanced include fog, rain, snow, dust, nighttime, and high-glare scenarios.

Claims

1. An active camera system configured to generate and present views of low-visibility objects on a vehicle display system inside a vehicle, comprising:

at least one emitter configured to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

an array of detectors configured to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

control circuitry operably coupled to the at least one emitter and the array of detectors and configured to;

control a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration; and

produce at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame; and

a processing system operably coupled to the control circuitry and the one or more frame buffers and configured to;

correlate pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

display a plurality of images of the scene on the vehicle display device inside the vehicle such that one or more low-visibility objects are displayed based on the intensity values of image pixels in the scene corresponding to the one or more low-visibility objects.

2. The system of claim 1, wherein the vehicle display device is configured to project the images on a vehicle windshield inside the vehicle.

3. The system of claim 1, wherein the vehicle display device is configured to project the images on a heads-up display inside the vehicle.

4. The system of claim 1, wherein the vehicle display device is configured to project the images on a computer video display inside the vehicle.

5. The system of claim 1, wherein the intensity values from the images are communicated to an autonomous vehicle control system of the vehicle.

6. The system of claim 1, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

7. The system of claim 1, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.

8. The system of claim 1, wherein each detector in the array of detectors has an individual field of view that is a different subset of the field of view of the array of detectors.

10. The method of claim 8, wherein the vehicle display device is configured to display the image of the scene by projecting the images on a vehicle windshield inside the vehicle.

11. The method of claim 8, wherein the vehicle display device is configured to display the images of the scene on a heads-up display inside the vehicle.

12. The method of claim 8, wherein the vehicle display device is configured to display the images of the scene on a computer video display inside the vehicle.

13. The method of claim 8, further comprising delivering the intensity values from the images to an autonomous vehicle control system of the vehicle.

14. The method of claim 8, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

15. The method of claim 8, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.

9. A method of generating and displaying images of low-visibility objects on a vehicle display system inside a vehicle, comprising:

causing at least one emitter to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

causing an array of detectors to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

controlling a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration;

producing at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame; and

utilizing a processing system operably coupled to the one or more frame buffers to;

correlate pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

display a plurality of images of the scene on the vehicle display device inside the vehicle such that one or more low-visibility objects are displayed based on the intensity values of image pixels in the scene corresponding to the one or more low-visibility objects.

16. An active camera system configured to generate image data representing low-visibility objects viewed from a vehicle, comprising:

at least one emitter configured to generate and emit light within a defined frequency range throughout a field of view of a scene that is in-motion relative to the vehicle;

an array of detectors configured to receive light within the defined frequency range for a field of view of the array, the field of view of the scene defined by the field of view of the array of detectors;

control circuitry operably coupled to the at least one emitter and the array of detectors and configured to;

control a start time and a cycle duration of a detector integration for the array of detectors for each of the two or more frames, a start time of an emitter pulse sequence and an amount of timing overlap between a cycle duration of the emitter pulse sequence and the cycle duration of the detector integration; and

produce at least two frames of image data of the scene to be stored in one or more frame buffers for which the amount of timing overlap between the emitter pulse sequence and the detector integration is different for each frame; and

a processing system operably coupled to the control circuitry and the one or more frame buffers and configured to;

correlate pixels for each of the at least two frames such that intensity values from common points in the scene are assigned to corresponding image pixels in an image of the scene; and

deliver the image data to an autonomous vehicle control system within the vehicle such that one or more low-visibility objects are computer-analyzed based on the intensity values of image pixels in the scene corresponding to the one or more low-visibility objects.

17. The method of claim 16, wherein the at least one emitter is inside the vehicle and emits light within the defined frequency range through one of a windshield or a window of the vehicle.

18. The method of claim 16, wherein the array of detectors is inside the vehicle and receives light within the defined frequency range through one of a windshield or a window of the vehicle.