System and method for processing video data to detect and eliminate flickering light sources through dynamic exposure control

• US 10,412,312 B2
• Filed: 08/07/2017
• Issued: 09/10/2019
• Est. Priority Date: 08/08/2016
• Status: Active Grant

• Current Assignee: Gentex Corporation
• Original Assignee: Gentex Corporation
• Inventors: Hilldore, Benjamin B.
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): Nguyen, Chan T
• Application Number:
  US15/670,471
• Publication Number:
  US 20180041684A1
• Time in Patent Office: 764 Days
• Field of Search: 3482261
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/745 Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination

Abstract

An image capture system is provided for a vehicle equipped with a display for displaying streamed video images of a scene proximate the vehicle. The image capture system includes a camera for capturing video images of the scene proximate the vehicle at a selected exposure time and streaming the video images, and an image processing unit for receiving the streamed video images, processing the streamed video images and supplying the streamed video images to the display. The image processing unit is configured to detect amplitude-modulated light sources in the streamed video images, and adjust the exposure time of the camera to match a multiple of a modulation frequency of the amplitude-modulated light sources decreasing a flicker intensity of the amplitude-modulated light sources in the streamed video images supplied to the display.

Claims

1. An image capture system for a vehicle equipped with a display for displaying streamed video images of a scene proximate the vehicle, the image capture system comprising:

a camera comprising a field of view for capturing video images of the scene proximate the vehicle and streaming the video images; and

an image processing unit for receiving the streamed video images, processing the streamed video images and supplying the streamed video images to the display,

wherein the image processing unit is configured to;

select a first plurality of exposure times comprising a first exposure time and at least a second exposure time;

select the first exposure time based on ambient light conditions in the field of view;

adjust the first exposure time to the second exposure time based on an amplitude-modulated light source in the streamed video images;

calculate a variance of a flicker intensity for the amplitude-modulated light source for each of the first plurality of exposure times; and

display the amplitude-modulated light source in the streamed video images supplied to the display with a reduced flicker intensity of the amplitude-modulated light sources.

2. The image capture system according to claim 1, wherein the processing unit is further configured to:

detect a modulation frequency of the amplitude modulated light source.

3. The image capture system according to claim 2,

wherein the second exposure time is approximately a multiple of a modulation frequency of the amplitude-modulated light sources.

4. The image capture system according to claim 1, wherein the processing unit is further configured to:

select the first plurality of exposure times based on the first exposure time, wherein the first plurality of exposure times comprises the second exposure time longer than the first exposure time and a third exposure time shorter than the first exposure time.

5. The image capture system according to claim 1, wherein the processing unit is further configured to:

capture the first plurality of exposure times comprising a fourth exposure time and a fifth exposure time selected based on the first exposure time.

6. The image capture system according to claim 1, wherein the processing unit is further configured to:

based on the variance of the flicker intensity of each of the first plurality of exposure times, determine whether the flicker intensity of each of the first plurality of exposure times is increasing, decreasing, or at an approximate local minimum.

7. The image capture system according to claim 6, wherein the processing unit is further configured to:

in response to the first plurality of exposure times failing to comprise an exposure time at the approximate local minimum, selecting a second plurality of exposure times based on a most improved exposure time of the first plurality of exposure times, wherein the most improved exposure time comprises the greatest decrease in the flicker intensity.

8. An image system for a vehicle, the image system comprising:

a camera for capturing video images of a scene proximate the vehicle at a selected exposure time and streaming the video images;

an image processing unit for receiving the streamed video images and processing the streamed video images; and

a display for displaying the streamed video images;

wherein the image processing unit is configured to;

detect an amplitude-modulated light source in the streamed video images;

adjust the exposure time of the camera to match the multiple of the modulation frequency by selecting a first exposure time and selecting a first plurality of exposure times based on the first exposure time;

determine a flicker strength of each of the first plurality of exposure times over a plurality of the video images;

determine a magnitude of a sampling function of the flicker strength of each of the first plurality of exposure times;

compare the magnitude of each of the first plurality of exposure times to determine if one of the plurality of exposure times is an approximate local minimum; and

adjust the exposure time of the camera to match a multiple of a modulation frequency of the amplitude-modulated light sources decreasing a flicker intensity of the amplitude-modulated light sources in the streamed video images supplied to the display.

9. The image system according to claim 8, wherein the image processing unit is further configured to adjust the exposure time of the camera to match the multiple of the modulation frequency by:

determining a beat frequency of the amplitude-modulated light sources; and

adjusting the exposure time of the camera based on the beat frequency.

10. The image system according to claim 8, wherein the image processing unit is further configured to:

in response to identifying an approximate local minimum, set the exposure time to the one of the first plurality of exposure times; and

in response to failing to identify the approximate minimum, selecting a second plurality of exposure times.

11. The image system according to claim 8, wherein the image processing unit is further configured to:

(a) adjust the exposure time of the camera to match the multiple of the modulation frequency by selecting a first exposure time;

(b) select a plurality of exposure times based on the first exposure time;

(c) determine a perceived brightness of the amplitude-modulated light source over a number of frames for each of the plurality of exposure times comprising the first exposure time and a plurality of additional exposure times;

(d) calculate a sampling function for each of the plurality of exposure times from the perceived brightness of the amplitude-modulated light source;

(e) determine whether a magnitude of the sampling function of each of the plurality of exposure times is at an approximate local minimum;

(f) in response to identifying a selected exposure time of the plurality of exposure times provides for the approximate local minimum, assign an exposure time of the camera to the selected exposure time; and

(g) in response to failing to identify one of the plurality of exposure times to be the approximate local minimum, repeat steps (a)-(g) with different exposure times until approximate local minima is identified.

12. A method of capturing video images streamed from a camera within a vehicle, the method comprising:

detecting amplitude-modulated light sources in the streamed video images;

iteratively selecting a plurality of exposure times;

calculating a sampling function of a flicker strength of each of the plurality of exposure times over a plurality of the video images;

comparing a magnitude of the sampling function for each of the plurality of exposure times to identify an approximate minimum of the sampling function; and

adjusting an exposure time of the camera to match a multiple of a modulation frequency of the amplitude-modulated light sources decreasing a flicker intensity of the amplitude-modulated light sources in the streamed video images when displayed.

13. The method according to claim 12, further comprising:

repeating the selecting of the plurality of exposure times until one of the plurality of exposure times yields the approximate minimum.

14. The method according to claim 13, wherein each iterative selection of the plurality of exposure times is based on a lowest standard deviation exposure time identified from a previous iteration of the plurality of exposure times.

15. The method according to claim 12, wherein, in order to adjust the exposure time of the camera to match a multiple of a modulation frequency of the amplitude-modulated light sources, the method further comprises:

determining a beat frequency of the amplitude-modulated light sources; and

adjusting the exposure time of the camera based on the beat frequency.

16. The method of claim 12, wherein, in order to adjust the exposure time of the camera to match a multiple of a modulation frequency of an amplitude-modulated light source, the method further comprises:

(a) selecting an exposure time;

(b) selecting a plurality of exposure times based on the first exposure time;

(c) determining a perceived brightness of the amplitude-modulated light source over a number of frames for each of a plurality of exposure times;

(d) calculating a sampling function for each of the plurality of exposure times from the perceived brightness of the amplitude-modulated light source;

(e) determining whether a magnitude of the sampling function of each of the plurality of exposure times is increasing, decreasing, or at an approximate local minimum;

(f) in response to identifying a selected exposure time of the plurality of exposure times provides for the approximate local minimum, assigning the exposure time of the camera to the selected exposure time; and

(g) in response to failing to identify one of the plurality of exposure times to be the approximate local minimum, repeating steps (a)-(g) with different exposure times until approximate local minima is identified.