Spatially-varying flicker detection

• US 9,088,727 B2
• Filed: 04/06/2011
• Issued: 07/21/2015
• Est. Priority Date: 04/06/2011
• Status: Active Grant

• Current Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Original Assignee: Pelco Incorporated (Motorola Solutions, Inc.)
• Inventors: Trumbo, Matthew L.
• Primary Examiner(s): SPINKS, ANTOINETTE T
• Application Number:
  US13/081,144
• Publication Number:
  US 20120320232A1
• Time in Patent Office: 1,567 Days
• Field of Search: 348/226.1
• US Class Current: 1/1
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/743 Bracketing, i.e. taking a series of images with varying exposure conditions
• CPC Class Codes: H04N 23/745 Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination
• CPC Class Codes: H04N 25/533 by using differing integration times for different sensor regions
• CPC Class Codes: H04N 25/583 with different integration times

Abstract

An imaging system includes: an image module configured to produce image frames at a frame rate F, the image module being configured to capture first and second portions of each of the image frames at different times; a frequency determination module, communicatively coupled to the image module, configured to determine frequency components associated with the first and second portions over a plurality, N, of the image frames; and a flicker analysis module, communicatively coupled to the frequency determination module, configured to determine whether any of the frequency components of the first and second portions are indicative of perceptible flicker.

Claims

1. An imaging system comprising:

an image module configured to produce image frames at a frame rate F, the image module being configured to capture first and second portions of each of the image frames, each of the first and second portions comprising a two-dimensional array of pixels having a plurality of horizontal pixels and a plurality of vertical pixels, the first portion being separate from the second portion, the image frames including a plurality of candidate images having different exposure durations;

a frequency determination module, communicatively coupled to the image module, configured to determine frequency components associated with the first and second portions over a plurality, N, of the image frames;

a flicker analysis module, communicatively coupled to the frequency determination module, configured to determine whether any frequency component associated with the first portion or the second portion, respectively, is indicative of perceptible flicker such that the flicker analysis module is capable of determining a frequency component indicative of perceptible flicker over less than an entire area of the image frames; and

a stitch module configured to replace the first portion of a first image frame with a first portion of a second image frame from the plurality of candidate images, the first portion of the first image frame corresponding to the first portion in the second image frame, the first portion of the first image being either underexposed or overexposed, and the first portion of the second image being properly exposed.

2. The system of claim 1 further comprising an accumulation module, communicatively coupled to the image module, configured to accumulate a plurality, N, sets of information associated with the N image frames, wherein the frequency determination module is communicatively coupled to the accumulation module and is configured to apply a Fourier transform to the N sets of information to determine the frequency components associated with the first and second portions segments of the N image frames.

3. The system of claim 2 further comprising a spatial downsampling module, communicatively coupled to the image module and the accumulation module, configured to spatially downsample each of the image frames to produce the sets of information.

4. The system of claim 3 wherein the spatial downsampling module is configured to average pixel values over each of a plurality of regions of each of the-image frames to produce the N sets of information associated with the N image frames, wherein distinct ones of the regions include the first and second portions, respectively.

5. The system of claim 1 wherein the flicker analysis module is configured to analyze at least one of the frequency components associated with the first portion of the image frames to determine whether the at least one of the frequency components has a (1) frequency corresponding to at least one flicker frequency associated with a light source frequency of concern, and (2) a magnitude with an undesired relationship relative to a flicker magnitude threshold.

6. The system of claim 5 wherein the flicker analysis module is configured to determine whether the at least one of the frequency components satisfies conditions (1) and (2) only if the at least one of the frequency components corresponds to a predominant signal associated with the first portion of the image frames.

7. The system of claim 5 wherein the at least one flicker frequency is a frequency to which at least one of 50 Hz, 60 Hz, 100 Hz, or 120 Hz aliases to when using a Fast Fourier Transform with N samples and a sampling frequency of F.

8. The system of claim 1 wherein the flicker analysis module is capable of determining that a first frequency component associated with the first portion is indicative of perceptible flicker and that a second frequency component associated with the second portion is indicative of perceptible flicker.

9. The system of claim 1 wherein the flicker analysis module is capable of determining that a first frequency component associated with the first portion is indicative of perceptible flicker and that a second frequency component associated with the second portion has no frequency component indicative of flicker.

10. The system of claim 1 wherein a size and shape of the first portion is configured to include an object having a different brightness of the first image than the second portion of the first image.

11. The imaging system of claim 1 wherein the first portion of the first image frame occupies an identical area of the first image frame as the first portion of the second image frame does in the second image frame.

12. The imaging system of claim 1 wherein the first portion of the second image frame is surrounded by the second portion of the second image frame.

13. The imaging system of claim 1 wherein the second portion of the first image frame is properly exposed and lacking of perceptible flicker.

14. A flicker detection system comprising:

an image module configured to produce image frames at a frame rate F, the image module being configured to capture first and second portions of each of the image frames, each of the first and second portions comprising a two-dimensional array of pixels having a plurality of horizontal pixels and a plurality of vertical pixels, the first portion being separate from the second portion, the image frames including a plurality of candidate images having different exposure durations;

downsampling means, communicatively coupled to the image module, for spatially downsampling the image frames to produce downsampled image frames each having third and fourth distinct regions including the first and second portions, respectively, each of the third and fourth regions corresponding to less than all of respective image frames;

frequency determination means, communicatively coupled to the downsampling means, for applying a Fourier transform to a plurality, N, of the downsampled image frames to produce separate indications of frequency components of the third and fourth portions, respectively;

flicker analysis means, communicatively coupled to the frequency determination means, for analyzing the indications of frequency components to determine whether any of the frequency components corresponds to a predominant signal in the respective region, corresponds to at least one frequency of concern, and would be perceptible to a person viewing the image frames; and

stitch means for replacing the first portion of a first image frame with a first portion of a second image frame from the plurality of candidate images, the first portion of the first image frame corresponding to the first portion in the second image frame, the first portion of the first image being either underexposed or overexposed, and the first portion of the second image being properly exposed.

15. The system of claim 14 further comprising accumulation means, communicatively coupled to the downsampling means and the frequency determination means, for accumulating the N downsampled frames.

16. The system of claim 14 wherein the downsampling means are configured to average pixel values over each of the distinct regions.

17. The system of claim 14 wherein the at least one frequency of concern is a frequency to which at least one of 50 Hz, 60 Hz, 100 Hz, or 120 Hz aliases to when using a Fast Fourier Transform with N samples and a sampling frequency of F.

18. The system of claim 14 wherein the flicker analysis means are capable of determining that a first frequency component associated with the first portion is indicative of perceptible flicker and that a second frequency component associated with the second portion is indicative of perceptible flicker.

19. The system of claim 14 wherein the flicker analysis means are capable of determining that a first frequency component associated with the first portion is indicative of perceptible flicker and that a second frequency component associated with the second portion has no frequency component indicative of flicker.

20. A computer program product residing on a non-transitory processor-readable medium and comprising instructions configured to cause a processor to:

produce image frames at a frame rate F and with first and second portions of each of the image frames being captured, each of the first and second portions comprising a two-dimensional array of pixels having a plurality of horizontal pixels and a plurality of vertical pixels, the first portion being separate from the second portion, the image frames including a plurality of candidate images having different exposure durations;

determine frequency components associated with the first and second portions over a plurality, N, of the image frames;

determine whether any frequency component associated with the first portion or the second portion, respectively, is indicative of perceptible flicker such that the instructions are configured to cause the processor to determine a frequency component indicative of perceptible flicker over less than an entire area of the image frames; and

assemble a composite image frame by adding the first portion of a first image frame to a second image frame from the plurality of candidate images in place of the first portion of the second image frame, the first portion of the second image frame being either underexposed or overexposed, and the first portion of the first image frame being properly exposed.

21. The computer program product of claim 20 further comprising instructions configured to cause the processor to:

spatially downsample each of the plurality, N, of the image frames to produce N sets of information; and

apply a Fourier transform to the N sets of information to determine the frequency components associated with the first and second portions of the N image frames.

22. The computer program product of claim 20 wherein the instructions configured to cause the processor to determine whether any of the frequency components in either of the first or second portions individually are indicative of perceptible flicker are configured to cause the processor to analyze at least one first frequency component associated with the first portion of the image frames to determine whether the first frequency component (1) has a frequency corresponding to at least one flicker frequency associated with a light source frequency of concern, (2) has a magnitude with an undesired relationship relative to a flicker magnitude threshold, and (3) corresponds to a predominant signal associated with the first-portion of the image frames.

23. A method of detecting flicker in image frames, the method comprising:

producing the image frames at a frame rate F, and with first and second portions of each of the image frames being captured, each of the first and second portions comprising a two-dimensional array of pixels having a plurality of horizontal pixels and a plurality of vertical pixels, the first portion being separate from the second portion, the image frames including a plurality of candidate images having different exposure durations;

determining a first frequency component associated with the first portion and a second frequency component, inequal to the first frequency component, associated with the second portion of the image frames;

determining whether at least one of the first frequency component associated with the first portion or the second frequency component associated with the second portion is indicative of perceptible flicker; and

replacing the first portion of a first image frame with the first portion of a second image frame from the plurality of candidate images, the first portion of the first image frame corresponding with first portion of the second image frame, the first portion of the first image being either underexposed or overexposed, and the first portion of the second image being properly exposed.

24. The method of claim 23 further comprising:

spatially downsampling each of a plurality, N, of the image frames to produce N sets of information; and

applying a Fourier transform to the N sets of information to determine the frequency components associated with the first and second portions of the N image frames.

25. The method of claim 23 wherein determining at least one of the first frequency component associated with the first portion or the second frequency component associated with the second portion is indicative of perceptible flicker includes determining whether the first frequency component (1) has a frequency corresponding to at least one flicker frequency associated with a light source frequency of concern, (2) has a magnitude with an undesired relationship relative to a flicker magnitude threshold, and (3) corresponds to a predominant signal associated with the first portion of the image frames.