Apparatus and method for detecting and compensating for illuminant intensity changes within an image

• US 7,471,315 B2
• Filed: 03/14/2003
• Issued: 12/30/2008
• Est. Priority Date: 03/14/2003
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Baer, Richard L., Poplin, Dwight, Silsby, Christopher
• Primary Examiner(s): Ho; Tuan V.
• Application Number:
  US10/388,813
• Publication Number:
  US 20040179114A1
• Time in Patent Office: 2,118 Days
• Field of Search: 348/226.1, 348/241, 348/370
• US Class Current: 348/226.1
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/745 Detection of flicker frequency or suppression of flicker wherein the flicker is caused by illumination, e.g. due to fluorescent tube illumination or pulsed LED illumination

Abstract

An apparatus such as a digital camera includes an image sensor array adapted to capture a scene into electrical values and a reference detector proximal to the sensor array for detecting illuminant intensities. The reference detector at least partially surrounds the image sensor array. The reference detector is read multiple times during the frame period in which the image sensor array captures a scene to detect illuminant intensities during the same frame period. Using the illuminant intensities, the phase and the amplitude of the flicker of the illuminant are extracted. Using the phase and the amplitude parameters, a flicker correction signal is synthesized. The flicker correction signal is used to correct the captured image data to reduce or eliminate adverse effects of flicker on the captured image.

Claims

1. An apparatus comprising:

an image sensor array adapted to capture a scene into electrical values, a border of the image sensor array defining a plurality of sides thereof;

a reference detector proximal to said sensor array for detecting an illuminant intensity, the reference detector being disposed to surround at least a portion of each of two adjacent sides of the image sensor array; and

a processor (1) receiving electrical values corresponding to the detected illuminant intensity from the reference detector, (2) computing at least a phase of the illuminant intensity using the electrical values from the reference detector and (3) controlling the generation of a flicker correction signal having a phase matching the computed phase of the illuminant intensity.

2. The apparatus recited in claim 1 wherein said apparatus is a digital camera.

3. The apparatus recited in claim 1 wherein said image sensor array comprises CMOS image sensors.

4. The apparatus recited in claim 1 wherein said reference detector is adjacent to at least two sides of said image sensor array.

5. The apparatus recited in claim 1 wherein said reference detector surrounds said image sensor array.

6. The apparatus recited in claim 1 wherein said image sensor array has a field of view and said reference detector has a field of view similar to said field of view of said image sensor array.

23. The apparatus recited in claim 1, wherein the processor further computes an amplitude of the illuminant intensity using the readings from the reference detector such that the computation of the amplitude includes correcting the amplitude of the illuminant intensity for unvarying components thereof to detect changes in the illuminant intensity.

7. A method of detecting illuminant intensity changes, said method comprising:

a. resetting a reference detector;

b. reading a set of sensors of an image sensor array;

c. reading the reference detector;

d. repeating steps a through c for each set of sensors of said image sensor array; and

computing an amplitude and a phase of an illuminant intensity using the readings from the reference detector;

wherein the computing of the amplitude includes correcting the amplitude of the illuminant intensity for unvarying components thereof to detect changes in the illuminant intensity.

8. The method recited in claim 7 wherein the set of sensors comprises a multiple row of sensors.

9. The method recited in claim 7 wherein said reference detector is adjacent to said image sensor array.

10. A method of processing a digital image, the method comprising:

detecting illuminant intensities during a frame period of the digital image, the illuminant intensities converted to digital values, the illuminant intensities having flicker at a flicker frequency;

extracting flicker correction parameters from the illuminant intensities, the flicker correction parameters include amplitude and phase components of the illuminant intensity, the amplitude component of the illuminant intensity being corrected for an unvarying component of the illuminant intensity;

synthesizing a flicker correction signal from the flicker correction parameters; and

applying the flicker correction signal to the digital image.

11. The method recited in claim 10 wherein said step of detecting illuminant intensity values comprises:

a. resetting a reference detector;

b. reading a row of sensors of an image sensor array;

c. reading the reference detector; and

d. repeating steps a through c for each of the rows of said image sensor array.

12. The method recited in claim 10 wherein said step of extracting flicker correction parameters comprises performing sine and cosine transforms on the illuminant intensity values.

13. The method recited in claim 12 wherein the sine and cosine transforms are obtained by integrating product of the illuminant intensity values with sine and cosine waves at the flicker frequency over an interval of a number of flicker periods.

14. The method recited in claim 13 further comprising applying apodization function to the integrand to improve accuracy.

15. The method recited in claim 10 wherein said step of extracting flicker correction parameters comprises performing IQ demodulation.

16. The method recited in claim 10 wherein said step of applying flicker correction to the digital image comprises multiplying the illuminant intensity values by the flicker correction signal.

17. The method recited in claim 16 wherein said step of applying flicker correction is skipped for saturated pixels of the illuminant intensity values.

18. An apparatus comprising:

an image sensor array adapted to capture a scene into electrical values within an image capture period, a border of the image sensor array defining a plurality of sides thereof;

a reference detector adapted to detect an illuminant intensity within the image capture period, the reference detector being disposed to surround at least a portion of each of two adjacent sides of the image sensor array;

a processor connected to said image sensor array and the reference detector and adapted to process the electrical values from the image sensor array and to compute at least a phase of the illuminant intensity using readings from the reference detector to generate a flicker correction signal having a phase matching the computed phase of the illuminant intensity; and

memory connected to said processor and adapted to store the electrical values.

19. The apparatus recited in claim 18 further comprising an extractor circuit.

20. The apparatus recited in claim 19 wherein said extractor circuit comprises an IQ demodulator.

21. The apparatus recited in claim 18 further comprising a correction signal synthesis circuit.

22. The apparatus recited in claim 21 wherein said correction signal synthesis circuit comprises a direct digital synthesizer.

24. A method of detecting illuminant intensity changes, said method comprising:

resetting a reference detector;

reading a set of sensors of an image sensor array;

reading the reference detector;

repeating the resetting of the reference detector and the readings of the set of sensors

and the reference detector for each set of sensors of said image sensor array;

computing at least a phase of an illuminant intensity using the readings from the reference detector; and

generating a flicker correction signal having a phase that matches the computed phase of the illuminant intensity.