Methods, systems and apparatuses for high-quality green imbalance compensation in images

• US 7,876,363 B2
• Filed: 04/19/2007
• Issued: 01/25/2011
• Est. Priority Date: 04/19/2007
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Ovsiannikov, Ilia
• Primary Examiner(s): SELBY, GEVELL V
• Application Number:
  US11/785,706
• Publication Number:
  US 20080259180A1
• Time in Patent Office: 1,377 Days
• Field of Search: 348/222.1, 348/223.1, 348/231.3, 348/241
• US Class Current: 348/223.1
• CPC Class Codes: H04N 2209/045 using mosaic colour filter
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Methods, systems and apparatuses for determining and compensating for green imbalance in imagers. An estimated local green balance correction term is limited, e.g., between an upper and lower limit. The upper and lower limits are determined from a set of parameters. The parameters are determined during a calibration procedure. The parameters may be stored for a subset of possible pixel locations and parameters for each pixel not at a location corresponding to stored data may be determined by interpolation.

Claims

1. A method of processing a plurality of pixel signals corresponding to an image comprising:

determining an estimated green balance correction term for each green pixel signal;

determining a limited green balance correction term for each green pixel signal by limiting the estimated green balance correction term based on a set of parameters; and

adjusting each green pixel signal by applying the limited green balance correction term to the green pixel signal.

2. The method of claim 1, wherein limiting the estimated green balance correction term further comprises applying at least one of an upper limit and a lower limit to the estimated green balance correction term, wherein the upper and lower limits are determined from the set of parameters.

3. The method of claim 1 wherein the set of parameters is obtained from a storage location that includes a plurality of sets of parameters, respectively associated with different ones of the plurality of green pixel signals.

4. The method of claim 1 wherein determining the estimated green balance correction term further comprises:

determining a first average of pixel signals in a pixel kernel surrounding a particular green pixel being processed and belong to a first green color channel, wherein the particular pixel belongs to the first green color channel;

determining a second average of pixel signals in a pixel kernel surrounding the particular green pixel and belong to a second green color channel; and

determining the estimated green balance correction term based on a difference between the first average and the second average.

5. The method of claim 4, wherein the estimated green balance correction term is determined as

6. The method of claim 2, wherein the set of parameters comprises a_pos, b_pos, a_neg, and b_neg and are stored in a storage location, wherein a_pos and a_neg are the slope and b_pos, and b_neg are the intercepts of functions determined during a calibration procedure and the parameters are related to at least one of the particular pixel signal, a sensor which captures the image and camera optics of a camera which captures the image,

wherein the upper and lower limits are determined by applying the set of parameters as follows:

Δ

_pos=a_pos*p_i,j+b_pos; and

Δ

_neg=a_neg*p_i,j+b_neg,

and wherein Δ

_pos is the upper limit, Δ

_neg is the lower limit and p_i,j is a particular pixel signal.

7. The method of claim 4, wherein adjusting the pixel signals further comprises applying the limited local green balance correction term to each green pixel signal as follows:

p′

_i,j=p_i,j−

|Δ

_i,j|, when the first average is greater than the second average;

p′

_i,j=p_i,j+|Δ

_i,j|, when the first average is less than the second average; and

p′

_i,j=p_i,j, when the first average is equal to the second average,

wherein p′

_i,j is the adjusted pixel signal, p_i,j is a particular pixel signal, Δ

_i,j is the limited local green balance correction term, and i and j represent a location of the particular green pixel within a pixel array.

8. The method of claim 1 wherein the set of parameters is related to at least one of an array location of a particular green pixel signal, a sensor which captures the image and camera optics of a camera which captures the image.

9. The method of claim 1, wherein the set of parameters is obtained by:

obtaining from a storage location, for each parameter, a set of available values associated with an array location nearest a particular green pixel being adjusted; and

using interpolation to determine each parameter for the particular green pixel being adjusted.

10. The method of claim 9 wherein the interpolation is a bi-linear interpolation.

11. A method of operating an imaging system, the method comprising:

acquiring a plurality of test images;

developing parameters for limiting green imbalance color correction terms from the test images; and

storing the parameters in an imager for use in correction of green color imbalance by the imager.

12. The method of claim 11, wherein the plurality of test images are acquired for at least one of a respective plurality of different color temperature illumination conditions and a respective plurality of locations relative to an imager array.

13. The method of claim 11 wherein the plurality of test images are acquired of a test object including a plurality of colors.

14. The method of claim 11, wherein developing the parameters further comprises:

determining an estimated local green balance correction term for each test image;

determining data points for the estimated local green balance correction term with respect to a pixel response for each test image;

fitting a linear function to the data points for each test image;

determining sets of possible parameters for each test image; and

choosing the appropriate set of parameters for use in correction term limiting for each test image.

15. The method of claim 14, wherein determining sets of possible parameters for each test image comprises determining a slope and an intercept of the linear function fitted to the data points for each test image.

16. The method of claim 14, wherein choosing the appropriate set of possible parameters for use in correction term limiting comprises choosing the sets of possible parameters which yield the largest positive and the largest negative green imbalance estimate.

17. The method of claim 11, further comprising storing a plurality of appropriate sets of parameters for use in correction term limiting, wherein each of the plurality of sets of parameters relates to a particular pixel location within a pixel array.

18. An imaging device comprising:

an array of pixels for capturing an image and providing pixel signals;

a storage device for storing at least one set of parameters; and

a processing circuit for processing pixel output signals produced by the array and corresponding to a captured image, the processing circuit being configured to;

determine an estimated green balance correction term for each green pixel output signal;

determine a limited green balance correction term for each green pixel output signal by limiting the estimated green balance correction term by applying at least one of an upper limit and a lower limit to the estimated green balance correction term, wherein the upper and lower limits are determined from the at least one set of parameters; and

adjust each green pixel output signal by applying the limited green balance correction term to the green pixel signal.

19. The imaging device of claim 18, wherein the set of parameters comprises a_pos, b_pos, a_neg, and b_neg and are stored in a storage location, wherein a_pos and a_neg are the slope and b_pos, and b_neg are the intercepts of functions determined during a calibration procedure and the parameters are related to at least one of the particular green pixel signal, a sensor which captures the image and camera optics of a camera which captures the image,

wherein determining the upper and lower limits comprises applying the set of parameters as follows:

Δ

_pos=a_pos*p_i,j+b_pos; and

Δ

_neg=a_neg*p_i,j+b_neg,

and wherein Δ

_pos is the upper limit and Δ

_neg is the lower limit.

20. The imaging device of claim 18, wherein determining the estimated green balance correction term further comprises:

determining a first average of pixel output signals in a pixel kernel surrounding a particular green pixel being processed and belong to a first green color channel, wherein the particular green pixel belongs to the first green color channel;

determining a second average of pixel output signals in a pixel kernel surrounding the particular green pixel and belong to a second green color channel; and

determining the estimated green balance correction term based on a difference between the first average and the second average.

21. The imaging device of claim 20, wherein adjusting the pixel output signals further comprises applying the limited green balance correction term to the green pixel output signal as follows:

p′

_i,j=p_i,j−

|Δ

_i,j|, when the first average is greater than the second average;

p′

_i,j=p_i,j+|Δ

_i,j|, when the first average is less than the second average; and

p′

_i,j=p_i,j, when the first average is equal to the second average,

wherein p′

_i,j is the adjusted pixel signal, p_ij is a particular green pixel output signal, Δ

,_ij is the limited local green balance correction term, and i and j represent a location of the particular green pixel within the array of pixels.

22. The imaging device of claim 18, wherein the set of parameters is obtained by:

obtaining from a storage location, for each parameter, a set of available values associated with an array location nearest a particular green pixel being adjusted; and

using interpolation to determine the set of parameters for the green particular pixel being adjusted.

23. A digital camera comprising:

a pixel array for capturing an image received through a lens;

a storage area for storing at, least one set of parameters; and

a pixel array processing circuit which, using the at least one set of parameters, is configured to determine a limited green balance correction term, and to adjust green pixel signals for a captured image using the limited green balance correction term, wherein the limited green balance correction term is determined by limiting an estimated green balance correction term using the at least one set of parameters, and wherein the estimated green balance correction term is limited such that the limited green balance correction term is greater than or equal to a lower limit and less than or equal to an upper limit, wherein the lower and upper limits are determined from the at least one set of parameters.

24. The digital camera of claim 23, wherein the set of parameters comprises a_pos, b_pos, a_neg, and b_neg and is stored in a storage location, wherein a_pos and a_neg are the slope and b_pos, and b_neg are the intercepts of functions determined during a calibration procedure and the parameters are related to at least one of the particular green pixel signal, a sensor which captures the image and camera optics of a camera which captures the image,

wherein the upper and lower limits are determined by applying the set of parameters as follows:

Δ

_pos=a_pos*p_i,j+b_pos; and

Δ

_neg=a_neg*p_i,j+b_neg,

wherein Δ

_pos is the upper limit, Δ

_neg is the lower limit and P_i,j is the green pixel signal.

25. A non-transitory computer readable medium having stored thereon a computer program comprising a set of instructions executed by a computer to implement a method for green imbalance correction, the method comprising the acts of:

determining an estimated green balance correction term for each green pixel signal associated with a captured image, wherein determining the estimated green balance correction term comprises;

determining a first average of pixel signals in a pixel kernel surrounding a particular green pixel being processed and belong to a first green color channel, wherein the particular pixel belongs to the first green color channel;

determining a second average of pixel signals in a pixel kernel surrounding the particular green pixel and belong to a second green color channel; and

determining the estimated green balance correction term based on a difference between the first average and the second average;

determining a limited green balance correction term for each green pixel signal by limiting the estimated green balance correction term by applying at least one of an upper limit and a lower limit to the estimated green balance correction term, wherein the upper and lower limits are determined from a set of parameters, the set of parameters being related to at least one of an array location of a particular green pixel signal, a sensor which captures the image and camera optics of a camera which captures the image, and the set of parameters being obtained by;

obtaining from a storage location, for each parameter, a set of available values associated with an array location nearest the particular green pixel being adjusted; and

using interpolation to determine the set of parameters for the green particular pixel being adjusted; and

adjusting each green pixel signal by applying the limited green balance correction term to the green pixel signal.