Automatic white balance method and apparatus

• US 20050286097A1
• Filed: 01/25/2005
• Published: 12/29/2005
• Est. Priority Date: 06/25/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chiu, Andrew Chinchuan, Kandhadai, Ananthapadmanabhan, Hung, Szepo
• Granted Patent: US 7,728,880 B2
• Time in Patent Office: Days
• US Class Current: 358/509
• CPC Class Codes: H04N 1/608 within the L, C1, C2 colour signals
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

Automatic white balance of captured images can be performed based on a gray world assumption. Initially, a flat field gray image is captured for one or more reference illuminations. The statistics of the captured gray image are determined and stored for each reference illumination during a calibration process. For each subsequent captured image, the image is filtered to determine a subset of gray pixels. The gray pixels are further divided into a one or more gray clusters. The average weight of the one or more gray clusters is determined and a distance from the average weights to the reference illuminants is determined. An estimate of the illuminant is determined depending on the distances. White balance gains are applied to the image based on the estimated illuminant.

Claims

1. A method of performing Automatic White Balance (AWB), the method comprising:

receiving a captured image;

filtering the captured image to select gray pixels;

determining a distance in a coordinate grid from each of a plurality of reference illuminant points to a location in the coordinate grid determined at least in part on the gray pixels,

determining an illuminant based in part on the distances; and

applying a white balance gain to the captured image based in part on the illuminant.

2. The method of claim 1, wherein receiving the captured image comprises receiving a digital image file.

3. The method of claim 1, wherein receiving the captured image comprises receiving a digital image captured by an optical sensor.

4. The method of claim 1, wherein filtering the captured image comprises:

selecting a first subset of pixels from the captured image having a luminance within a predetermined luminance range;

selecting a second subset of pixels from the first subset of pixels, the second subset of pixels having a Cb chrominance component within a predetermined range; and

selecting a third subset of pixels from the second subset of pixels, the third subset of pixels having a Cr chrominance component within a predetermined range.

5. The method of claim 1, wherein filtering the captured image comprises:

selecting a first subset of pixels from the captured image having a luminance within a predetermined luminance range; and

selecting a second subset of pixels from the first subset, the second subset of pixels contained within a predetermined rectangular region defined in Cb—

Cr domain.

6. The method of claim 1, wherein determining the distance to a first reference illuminant point comprises:

determining a plurality of distances corresponding to a distance from the first reference illuminant point to each gray pixel; and

summing the plurality of distances.

7. The method of claim 1, wherein determining the distance to a first reference illuminant comprises:

determining a location of each of the gray pixels in the coordinate grid;

determining a center of gravity based on the location of the gray pixels in the coordinate grid; and

determining a distance from the center of gravity to each of the reference illuminant points.

8. The method of claim 1, wherein determining the illuminant comprises determining an illuminant corresponding to the reference illuminant point having a least distance.

9. A method of performing Automatic White Balance (AWB), the method comprising:

receiving a captured image in RGB format;

filtering the captured image to select gray pixels;

determining R/G and B/G ratios for each of the gray pixels in the captured image;

partitioning the gray pixels into a plurality of gray clusters;

quantizing the R/G and B/G ratios into a predetermined coordinate grid;

determining a center in the coordinate grid for each of the plurality of gray clusters;

determining a distance in the coordinate grid from each of the centers to each of a plurality of reference illuminant points;

determining an estimated illuminant for each cluster based in part on the distance to each of the centers;

determining a scene illuminant based in part on the estimated illuminant for each of the plurality of gray clusters; and

applying a white balance gain to the captured image based in part on the scene illuminant.

10. The method of claim 9, wherein filtering the captured image to select gray pixels comprises:

transforming the captured image to a YCrCb format;

excluding pixels from the captured image having Y values outside of a predetermined Y range; and

excluding pixels from the captured image having Cb and Cr values outside of a predetermined Cb and Cr relationship.

11. The method of claim 9, wherein partitioning the gray pixels comprises partitioning each of the gray pixels into one of a plurality of concentric clusters.

12. The method of claim 9, wherein quantizing the R/G and B/G ratios comprises:

dividing the captured image into a plurality of regions; and

determining an average R/G ratio and an average B/G ratio for each of the plurality of regions based in part on a subset of gray pixels within the region.

13. The method of claim 9, wherein determining the distance comprises determining a distance to a line joining two of the plurality of reference illuminant points.

14. The method of claim 9, wherein determining the distance comprises determining a Euclidean distance.

15. The method of claim 9, wherein determining the distance comprises determining a Manhattan distance.

16. The method of claim 9, wherein determining the estimated illuminant comprises determining an illuminant corresponding to a least distance.

17. The method of claim 9, wherein determining the scene illuminant comprises:

comparing a first estimated illuminant corresponding to a first of the plurality of gray clusters to a second estimated illuminant corresponding to a second of the plurality of gray clusters; and

determining the scene illuminant to be the first estimated illuminant if the first and second estimated illuminants are the same.

18. The method of claim 9, wherein determining the scene illuminant comprises:

determining a first estimated illuminant corresponding to a first daylight illuminant;

determining a second estimated illuminant corresponding to a second daylight illuminant; and

determining the scene illuminant as an average of the first and second daylight illuminants.

19. One or more storage devices configured to store one or more processor usable instructions, when executed by one or more processors, performing the method comprising:

receiving a captured image in RGB format;

filtering the captured image to select gray pixels;

determining R/G and B/G ratios for each of the gray pixels in the captured image;

partitioning the gray pixels into a plurality of gray clusters;

quantizing the R/G and B/G ratios into a predetermined coordinate grid;

determining a center in the coordinate grid for each of the plurality of gray clusters;

determining a distance in the coordinate grid from each of the centers to each of a plurality of reference illuminant points;

determining an estimated illuminant for each cluster based in part on the distance to each of the centers;

determining a scene illuminant based in part on the estimated illuminant for each of the plurality of gray clusters; and

applying a white balance gain to the captured image based in part on the scene illuminant.

20. An apparatus for performing Automatic White Balance (AWB), the apparatus comprising:

a gray filter configured to receive a captured image and select gray regions from the captured image;

a grid converter coupled to the gray filter and configured to transform the gray regions to a predetermined coordinate grid;

a distance module coupled to the grid converter and configured to determine distances in the predetermined coordinate grid from one or more locations based in part on the gray regions to each of a plurality of reference illuminant points;

an illuminant estimator coupled to the distance module and configured to determine an illuminant based in part on the distances; and

a white balance gain module coupled to the illuminant estimator and configured to apply a white balance gain to the captured image;

the white balance gain based in part on the illuminant.

21. The apparatus of claim 20, further comprising a cluster filter coupled to the grid converter and configured to partition the gray regions into a plurality of gray clusters.

22. The apparatus of claim 20, further comprising an illuminant reference module coupled to the distance module and configured to store locations of the plurality of reference illuminant points.

23. The apparatus of claim 20, wherein each of the gray regions comprises one of a plurality of substantially equal sized regions from the captured image.

24. The apparatus of claim 20, wherein each of the gray regions comprises a pixel from the captured image.

25. The apparatus of claim 20, wherein each of the gray regions comprises a predetermined number of pixels from the captured image.

26. The apparatus of claim 20, wherein the gray filter is configured to select the gray regions based on a predetermined set of inequalities.

27. The apparatus of claim 20, wherein the gray filter is configured to select the gray regions having a luminance within a predetermined range.

28. The apparatus of claim 20, wherein the grid converter is configured to transform the gray regions to an R/G, B/G coordinate space.

29. The apparatus of claim 20, wherein each of the one or more locations comprises a center of gravity of a subset of the gray regions.

30. The apparatus of claim 20, wherein each of the one or more locations comprises a location of one of the gray regions in a R/G, B/G coordinate space.

31. The apparatus of claim 20, wherein at least one of the distances comprises a distance from a center of gravity of a subset of gray regions to a reference illuminant point.

32. The apparatus of claim 20, wherein at least one of the distances comprises a distance from a center of gravity of a subset of gray regions to a line joining two of the plurality of reference illuminant points.

33. The apparatus of claim 20, wherein the plurality of reference illuminant points comprises an incandescent reference illuminant point.

34. The apparatus of claim 20, wherein the plurality of reference illuminant points comprises a fluorescent reference illuminant point.

35. The apparatus of claim 20, wherein the plurality of reference illuminant points comprises a daylight reference illuminant point.

36. An apparatus for performing Automatic White Balance (AWB), the apparatus comprising:

means for receiving a captured image;

means for filtering the captured image to select gray pixels;

means for determining a distance in a coordinate grid from each of a plurality of reference illuminant points to a location in the coordinate grid determined at least in part on the gray pixels, means for determining an illuminant based in part on the distances; and

means for applying a white balance gain to the captured image based in part on the illuminant.

37. The apparatus of claim 36, further comprising means for generating the captured image.

38. The apparatus of claim 36, wherein the means for filtering comprises:

means for selecting a first subset of pixels from the captured image having a luminance within a predetermined luminance range;

means for selecting a second subset of pixels from the first subset of pixels, the second subset of pixels having a Cb chrominance component within a predetermined range; and

means for selecting a third subset of pixels from the second subset of pixels, the third subset of pixels having a Cr chrominance component within a predetermined range.

39. The apparatus of claim 36, wherein the means for filtering comprises:

means for transforming the captured image to a YCrCb format;

means for excluding pixels from the captured image having Y values outside of a predetermined Y range; and

means for excluding pixels from the captured image having Cb and Cr values outside of a predetermined Cb and Cr relationship.

40. The apparatus of claim 36, wherein the means for determining the distance to the plurality of reference illuminant points comprises:

means for determining a location of each of the gray pixels in the coordinate grid;

means for determining a center of gravity based on locations of a subset of the gray pixels in the coordinate grid; and

means for determining a distance from the center of gravity to each of the plurality of reference illuminant points.