Pixel selective white balancing

• US 6,876,384 B1
• Filed: 04/19/2000
• Issued: 04/05/2005
• Est. Priority Date: 04/19/2000
• Status: Expired due to Term

• Current Assignee: Biomorphic Vlsi, Inc.
• Original Assignee: Biomorphic Vlsi, Inc.
• Inventors: Hubina, Thomas Eugene, Reinhart, Craig Charles
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Jerabek, Kelly L.
• Application Number:
  US09/552,217
• Time in Patent Office: 1,812 Days
• Field of Search: 3482221-2251, 348/252, 348/266, 348/655, 348/223.1, 358/516, 382/266
• US Class Current: 348/223.1
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light

Abstract

A system and method for processing data representative of a color image is disclosed. Image data represents an intensity of photoexposure of an imaging array at specific locations in the imaging array and in distinct spectral regions corresponding to color channels. A process identifies “white” regions in the color image by comparing the intensities of photoexposure of groups of associated pixels which are responsive to photon energy in different spectral regions. If the intensities of photoexposure of the pixels in the group of associated pixels are proportionally equivalent, these pixels are determined to be in a white region of the image. White balancing gain coefficients are then based upon the pixel intensity values at pixel locations in the white regions of the image.

Claims

1. A method of processing data representative of color information extracted from an array of pixels in an imaging array, the imaging array including a plurality of pixels which are responsive to photon energy in a distinct spectral region, each of the pixels being capable of outputting an intensity value which is representative of an intensity of photoexposure in the spectral region associated with the pixel over an exposure period, the method comprising:

identifying each pixel responsive to photoexposure in a first spectral region having an intensity value between a minimum intensity value and a maximum intensity value to provide a plurality of first pixels,

selecting a first pixel from the plurality of first pixels to form a selected first pixel,

selecting, for each selected first pixel, at least one pixel associated with a second spectral region to determine at least one associated second pixel for each selected first pixel and selecting at least one pixel associated with a third spectral region to determine at least one associated third pixel for each selected first pixel, and

associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel to determine a matching set therewith, the matching set including the selected first pixel, the associated second pixel, and the associated third pixel, wherein associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel includes;

determining whether the intensity value of the associated second pixel is within a first range of the intensity value of the selected first pixel, and

determining whether the intensity value of the associated third pixel is within a second range of the intensity value of the associated second pixel, and

determining a first gain coefficient for application to the intensity values of each of the pixels associated with the second spectral region based upon an accumulation of the intensity values associated with the associated second pixels and determining a second gain coefficient for application to the intensity values of each of the pixels associated with the third spectral region based upon an accumulation of the intensity values associated with the associated third pixels.

2. The method of claim 1, wherein the step of determining the first and second gain coefficients further includes:

determining the first gain coefficent as being proportional to an average intensity value of all of the selected second pixels divided by an average intensity value of each of the selected first pixels; and

determining the second gain coefficient as being proportional to an average intensity value of all of the selected third pixels divided by the average intensity value of each of the selected first pixels.

3. The method of claim 1, the method further including:

scaling each of the pixels associated with second spectral region by the first gain coefficient; and

scaling each of the pixels associated with the third spectral region by the second gain coefficient.

4. The method of claim 1, wherein each of the pixels is associated with a location on the imaging array, and wherein the selecting step further includes:

selecting the at least one second associated pixel as having the same location as the first pixel; and

selecting the at least one third associated pixel as having the same location as the first pixel.

5. The method of claim 1, wherein each of the pixels is associated with a location on the imaging array, and wherein the selecting step further includes:

selecting the at least one second associated pixel as having a first adjacent location to the location of the selected first pixel; and

selecting the at least one third associated pixel as having a second adjacent location to the location of the selected first pixel.

6. The method of claim 5, the method further including:

scaling each of the pixels associated with the second spectral region by the first gain coefficient to provide a first scaled intensity value;

scaling each of the pixels associated with the third spectral region by the second gain coefficient to provide a second scaled intensity value;

determining an intensity value of a pixel associated with the second spectral region at the location of the first pixel based upon the first scaled intensity value; and

determining an intensity value of a pixel associated with the third spectral region at the location of the first pixel based upon the second scaled intensity value.

7. In a camera, the camera having an imaging array, the imaging array including a plurality of pixels which are responsive to photon energy in a distinct spectral region, each of the pixels being capable of outputting an intensity value which is representative of an intensity of photoexposure in the spectral region associated with the pixel over an exposure period, a lens for focusing an image of an object onto the imaging array, and a processor, the improvement including:

logic for identifying all pixels responsive to photoexposure in a first spectral region having an intensity value between a minimum intensity value and a maximum intensity value to provide a plurality of first pixels;

logic for selecting for each of the first pixels at least one associated second pixel responsive to photoexposure in a second spectral region;

logic for selecting for each of the first pixels at least one associated third pixel responsive to photoexposure in a third spectral region;

logic for associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel to determine a matching set therewith, the matching set including the selected first pixel, the associated second pixel and the associated third pixel, wherein associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel includes;

determining whether the intensity value of the associated second pixel is within a first range of the intensity value of the selected first pixel; and

determining whether the intensity value of the associated third pixel is within a second range of the intensity value of the associated second pixel; and

logic for determining a first gain coefficient for application to the intensity values of each of the pixels associated with the second spectral region based upon an accumulation of the intensity values associated with the selected second pixels and for determining a second gain coefficient for application to the intensity values of each of the pixels associated with the third spectral region based upon an accumulation of the intensity values associated with the selected third pixels.

8. A computer readable medium for use in conjunction with an imaging array, the imaging array having a plurality of pixels which are responsive to photon energy in a distinct spectral region, each of the pixels being capable of outputting an intensity value which is representative of an intensity of photoexposure in the spectral region associated with the pixel over an exposure period, the computer readable medium having computer readable instructions encoded thereon for performing the following:

identifying all pixels responsive to photoexposure in a first spectral region having an intensity value between a minimum intensity value and a maximum intensity value to provide a plurality of first pixels;

for each of the first pixels;

selecting at least one pixel responsive to photoexposure in a second spectral region, the second spectral region being distinct from the first spectral region, to determine at least one associated second pixel and selecting at least one pixel responsive to photoexposure in a third spectral region, the third spectral region being distinct from the first and second spectral regions, to determine at least one associated third pixel;

associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel to determine a matching set therewith, the matching set including the selected first pixel, the associated second pixel and the associated third pixel, wherein associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel includes;

determining whether the intensity value of the associated second pixel is within a first range of the intensity value of the selected first pixel; and

determining whether the intensity value of the associated third pixel is within a second range of the intensity value of the associated second pixel; and

determining a first gain coefficient for application to the intensity values of each of the pixels associated with the second spectral region based upon an accumulation of the intensity values associated with the selected second pixels and determining a second gain coefficient for application to the intensity values of each of the pixels associated with the third spectral region based upon an accumulation of the intensity values associated with the selected third pixels.

9. An image processor for use in conjunction with an imaging array, the imaging array including a plurality of pixels which are responsive to photon energy in a distinct spectral region, each of the pixels being capable of outputting an intensity value which is representative of an intensity of photoexposure in the spectral region associated with the pixel over an exposure period, the image processor comprising:

logic for identifying all pixels responsive to photoexposure in a first spectral region having an intensity value between a minimum intensity value and a maximum intensity value to provide a plurality of first pixels;

logic for selecting for each of the first pixels at least one pixel associated responsive to photoexposure in a second spectral region distinct from the first spectral region;

logic for selecting for each of the first pixels at least one spatially pixel responsive to photoexposure in a third spectral region, the third spectral region being distinct from the first and second spectral regions, to determine at least one associated third pixel;

logic for associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel to determine a matching set therewith, the matching set including the selected first pixel, the associated second pixel and the associated third pixel, wherein associating the intensity value of the associated second pixel and the intensity value of the associated third pixel with the intensity value of the selected first pixel includes;

determining whether the intensity value of the associated second pixel is within a first range of the intensity value of the selected first pixel; and

determining whether the intensity value of the associated third pixel is within a second range of the intensity value of the associated second pixel; and

logic for determining a first gain coefficient for application to the intensity values of each of the pixels associated with the second spectral region based upon an accumulation of the intensity values associated with the selected second pixels and for determining a second gain coefficient for application to the intensity values of each of the pixels associated with the third spectral region based upon an accumulation of the intensity values associated with the selected third pixels.

10. A method of processing data representative of a color, image based upon color information extracted from pixels in an imaging array, the imaging array including a plurality of pixels, each of the plurality of pixels being responsive to photon energy in one of a plurality of distinct spectral regions, each of the spectral regions being associated with one of a plurality of color channels, each of the pixels being capable of providing data representative of an intensity of photoexposure in the spectral region and color channel associated with the pixel over an exposure period, the method comprising:

identifying white regions in the image based upon a dispersion of the intensities of photoexposure at a group of associated pixels in the imaging array, each of the associated pixels being responsive to photoexposure in a distinct one of the plurality spectral regions or color channels, wherein identifying the white regions in the image includes;

selecting a reference channel from among the plurality of color channels,

determining groups of associated pixels in the image, each of the groups including at least one reference channel pixel associated with the reference channel and at least one non-reference channel pixel associated with a color channel distinct from the reference channel,

for each group of associated pixels, associating first and second non-reference channel pixels with each group of associated pixels,

for each group of associated pixels, determining whether an intensity of photoexposure of the first non-reference channel pixel and an intensity of photoexposure of a second non-reference channel are within a predetermined range about the intensity of photoexposure of the reference channel pixel, and

determining whether the difference between the intensities of photoexposure of the first and second non-reference channel pixels is less than a predetermined difference, and

determining gain coefficients to be applied to intensities of photoexposure in the image for pixels associated with at least one of the color channels based upon an accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image.

11. The method of claim 10, the method further including:

calculating intermediate gain coefficients based upon the accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image; and

selecting the gain coefficients to be applied to the intensities of photoexposure in the image from among a plurality of sets of gain coefficients stored in memory based upon a closeness of the intermediate gain coefficients to the selected set of gain coefficients.

12. In a camera, the camera having an imaging array, the imaging array including a plurality of pixels which are responsive to photon energy in a distinct spectral region, each of the pixels being capable of outputting an intensity value which is representative of an intensity of photoexposure in the spectral region associated with the pixel over an exposure period, a lens for focusing an image of an object onto the imaging array, and a processor, the improvement including:

logic for identifying white regions in the image based upon a dispersion of the intensities of photoexposure at associated pixels, each of the associated pixels being responsive to photoexposure in a distinct one of the plurality spectral regions or color channels, wherein the logic for identifying the white regions in the image includes;

logic for selecting a reference channel from among the plurality of color channels,

logic for determining groups of associated pixels in the image, each of the groups including at least one reference channel pixel associated with the reference channel and at least one non-reference channel pixel associated with a color channel distinct from the reference channel,

for each group of associated pixels, logic for associating first and second non-reference channel pixels with each group of associated pixels,

for each group of associated pixels,

logic for determining whether an intensity of photoexposure of the first non-reference channel pixel and an intensity of photoexposure of a second non-reference channel are within a predetermined range about the intensity of photoexposure of the reference channel pixel, and

logic for determining whether the difference between the intensities of photoexposure of the first and second non-reference channel pixels is less than a predetermined difference, and

logic for determining gain coefficients to be applied to intensities of photoexposure in the image for pixels associated with at least one of the color channels based upon an accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image.

13. The camera of claim 12, the camera further comprising:

logic for calculating intermediate gain coefficients based upon the accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image; and

logic for selecting the gain coefficients to be applied to the intensities of photoexposure in the image from among a plurality of sets of gain coefficients stored in memory based upon a closeness of the intermediate gain coefficients to the selected set of gain coefficients.

14. A computer readable medium for use in conjunction with an imaging array for receiving an image of an object, the imaging array including a plurality of pixels which are responsive to photon energy in one of a plurality of distinct spectral regions, each of the spectral regions being associated with one of a plurality of color channels, each of the pixels being capable of providing data representative of an intensity of photoexposure in the spectral region and color channel associated with the pixel over an exposure period, the computer readable medium having computer readable instructions encoded thereon for performing the following:

identifying white regions in the image based upon a dispersion of the intensities of photoexposure at associated pixels, each of the associated pixels being responsive to photoexposure in a distinct one of the plurality spectral regions or color channels, wherein identifying the white regions in the image includes;

selecting a reference channel from among the plurality of color channels,

determining groups of associated pixels in the image, each of the groups including at least one reference channel pixel associated with the reference channel and at least one non-reference channel pixel associated with a color channel distinct from the reference channel,

for each group of associated pixels, associating first and second non-reference channel pixels with each group of associated pixels,

for each group of associated pixels, determining whether an intensity of photoexposure of the first non-reference channel pixel and an intensity of photoexposure of a second non-reference channel are within a predetermined range about the intensity of photoexposure of the reference channel pixel, and

determining whether the difference between the intensities of photoexposure of the first and second non-reference channel pixels is less than a predetermined difference, and

determining gain coefficients to be applied to intensities of photoexposure in the image for pixels associated with at least one of the color channels based upon an accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image.

15. The computer readable medium of claim 14, the computer readable medium further including computer readable instructions encoded thereon for:

calculating intermediate gain coefficients based upon the accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image; and

selecting the gain coefficients to be applied to the intensities of photoexposure in the image from among a plurality of sets of gain coefficients stored in memory based upon a closeness of the intermediate gain coefficients to the selected set of gain coefficients.

16. An image processor for use in conjunction with an imaging array, the imaging array including a plurality of pixels which are responsive to photon energy in one of a plurality of distinct spectral regions, each of the spectral regions being associated with one of a plurality of color channels, each of the pixels being capable of providing data representative of an intensity of photoexposure in the spectral region and color channel associated with the pixel over an exposure period, the image processor comprising:

logic for identifying white regions in the image based upon a dispersion of the intensities of photoexposure at associated pixels, each of the associated pixels being responsive to photoexposure in a distinct one of the plurality spectral regions or color channels, wherein the logic for identifying the white regions in the image includes;

logic for selecting a reference channel from among the plurality of color channels,

logic for determining groups of associated pixels in the image, each of the groups including at least one reference channel pixel associated with the reference channel and at least one non-reference channel pixel associated with a color channel distinct from the reference channel,

for each group of associated pixels, logic for associating first and second non-reference channel pixels with each group of associated pixels,

for each group of associated pixels, logic for determining whether an intensity of photoexposure of the first non-reference channel pixel and an intensity of photoexposure of a second non-reference channel are within a predetermined range about the intensity of photoexposure of the reference channel pixel, and

logic for determining whether the difference between the intensities of photoexposure of the first and second non-reference channel pixels is less than a predetermined difference, and

logic for determining gain coefficients to be applied to intensities of photoexposure in the image for pixels associated with at least one of the color channels based upon an accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image.

17. The image processor of claim 16, the image processor further including:

logic for calculating intermediate gain coefficients based upon the accumulation of the intensities of photoexposure of the pixels associated with the at least one color channel in the white regions of the image; and

logic for selecting the gain coefficients to be applied to the intensities of photoexposure in the image from among a plurality of sets of gain coefficients stored in memory based upon a closeness of the intermediate gain coefficients to the selected set of gain coefficients.