Processing color and panchromatic pixels

• US 20070024879A1
• Filed: 07/28/2005
• Published: 02/01/2007
• Est. Priority Date: 07/28/2005
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Eastman Kodak Company
• Inventors: Hamilton, John, Compton, John
• Granted Patent: US 8,274,715 B2
• Time in Patent Office: Days
• US Class Current: 358/1.900
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements

Abstract

A method for forming a final digital color image includes capturing an image using an image sensor having panchromatic pixels and color pixels corresponding to at least two color photoresponses; providing from the captured image a digital panchromatic image and an intermediate digital color image; and using the digital panchromatic image and the intermediate digital color image to provide the final digital color image.

Claims

1. A method for forming a final digital color image, comprising:

a) capturing an image using an image sensor having panchromatic pixels and color pixels corresponding to at least two color photoresponses;

b) providing from the captured image a digital panchromatic image and an intermediate digital color image; and

c) using the digital panchromatic image and the intermediate digital color image to provide the final digital color image.

2. The method of claim 1, wherein the pixel pattern of the image sensor of element a) has a minimal repeating unit of size four by four or greater.

3. The method of claim 2, wherein the minimal repeating unit is further subdivided into cells each of which has at least two panchromatic pixels and at least two color pixels having the same spectral photoresponse.

4. The method of claim 2, wherein the pixel pattern of the image sensor of element a) has a minimal repeating unit of size eight by eight or greater.

5. A method of claim 4, wherein the minimal repeating unit is further subdivided into cells each of which has at least two panchromatic pixels and at least two color pixels having the same spectral photoresponse.

6. A method for processing a color image captured by an image sensor to permit the effective reproduction of a captured color image under different lighting conditions, comprising:

a.) capturing an image represented by signals produced by a two-dimensional array of pixels having first and second groups of pixels, wherein the first group of pixels have a narrower spectral photoresponse than the second group of pixels and wherein the first group of pixels contains pixels with differing spectral photoresponses that correspond to a set of at least two colors, the image array arranged into cells each of which has at least two pixels of the first group having the same spectral photoresponse,

b.) using the signals for the pixels in each cell corresponding to like colored pixels from the first group of pixels to produce at least one first signal for each cell,

c.) using the set of first signals for each cell and the signals for each pixel in the second group of pixels to produce an adjusted color digital image that compensates for different lighting conditions.

7. The method of claim 6, wherein element c) includes:

i.) using the signals for pixels from the second group of pixels to produce a second signal for each cell,

ii.) using the first and second signals from each cell and first and second signals from at least one adjacent cell to produce a set of color interpolated signals for each cell, and

iii.) using the set of color interpolated signals for each cell and the signals for each pixel in the second group of pixels to produce an adjusted color digital image that compensates for different lighting conditions.

8. The method of claim 7, wherein the signals for pixels in each of the first and second groups are analog signals and wherein these analog signals are used in elements b.) and c i.) to produce the first and second signals for each cell.

9. The method of claim 8, wherein the analog signals are electrical charges and the first and second signals for each cell are produced by adding electrical charges together.

10. The method of claim 8, wherein the analog signals produce voltages and the first and second signals for each cell are produced by adding voltages together.

11. The method of claim 7, wherein the first and second signals are digitized.

12. The method of claim 11, wherein element c ii) includes using color differences to provide color interpolation, and element c iii) includes upsampling color differences.

13. A method for processing a color image captured by an image sensor to permit the effective reproduction of a captured color image under different lighting conditions, comprising:

a.) using an image sensor to capture an image represented by signals wherein the image sensor is a two-dimensional array of pixels having first and second groups of pixels that produce signals, wherein the first group of pixels have a narrower spectral photoresponse than the second group of pixels and wherein the first group of pixels contains pixels with differing spectral photoresponses that correspond to a set of at least two color filters and the second group of pixels defining a high-resolution panchromatic image,

b.) combining first signals from like color pixels from the first group of pixels in each cell to form a low-resolution sparse color image,

c.) using second signals from the pixels in the second group of pixels in the high-resolution panchromatic image to interpolate pixel values corresponding to pixels in the first group of pixels and to provide a panchromatic interpolated high-resolution image,

d.) using the low-resolution sparse color image and the panchromatic interpolated high-resolution image to produce a high-resolution color image.

14. The method of claim 13, wherein element b) includes summing the first signals from like color pixels.

15. The method of claim 13, wherein element d) includes summing pixel values from the high-resolution interpolated panchromatic image to produce a low-resolution panchromatic image;

using the low-resolution panchromatic image and the low-resolution color filter array image to produce a high-resolution color difference image; and

using the high-resolution color difference image and the high-resolution panchromatic interpolated image to produce a high-resolution color image.

16. The method of claim 14 wherein element d) includes summing pixel values from the high-resolution interpolated panchromatic image to produce a low-resolution panchromatic image;

using the low-resolution panchromatic image and the low-resolution color filter array image to produce a high-resolution color difference image; and

using the high-resolution color difference image and the high-resolution panchromatic interpolated image to produce a high-resolution color image.

17. A method for processing a color image captured by an image sensor to permit the effective reproduction of a captured color image under different lighting conditions, comprising:

a.) capturing an image represented by signals produced by a two-dimensional array of pixels having first and second groups of pixels, wherein pixels from the first group of pixels have narrower spectral photoresponses than pixels from the second group of pixels and wherein the first group of pixels has individual pixels that have spectral photoresponses that correspond to a set of at least two colors, the placement of the first and second groups of pixels defining a pattern that has a minimal repeating unit including at least twelve pixels, the minimal repeating unit having a plurality of cells wherein each cell has a plurality of pixels selected from the second group of pixels and at least two pixels having the same spectral photoresponse corresponding to a specific color selected from the first group of pixels,

b.) using the signals for the pixels in each cell corresponding to like colored pixels from the first group of pixels to produce at least one first signal for each cell,

c.) using the set of first signals for each cell and the signals for each pixel in the second group of pixels to produce an adjusted color digital image that compensates for different lighting conditions.