Defective color and panchromatic CFA image

• US 8,111,307 B2
• Filed: 10/25/2008
• Issued: 02/07/2012
• Est. Priority Date: 10/25/2008
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Deever, Aaron T., Adams, James E. Jr., Hamilton, John F. Jr.
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): Berhan, Ahmed A
• Application Number:
  US12/258,389
• Publication Number:
  US 20100104209A1
• Time in Patent Office: 1,200 Days
• Field of Search: 348277-282, 348/241, 348/245, 348/246, 348/264, 348/265, 348/272
• US Class Current: 348/246
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection

Abstract

A method of improving a first color filter array image from an image sensor having a plurality of color channels and a panchromatic channel, includes capturing the panchromatic channel at a different exposure time than at least one of the color channels with the image sensor; using the color channels to provide a luminance channel; and analyzing the color filter array image and the luminance channel to determine defective pixels in the color channels and using neighboring color and luminance pixel values to improve the defective pixels to produce a second color filter array image or full-color image having at least one improved channel.

Claims

1. A method of improving a first color filter array image from an image sensor having a plurality of color channels and a panchromatic channel, comprising:

(a) capturing the panchromatic channel at a different exposure time than at least one of the plurality of color channels with the image sensor;

(b) using the plurality of color channels to provide a luminance channel; and

(c) analyzing the first color filter array image and the luminance channel to determine defective pixels in the plurality of color channels, wherein a defective pixel is a pixel whose luminance value does not match a corresponding panchromatic value, and using neighboring color and luminance pixel values to improve the defective pixels to produce a second color filter array image or full-color image having at least one improved channel, including

for each of the defective pixels, computing a color difference between a color value of at least one of the color channels and a luminance value of the luminance channel, and

adding the color difference to a panchromatic value of the panchromatic channel to improve each defective pixel.

2. The method of claim 1 wherein step (b) further includes producing an interpolated color image and using the interpolated color image to provide the luminance channel.

3. The method of claim 2 wherein step (c) further includes using the interpolated color image in determining and improving the defective pixels.

4. The method of claim 1 wherein the luminance channel is selected such that it is comparable to the panchromatic channel.

5. The method of claim 1, further comprising:

prior to analyzing the first color filter array image and the luminance channel, perform a motion estimate and compensation on the panchromatic channel and the luminance channel, wherein the analysis is performed based on the motion estimated and compensated panchromatic channel and luminance channel.

6. The method of claim 5 further comprising:

producing an interpolated color image and using the interpolated color image to provide the luminance channel; and

producing an interpolated panchromatic channel.

7. The method of claim 5 wherein the luminance channel is selected such that it is comparable to the panchromatic channel.

8. The method of claim 6, further comprising performing an up-sample operation on the interpolated color image and the luminance channel to produce up-sampled color channels and up-sampled luminance channels at an identical resolution as the interpolated panchromatic channel.

9. The method of claim 8, wherein the up-sampled luminance channel is used in conjunction with the interpolated panchromatic channel to estimate and compensate for a motion between panchromatic data, color data, and luminance data.

10. The method of claim 9, wherein the motion estimate and compensation is performed on pixels restricted to be translational offsets of a multiple of two pixels in each of a horizontal direction and a vertical direction.

11. The method of claim 5, wherein performing motion estimate and compensation comprises aligning the plurality of color channels with the panchromatic channel based on the luminance channel.

12. The method of claim 1, further comprising:

classifying all pixels as defective pixels;

for each of the defective pixels, the color differences of neighboring pixels within a predetermined proximity with respect to a corresponding defective pixel are averaged to generate an average color difference value; and

adding the average color difference value to the panchromatic value of the panchromatic channel to generate an improved color value for the corresponding defective pixel, wherein the improved color value is utilized to produce the second color filter array image or the full-color image having at least one improved channel.

13. The method of claim 12, wherein the neighboring pixels are identified having an identical spatial location in a Bayer pattern.

14. The method of claim 13, wherein the neighboring pixels are identified in which a difference between a luminance value of each neighboring pixel and the panchromatic value is less than a predetermined threshold.

15. The method of claim 14, further comprising performing a void filling operation on one or more pixels that are not within the identical spatial location of the Bayer pattern or the difference between the luminance value and the panchromatic value is greater than the predetermined threshold.

16. The method of claim 15, wherein performing the void filling operation comprises:

assigning a weight to each of the pixels as a function of a panchromatic value of each pixel and panchromatic values of the neighboring pixels having an identical spatial location within the Bayer pattern whose improved color values have already been calculated;

sorting the pixels based on weights assigned to the pixels;

selecting one of the pixels having a highest weight amongst all;

computing an improved color value for the selected pixel for the selected pixel;

removing the selected pixel from the sorted pixels; and

repeating the assigning, sorting, selecting, computing, and removing, until all pixels have been processed.