Method and apparatus for spatial processing of a digital image

• US 20090174797A1
• Filed: 01/03/2008
• Published: 07/09/2009
• Est. Priority Date: 01/03/2008
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Micron Technology, Inc.
• Inventors: Hu, Shane C., Mullis, Robert
• Granted Patent: US 8,035,704 B2
• Time in Patent Office: Days
• US Class Current: 348/247
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

A method and apparatus for image processing which performs pixel defect correction and color demosaicing on pixel image data while held in a line buffer.

Claims

1. A method of processing an image, the method comprising:

storing pixel information of a portion of the image in a line buffer capable of storing multiple lines of pixel information, wherein each pixel information relates to a single color;

determining whether a target pixel in the portion of the image stored in the line buffer is defective;

adjusting a value of the target pixel if it is determined to be a defective pixel; and

adjusting the target pixel value to interpolate color values containing a plurality of color components through an interpolation process using the pixel information stored in the line buffer.

2. The method of claim 1, wherein the interpolation process comprises adjusting the target pixel value to enhance the image contrast and to reduce noise in the image based on predefined parameters.

3. The method of claim 1, wherein the portion of the image comprises three rows of pixels.

4. The method of claim 1, wherein the portion of the image comprises three columns of pixels.

5. The method of claim 1, wherein a pixel is determined to be defective by detecting high contrast in a target pixel with respect to proximal like-colored pixels within the portion of the image.

6. The method of claim 5, wherein high contrast is detected by:

determining a first sum value from output values of a first set of pixels proximal to the target pixel;

determining a second sum value from output values of a second set of pixels proximal to the target pixel, wherein pixels in the second set of pixels is exclusive of pixels in the first set of pixels; and

comparing the target pixel value to the greater of the first and second sum values.

7. The method of claim 6, further comprising adjusting the target pixel value to at least a value related to the greater sum value when the target pixel value exceeds the greater sum value.

8. The method of claim 1, wherein the identified defective pixels are corrected by replacing the target pixel value with a value calculated based on proximal like-colored pixel values.

9. The method of claim 1, wherein the interpolation process comprises:

determining whether pixels that are in the proximity of the target pixel which have not yet been checked for defect are defective; and

storing a temporary adjusted value for each identified proximal defective pixel for use in the target pixel interpolation.

10. The method of claim 9, wherein the interpolation process further comprises:

calculating a local average color value for each color collected by same color pixels in a locality of the target pixel, using the temporary adjusted value for a pixel;

calculating a local average luminance value in the locality of the target pixel; and

calculating interpolated color values for the target pixel, where the value for each color is adjusted to enhance image sharpness based in part on the local average color value and the local average luminance value.

11. The method of claim 10, wherein the local average color value is calculated based on the target pixel value and a first number of local pixel values when the target pixel represents a first color, and on the target pixel value and a second number of local pixel values when the target pixel represents a second color.

12. The method of claim 11, wherein the first number of local pixel values is three, and the second number of local pixel values is two.

13. A method of processing an image comprised of an array of pixel values, each pixel value representing a single color, the method comprising:

storing pixel values of a first line of pixels of the image, a second line of pixels of the image and a third line of pixels of the image, where the second line of pixels is in between and adjacent to the first and third lines of pixels;

analyzing the stored pixel values of pixels in the second line of pixels to identify whether any pixel is a defective pixel;

calculating a corrected pixel data value for any pixel identified as a defective pixel;

storing the corrected pixel data value in place of the defective pixel data value; and

applying a demosaicing process to pixels in the second line that have been analyzed for defective pixel determination and correction, the demosaicing process also enhancing the contrast of the image and reducing the effects of pixel noise in the image.

14. The method of claim 13, wherein the pixel values are stored in a three line buffer.

15. The method of claim 13, wherein the pixel values are stored in a two line buffer and associated storage circuits.

16. The method of claim 13, wherein the defective pixels are hot pixels and wherein the identification of hot pixels comprises comparing a target pixel value to sum values calculated from four neighboring pixel values when the target pixel represents a first color, and comparing the target pixel value to a sum value calculated from two neighboring pixel values when the target pixel represents a second color.

17. The method of claim 13, wherein the demosaicing process comprises:

calculating a local average color value for each color collected by the pixels in a locality of the target pixel;

calculating a local average luminance value in the locality of the target pixel; and

calculating a value for the target pixel for each color collected by the pixels, where the value for each color is adjusted based in part on the local average color value and local average luminance value.

18. A method of processing a Bayer patterned filtered mosaic image comprised of an array of single color pixel values, the method comprising:

storing pixel values of three lines of pixels in a buffer storage circuit; and

performing a defect correction on target pixels stored in the buffer storage circuit and then performing a demosaicing process on the target pixel in the storage circuit after the defect correction is performed,

wherein the defect correction of a target pixel is completed prior to demosaicing the target pixel;

local and surrounding pixel information is used to perform the demosaicing, contrast enhancement and noise reduction; and

red and blue pixel values in the row subsequent to the target pixel row are checked for hot pixel defects and a temporary corrected value is stored for use in demosaicing if a hot pixel defect is detected.

19. The method of claim 18, further comprising controlling contrast enhancement in the demosaicing process in accordance with a user-adjustable parameter.

20. The method of claim 18, further comprising limiting noise reduction by a noise floor offset value in the demosaicing process in accordance with a user-adjustable parameter.

21. An image processing apparatus, comprising:

a storage circuit for receiving and storing at least two successive lines of single color pixel image data; and

a processor, configured to;

perform defect correction of a target pixel by;

comparing a target pixel value in the stored lines of pixel image data to neighboring stored pixel output values to determine whether the target pixel is a defective pixel;

determining and storing a replacement value for the target pixel if it is determined to be a defective pixel; and

after the defect correction, interpolate neighboring stored pixel values to determine a multiple color component pixel color value for the target pixel.

22. The image processing apparatus of claim 21, wherein the processor is further configured to calculate a temporary replacement value for neighboring defective pixels that have not yet been adjusted and use the temporary replacement value in the interpolation.

23. The image processing apparatus of claim 21, wherein the processor is further configured to determine whether a target pixel is defective by comparing the target pixel value to a sum value of two or more neighboring pixels values, where the target pixel is determined defective if the target pixel value exceeds the sum value.

24. The image processing apparatus of claim 21, wherein the processor is further configured to adjust the target pixel color value and associated color values during the interpolation to enhance image sharpness and decrease image noise.

25. The image processing apparatus as in claim 21, further comprising:

an array of pixels for capturing an image;

a lens for focusing an image on the array of pixels; and

a digitizing structure for providing single color pixel image data from an image captured by the array of pixels to the storage circuit.