Four-channel color filter array interpolation

• US 8,237,831 B2
• Filed: 05/28/2009
• Issued: 08/07/2012
• Est. Priority Date: 05/28/2009
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Adams, James E. Jr., Kumar, Mrityunjay, Pillman, Bruce H., Hamilton, James A.
• Primary Examiner(s): Tran, Sinh
• Assistant Examiner(s): BERARDESCA, PAUL M
• Application Number:
  US12/473,305
• Publication Number:
  US 20100302418A1
• Time in Patent Office: 1,167 Days
• Field of Search: 348/276, 348/277, 348/290, 348/279
• US Class Current: 348/277
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements

Abstract

A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising capturing a color filter array image using an image sensor including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels being panchromatic pixels; computing an interpolated panchromatic image from the color filter array image; computing an interpolated color image from the color filter array image; and forming the full color output image from the interpolated panchromatic image and the interpolated color image.

Claims

1. A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising one or more processors for providing the following:

a) capturing a color filter array image using an image sensor comprising a two dimensional array of light-sensitive pixels including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels within the minimal repeating unit being panchromatic pixels;

b) computing an interpolated panchromatic image from the color filter array image;

c) computing an interpolated color image from the color filter array image; and

d) forming the full color output image from the interpolated panchromatic image and the interpolated color image, wherein step d) further includes;

i) applying a high-pass filter to the interpolated panchromatic image to form a high frequency panchromatic image,

ii) applying a low-pass filter to the interpolated color image to form a low frequency color image, and

iii) merging the high frequency panchromatic image and the low frequency color image to form the full color output image.

2. The method of claim 1 wherein step c) further includes using the interpolated panchromatic image in the process of computing the interpolated color image.

3. The method of claim 2 wherein the process of computing the interpolated color image includes forming an interpolated color difference image using color difference values formed from the interpolated panchromatic image and the color filter array image.

4. The method of claim 1 wherein the positions of the color pixels along the diagonal are arranged to minimize the maximum distance between color pixels of the same color.

5. The method of claim 1 wherein there are no panchromatic pixels on the diagonal of the minimal repeating unit that includes color pixels.

6. A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising one or more processors for providing the following:

a) capturing a color filter array image using an image sensor comprising a two dimensional array of light-sensitive pixels including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels within the minimal repeating unit being panchromatic pixels;

b) computing an interpolated panchromatic image from the color filter array image;

c) computing an interpolated color image from the color filter array image; and

d) forming the full color output image from the interpolated panchromatic image and the interpolated color image, wherein step d) further includes;

i) performing a pyramid decomposition to the interpolated panchromatic image to form a panchromatic image pyramid,

ii) performing a pyramid decomposition to the interpolated color image to form a color image pyramid,

iii) merging the panchromatic image pyramid and the color image pyramid to form the output image pyramid, and

iv) forming the full color output image from the output image pyramid.

7. The method of claim 6 wherein step iii) includes applying a high-pass filter to each level of the panchromatic image pyramid to form a high frequency panchromatic image pyramid, applying a low-pass filter to each level of the color image pyramid to form a low frequency color image pyramid and merging each level of the high frequency panchromatic image pyramid and the low frequency color image pyramid to form the output image pyramid.

8. A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising one or more processors for providing the following:

a) capturing a color filter array image using an image sensor comprising a two dimensional array of light-sensitive pixels including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels within the minimal repeating unit being panchromatic pixels, wherein the color pixels are red, green and blue pixels;

b) computing an interpolated panchromatic image from the color filter array image;

c) computing an interpolated color image from the color filter array image; and

d) forming the full color output image from the interpolated panchromatic image and the interpolated color image.

9. The method of claim 8 wherein step c) further includes using the interpolated panchromatic image in the process of computing the interpolated color image.

10. The method of claim 9 wherein the process of computing the interpolated color image includes forming an interpolated color difference image using color difference values formed from the interpolated panchromatic image and the color filter array image.

11. The method of claim 8 wherein the positions of the color pixels along the diagonal are arranged to minimize the maximum distance between color pixels of the same color.

12. The method of claim 8 wherein there are no panchromatic pixels on the diagonal of the minimal repeating unit that includes color pixels.

13. A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising one or more processors for providing the following:

a) capturing a color filter array image using an image sensor comprising a two dimensional array of light-sensitive pixels including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels within the minimal repeating unit being panchromatic pixels, wherein the color pixels are cyan, magenta and yellow pixels;

b) computing an interpolated panchromatic image from the color filter array image;

c) computing an interpolated color image from the color filter array image; and

d) forming the full color output image from the interpolated panchromatic image and the interpolated color image.

14. A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising one or more processors for providing the following:

a) capturing a color filter array image using an image sensor comprising a two dimensional array of light-sensitive pixels including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels within the minimal repeating unit being panchromatic pixels, wherein the minimal repeating unit has three rows and three columns, and wherein one of the diagonals of the minimal repeating unit has one red pixel, one green pixel and one blue pixel;

b) computing an interpolated panchromatic image from the color filter array image;

c) computing an interpolated color image from the color filter array image; and

d) forming the full color output image from the interpolated panchromatic image and the interpolated color image.

15. The method of claim 14 wherein step c) further includes using the interpolated panchromatic image in the process of computing the interpolated color image.

16. The method of claim 15 wherein the process of computing the interpolated color image includes forming an interpolated color difference image using color difference values formed from the interpolated panchromatic image and the color filter array image.

17. The method of claim 14 wherein the positions of the color pixels along the diagonal are arranged to minimize the maximum distance between color pixels of the same color.

18. The method of claim 14 wherein there are no panchromatic pixels on the diagonal of the minimal repeating unit that includes color pixels.

19. A method of forming a full-color output image from a color filter array image having a plurality of color pixels having at least two different color responses and panchromatic pixels, comprising one or more processors for providing the following:

a) capturing a color filter array image using an image sensor comprising a two dimensional array of light-sensitive pixels including panchromatic pixels and color pixels having at least two different color responses, the pixels being arranged in a repeating pattern having a square minimal repeating unit having at least three rows and three columns, the color pixels being arranged along one of the diagonals of the minimal repeating unit, and all other pixels within the minimal repeating unit being panchromatic pixels, wherein the minimal repeating unit has four rows and four columns, and wherein one of the diagonals of the minimal repeating unit has one red pixel, two green pixels and one blue pixel;

b) computing an interpolated panchromatic image from the color filter array image;

c) computing an interpolated color image from the color filter array image; and

d) forming the full color output image from the interpolated panchromatic image and the interpolated color image.

20. The method of claim 19 wherein the two green pixels are separated by either a red pixel or a blue pixel.

21. The method of claim 19 wherein step c) further includes using the interpolated panchromatic image in the process of computing the interpolated color image.

22. The method of claim 21 wherein the process of computing the interpolated color image includes forming an interpolated color difference image using color difference values formed from the interpolated panchromatic image and the color filter array image.

23. The method of claim 19 wherein the positions of the color pixels along the diagonal are arranged to minimize the maximum distance between color pixels of the same color.

24. The method of claim 19 wherein there are no panchromatic pixels on the diagonal of the minimal repeating unit that includes color pixels.