Image processing apparatus and method, recording medium, and program

• US 20040169747A1
• Filed: 12/10/2003
• Published: 09/02/2004
• Est. Priority Date: 01/14/2003
• Status: Active Grant

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ono, Hiroaki, Mitsunaga, Tomoo
• Granted Patent: US 7,362,894 B2
• Time in Patent Office: Days
• US Class Current: 348/272
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours

Abstract

In an image processing apparatus, a sensitivity compensation unit converts a color-and-sensitivity mosaic image into a color mosaic image. A pixel-of-interest determination unit extracts local area information from the color mosaic image. An edge-direction detector detects an edge of the local area. A G-component computing unit performs weighted interpolation in the edge direction of the green (G) component associated with a pixel of interest. A statistic computing unit computes statistic information of the local area. A first or second R-and-B-component computing unit computes the red (R) and blue (B) components of the pixel of interest on the basis of the statistic information. An inverse gamma conversion unit performs inverse gamma conversion of the red (R), green (G), and blue (B) components of the pixel of interest. The present invention is applicable to, for example, a digital still camera.

Claims

1. An image processing apparatus for generating a color image in which each pixel has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, comprising:

extraction means for extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

generation means for making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted by the extraction means and generating local area information including the pixels, each pixel having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

edge detection means for detecting an edge of the local area information on the basis of, of the pixels included in the local area information, pixels having a first color component;

first interpolation means for interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, the pixels having the first color component on the basis of the direction of the edge detected by the edge detection means;

statistic-information computing means for computing statistic information on the basis of the pixels included in the local area information; and

second interpolation means for interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the statistic information.

2. An image processing apparatus according to claim 1, further comprising defective-pixel interpolation means for interpolating a defective pixel included in the local area information using pixels neighboring the defective pixel.

3. An image processing apparatus according to claim 1, wherein the statistic-information computing means computes, as the statistic information, at least one of the average of each color component, standard deviation of each color component, and a correlation coefficient between the first color component and the other color component on the basis of the pixels included in the local area information.

4. An image processing apparatus according to claim 3, wherein the second interpolation means interpolates the color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of each color component, the standard deviation of each color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing means.

5. An image processing apparatus according to claim 3, wherein the second interpolation means interpolates the color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of the color component other than the first color component, which is computed by the statistic-information computing means.

6. An image processing apparatus according to claim 3, wherein the second interpolation means includes:

first computation means for computing the color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of each color component, the standard deviation of each color component, and the correlation coefficient between the first color component and the other color component, which are computed by the statistic-information computing means; and

second computation means for computing the color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the average of the color component other than the first color component, which is computed by the statistic-information computing means,

wherein one of the first computation means and the second computation means is selected to interpolate the color component other than the first color component associated with the pixel of interest.

7. An image processing apparatus according to claim 6, wherein the second interpolation means selects one of the first computation means and the second computation means on the basis of the standard deviation of the first color component, which is computed by the statistic-information computing means, to interpolate the color component other than the first color component associated with the pixel of interest.

8. An image processing apparatus according to claim 1, further comprising:

gamma conversion means for performing gamma conversion of the pixels included in the local area information; and

inverse gamma conversion means for performing inverse gamma conversion of the first color component associated with the pixel of interest, which is interpolated by the first interpolation means, and the color component other than the first color component associated with the pixel of interest, which is interpolated by the second interpolation means.

9. An image processing apparatus according to claim 1, wherein the first color component is a color component that statistically has the highest signal level of the plurality of color components.

10. An image processing apparatus according to claim 1, wherein the first color component is a color component that occupies the largest portion of the color mosaic image of the plurality of color components.

11. An image processing method for generating a color image in which each pixel has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the method comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each pixel having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, the pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistic information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistic information.

12. A recording medium having a computer-readable program recorded thereon for generating a color image in which each pixel has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the program comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each pixel having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, the pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistic information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistic information.

13. A program for generating a color image in which each pixel has a plurality of color components and a uniform sensitivity characteristic relative to optical intensity on the basis of a color-and-sensitivity mosaic image in which each pixel has one of the plurality of color components and is captured with one of a plurality of sensitivity characteristics relative to the optical intensity, the program causing a computer to perform a process comprising:

an extraction step of extracting a predetermined area centered on a pixel of interest, which is an object to be processed, from the color-and-sensitivity mosaic image;

a generation step of making uniform the sensitivity characteristics relative to the optical intensity of pixels included in the predetermined area extracted in the extraction step and generating local area information including the pixels, each pixel having one of the plurality of color components and the uniform sensitivity characteristic relative to the optical intensity;

an edge detection step of detecting an edge of the local area information on the basis of, of the pixels included in the local area information, pixels having a first color component;

a first interpolation step of interpolating the first color component associated with the pixel of interest by computing a weighted average using, of the pixels included in the local area information, the pixels having the first color component on the basis of the direction of the edge detected in the edge detection step;

a statistic-information computing step of computing statistic information on the basis of the pixels included in the local area information; and

a second interpolation step of interpolating a color component other than the first color component associated with the pixel of interest on the basis of the first color component associated with the pixel of interest, which is interpolated in the first interpolation step, and the statistic information.