Method and apparatus for processing images

• US 6,667,815 B1
• Filed: 09/10/1999
• Issued: 12/23/2003
• Est. Priority Date: 09/30/1998
• Status: Expired due to Term

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Nagao, Kimitoshi
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US09/393,364
• Time in Patent Office: 1,565 Days
• Field of Search: 358/1.9, 358/3.26, 358/3.27, 358/530, 358/532, 382/254, 382/261, 382/263, 382/264, 382/275, 382/266
• US Class Current: 358/1.9
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: H04N 1/58 Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction H04N1/62 takes precedence

Abstract

According to the method and apparatus of processing a digital image for noise suppression and sharpness enhancement, sharpness enhancement and smoothing are performed on an original image to obtain a sharpness enhanced image and a smoothed image, from which image data containing subject image edges and noise (graininess) is determined; edge detection is performed from the original image to determine weighting data for a noise (or grainy) region, which is multiplied by the above image data to determined noise (grain) data of each color in a noise region, from which a black-and-white noise (grain) component and a color noise (dye grain) component are discriminated and separated based on a color correlation component obtained by calculating a color correlation or an obtained local grain coefficient representing a spatial magnitude and a variation magnitude of density variations due to graininess; the thus obtained black-and-white noise component and color noise component are multiplied by their suppressing coefficients to determine a black-and-white noise suppressing component and a color noise suppressing component, which are selectively removed from the sharpness enhanced image data, whereupon a processed image in which the noise suppression is achieved while retaining the sharpness enhancement in the edge region of the image is created. Consequently, graininess can be suppressed while enhancing image sharpness, without causing the unnatural artifacts which may be produced by the discoptinuousness of the boundary between a region from which graininess has been removed and a region where sharpness enhancement has been done.

Claims

1. A method of processing a digital image for noise suppression and sharpness enhancement, comprising the steps of:

performing a sharpness enhancing process on original image data of an original image to create sharpness enhanced image data in which an image and a noise included therein are both sharpened;

performing a smoothing process on said original image data to create smoothed image data;

subtracting the smoothed image data from said sharpness enhanced image data to create image data containing subject image edges and the noise of which the sharpness enhancement is achieved;

performing an edge detection from said original image data to determine weighting data for an edge region and weighting data for a noise region used to discriminate the edge region of a subject and the noise region;

multiplying said image data containing the subject image edges and the noise by said weighting data for the noise region to determine noise data of each color in the noise region;

determining a black-and-white noise component and a color noise component from the noise data of said each color;

multiplying the thus obtained black-and-white noise component and color noise component by their suppressing coefficients to determine a black-and-white noise suppressing component and a color noise suppressing component; and

selectively removing said black-and-white noise suppressing component and said color noise suppressing component from said sharpness enhanced image data, thereby creating a processed image in which the noise suppression is achieved while retaining the sharpness enhancement in the edge region of the image.

2. The method according to claim 1, wherein said black-and-white noise component and said color noise component are respectively determined as a component which exists equally in said each color and as a component which exists independently in said each color, based on a color correlation component obtained by calculating a color correlation of the noise data of said each color in said noise region.

3. The method according to claim 1, wherein said weighting data for the noise region is determined from said weighting data for the edge region.

4. The method according to claim 1, wherein said noise includes graininess, and said noise region, said black-and-white noise component and said color noise component are a grainy region, a black-and-white grain component and a dye grain component, respectively.

5. The method according to claim 4, wherein said original image is an image recorded a silver halide color photographic material and wherein said black-and-white grain component includes at least one of undeveloped silver halide grains and developed silver grains.

6. The method according to claim 1, wherein said original image data is image data captured with an image pickup device from an image recorded on a silver halide color photographic material, and wherein said black-and-white noise component includes not only a black-and-white grain component formed of at least one of undeveloped silver halide grains and developed silver grains, but also at least one of a random noise of said each color, a fixed pattern noise of said image pickup device and moire due to aliasing.

7. The method according to claim 2, wherein said original image is an image recorded with an image pickup device, and wherein said black-and-white noise component is a noise which is strong in said color correlation and which includes at least one of a random noise of said each color, a fixed pattern noise of said image pickup device and moire due to aliasing, whereas the color noise component is a noise which is weak in said color correlation.

8. The method according to claim 1, wherein said edge detection is performed by a local variance.

9. The method according to claim 1, wherein said sharpness enhancing process is Gaussian unsharp masking.

10. The method according to claim 1, wherein said smoothing process is Gaussian masking.

11. The method according to claim 1, wherein said sharpness enhancement is applied in a necessary and sufficiently intense manner, although graininess is considerably marked without being suppressed.

12. The method according to claim 1, wherein said noise includes graininess, and said noise region, said noise data, said black-and-white noise component, said color noise component, said black-and-white noise suppressing component and said color noise suppressing component are a grainy region, grain data, a black-and-white grain component, a dye grain component, a black-and-white grain suppressing component and a dye grain suppressing component, respectively, and wherein a local grain coefficient representing a spatial magnitude and a variation magnitude of density variations due to the graininess is determined and used to discriminate and separate said black-and-white grain component and said dye grain component from the grain data of said each color in said grainy region.

13. The method according to claim 12, wherein said original image is an image recorded on a sliver halide color photographic material, and wherein said black-and-white grain component includes at least one of undeveloped silver halide grains and developed silver grains.

14. The method according to claim 12, wherein said original image data is image data captured with an image pickup device from an image recorded on a silver halide color photographic material, and wherein said black-and-white grain component includes not only a grain component formed of at least one of undeveloped silver halide grains and developed silver grains, but also at least one of a random noise of said each color, a fixed pattern noise of said image pickup device and moire due to aliasing.

15. An apparatus for processing a digital image for noise suppression and sharpness enhancement, comprising:

a sharpness enhancing unit which performs a sharpness enhancing process on original image data of an original image to create sharpness enhanced image data in which an image and a noise included therein are both sharpened;

a smoothing unit which performs a smoothing process on said original image data to create smoothed image data;

an edge/noise component extracting unit which subtracts the smoothed image data from said sharpness enhanced image data to create image data containing subject image edges and the noise of which the sharpness enhancement is achieved;

an edge detecting unit which performs an edge detection from said original image data to determine weighting data for an edge region used to discriminate the edge region of a subject and a noise region;

a weighting coefficient calculating unit for the noise region in which weighting data for the noise region is determined from the weighting data for the edge region;

a noise component discriminating unit in which the image data containing the subject image edges and the noise is multiplied by said weighting data for the noise region to determine noise data of each color in the noise region, from which a black-and-white noise component and a color noise component are determined, after which the thus obtained black-and-white noise component and color noise component are multiplied by their suppressing coefficients to determine a black-and-white noise suppressing component and a color noise suppressing component; and

an output image calculating unit which selectively removes said black-and-white noise suppressing component and said color noise suppressing component from said sharpness enhanced image data, thereby creating a processed image in which the noise suppression is achieved while retaining the sharpness enhancement in the edge region of the image.

16. The apparatus according to claim 15, wherein said noise component discriminating unit determines said black-and-white noise component which exists equally in said each color and said color noise component based on a color correlation component obtained by calculating a color correlation of the noise data of said each color in said noise region.

17. The apparatus according to claim 15, wherein said noise includes graininess, and said noise region, said noise data, said black-and-white noise component, said color noise component, said black-and-white noise suppressing component, said color noise suppressing component, said edge/noise component extracting unit, said weighting coefficient calculating unit for the noise region and said noise component discriminating unit are a grainy region, grain data, a black-and-white grain component, a dye grain component, a black-and-white grain suppressing component, a dye grain suppressing component, an edge/grain component extracting subsection, a weighting coefficient calculating subsection for the grainy region and a processing subsection for discriminating a grain component, respectively, and wherein said processing subsection for discriminating the grain component determines a local grain coefficient representing a spatial magnitude and a variation magnitude of density variations due to the graininess to discriminate and separate said black-and-white grain component and said dye grain component from the grain data of said each color in said grainy region.