Image processing method, image processing apparatus, and computer-readable recording medium storing image processing program

• US 7,756,343 B2
• Filed: 09/25/2006
• Issued: 07/13/2010
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Ito, Wataru, Li, Yuanzhong
• Primary Examiner(s): Chawan; Sheela C
• Application Number:
  US11/525,839
• Publication Number:
  US 20070071347A1
• Time in Patent Office: 1,387 Days
• Field of Search: 382/128, 382/260, 382/224, 382/313, 382/100, 382/254, 382/181, 382/312, 382/228, 382/118, 606/9, 430/544, 430/505, 430/553, 430/555, 430/557, 430/558, 430/270.1, 430/281.1, 430/909, 101/467, 101/478, 250/557, 250/208.1, 358/473
• US Class Current: 382/228
• CPC Class Codes: G06F 18/2134 based on separation criteria, e.g. independent component analysis
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

An unnecessary component such as acne is removed completely from a predetermined structure such as a face in a photograph image without manual operation and skills. An acne removal unit fits to a face region as the structure in the image a mathematical model generated according to a statistical method such as AAM using sample images representing the structure without the component to be removed, and an image reconstruction unit reconstructs an image of the face region based on parameters corresponding to the face region obtained by the fitting of the model. An image is then generated by replacing the face region with the reconstructed image. Since the mathematical model has been generated from the sample images of human faces without acne, the model does not include acne. Therefore, the reconstructed face image generated by fitting the model to the face region does not include acne.

Claims

1. An image processing method for removing a first component which is included in a predetermined structure in an input image, comprising the steps of:

obtaining weighting parameters for statistical characteristic quantities corresponding to the predetermined structure in the input image by fitting to the predetermined structure in the input image a model representing the predetermined structure that does not include the first component, the model obtained through execution of predetermined statistical processing on a plurality of images representing the predetermined structure not having the first component, and the model representing the structure by one or more statistical characteristic quantities representing components in the structure other than the first component to be removed and by one or more weighting parameters for weighting the corresponding statistical characteristic quantities according to an individual characteristic of the predetermined structure; and

generating an output image in which the first component has been removed based on the obtained weighting parameters and the statistical characteristic quantities.

2. An image processing apparatus for removing a first component which is included in a predetermined structure in an input image, comprising:

a model representing the predetermined structure that does not include the first component by one or more statistical characteristic quantities representing components in the structure other than the first component to be removed and by one or more weighting parameters for weighting the corresponding statistical characteristic quantities according to an individual characteristic of the predetermined structure, the model obtained through execution of predetermined statistical processing on a plurality of images representing the predetermined structure not having the first component;

obtaining means for obtaining weighting parameters for statistical characteristic quantities corresponding to the predetermined structure in the input image by fitting the model to the predetermined structure in the input image; and

means for generating an output image in which the first component has been removed based on the obtained weighting parameters and the statistical characteristic quantities.

3. The image processing apparatus according to claim 2 further comprising detection means for detecting the predetermined structure in the input image, wherein

the obtaining means fits the model to the structure having been detected by the detection means.

4. The image processing apparatus according to claim 2 further comprising:

a plurality of the models representing the predetermined structure for respective properties of the structure; and

selection means for obtaining at least one property of the structure in the input image and for selecting the model corresponding to the obtained property, wherein

the obtaining means fits the model selected by the selection means to the structure in the input image.

5. The image processing apparatus according to claim 2, wherein the predetermined structure is a human face.

6. The image processing apparatus according to claim 2 wherein the first component to be removed is at least one of acne, a wrinkle, a spot, a small scar, stubble, and a flare.

7. The image processing apparatus according to claim 2, wherein the model and the fitting of the model are realized by a method of Active Appearance Model.

8. A computer-readable recording medium storing an image processing program for removing a first component which is included in a predetermined structure in an input image, said processing program causing a computer to carry out the procedure of:

obtaining weighting parameters for statistical characteristic quantities corresponding to the predetermined structure in the input image by fitting to the predetermined structure in the input image a model representing the predetermined structure that does not include the first component, the model obtained through execution of predetermined statistical processing on a plurality of images representing the predetermined structure not having the first component, and the model representing the predetermined structure by one or more statistical characteristic quantities representing components in the structure other than the first component to be removed and by one or more weighting parameters for weighting the corresponding statistical characteristic quantities according to an individual characteristic of the predetermined structure; and

generating an output image in which the first component has been removed based on the obtained weighting parameters and the statistical characteristic quantities.