Image processing method, image processing apparatus, and computer-readable recording medium storing image processing program

• US 20070071347A1
• Filed: 09/25/2006
• Published: 03/29/2007
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Ito, Wataru, Li, Yuanzhong
• Granted Patent: US 7,756,343 B2
• Time in Patent Office: Days
• US Class Current: 382/254
• CPC Class Codes: G06F 18/2134 based on separation criteria, e.g. independent component analysis
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

An unnecessary component such as acne is removed completely from a predetermined structure such as a face in a photograph image without manual operation and skills. An acne removal unit fits to a face region as the structure in the image a mathematical model generated according to a statistical method such as AAM using sample images representing the structure without the component to be removed, and an image reconstruction unit reconstructs an image of the face region based on parameters corresponding to the face region obtained by the fitting of the model. An image is then generated by replacing the face region with the reconstructed image. Since the mathematical model has been generated from the sample images of human faces without acne, the model does not include acne. Therefore, the reconstructed face image generated by fitting the model to the face region does not include acne.

Claims

1. An image processing method comprising the step of:

removing a component as a target of removal in a predetermined structure in an input image, by fitting to the structure in the input image a model representing the structure by one or more statistical characteristic quantities having been obtained through execution of predetermined statistical processing on a plurality of images representing the structure not having the component as the target of removal.

2. An image processing apparatus comprising:

a model representing a predetermined structure by one or more statistical characteristic quantities having been obtained through execution of predetermined statistical processing on a plurality of images representing the structure not having a component as a target of removal; and

removal means for removing the component as the target of removal in the predetermined structure in an input image, by fitting the model to the structure in the input image.

3. The image processing apparatus according to claim 2 further comprising detection means for detecting the structure in the input image, wherein the removal means fits the model to the structure having been detected by the detection means.

4. The image processing apparatus according to claim 2 further comprising:

a plurality of the models representing the predetermined structure for respective properties of the structure; and

selection means for obtaining at least one property of the structure in the input image and for selecting the model corresponding to the obtained property, wherein the removal means fits the model selected by the selection means to the structure in the input image.

5. The image processing apparatus according to claim 2, wherein the predetermined structure is a human face.

6. The image processing apparatus according to claim 2 wherein the component as the target of removal is at least one of acne, a wrinkle, a spot, a small scar, stubble, and a flare.

7. The image processing apparatus according to claim 2, wherein the model and the fitting of the model are realized by a method of Active Appearance Model.

8. A computer-readable recording medium storing an image processing program causing a computer to carry out the procedure of:

removing a component as a target of removal in a predetermined structure in an input image, by fitting to the structure in the input image a model representing the structure by one or more statistical characteristic quantities having been obtained through execution of predetermined statistical processing on a plurality of images representing the structure not having the component as the target of removal.