IMAGE PROCESSING APPARATUS, METHOD THEREOF, PROGRAM, AND IMAGE CAPTURING APPARATUS

• US 20120045095A1
• Filed: 08/11/2011
• Published: 02/23/2012
• Est. Priority Date: 08/18/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tate, Shunta, Mitarai, Yusuke, Mori, Katsuhiko, Matsugu, Masakazu, Takimoto, Masafumi
• Granted Patent: US 8,861,881 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/75 involving models

Abstract

An image processing apparatus stores model information representing a subject model belonging to a specific category, detects the subject from an input image by referring to the model information, determines a region for which an image correction is to be performed within a region occupied by the detected subject in the input image, stores, for a local region of the image, a plurality of correction data sets representing correspondence between a feature vector representing a feature before correction and a feature vector representing a feature after correction, selects at least one of the correction data sets to be used to correct a local region included in the region determined to undergo the image correction, and corrects the region determined to undergo the image correction using the selected correction data sets.

Claims

1. An image processing apparatus comprising:

a model storage unit adapted to store model information representing a subject model belonging to a specific category;

a detection unit adapted to detect the subject from an input image by referring to the model information;

a determination unit adapted to determine a region for which an image correction is to be performed within a region occupied by the detected subject in the input image;

a correction data set storage unit adapted to store, for a local region of the image, a plurality of correction data sets representing correspondence between a feature vector representing a feature before correction and a feature vector representing a feature after correction;

a selection unit adapted to select, from said correction data set storage unit, at least one of the correction data sets to be used to correct a local region included in the region determined to undergo the image correction; and

a correction unit adapted to correct the region determined to undergo the image correction using the selected correction data sets.

2. The apparatus according to claim 1, wherein

the subject model includes an object model representing a shape of the subject,

said detection unit identifies a position of the subject from the input image by referring to model information representing the object model, and

said selection unit selects the correction data sets in accordance with the identified position.

3. The apparatus according to claim 2, wherein

the object model includes a model representing a layout relationship between parts of an object, and

said selection unit normalizes each part of the subject that is detected by said detection unit by referring to the model information representing the object model, and selects the correction data sets in accordance with a position of a local region included in the normalized part.

4. The apparatus according to claim 3, wherein said correction unit performs correction of a background region, a region of the part of the object, and a region of an object other than the part using different correction data sets.

5. The apparatus according to claim 1, wherein

said model storage unit stores the model information for each subcategory representing a more detailed classification of the category,

the apparatus further comprises a discrimination unit adapted to discriminate a subcategory of the subject detected by said detection unit, and

said selection unit selects the correction data sets based on a discrimination result of the subcategory.

6. The apparatus according to claim 5, wherein the correction data sets stored in said correction data set storage unit include correction data sets common to all subjects belonging to a single category and correction data sets that are different for each subcategory.

7. The apparatus according to claim 5, wherein

the category is a classification representing one of a person and an animal species;

and

said discrimination unit discriminates the subcategory of the subject as a registered individual or an unknown individual.

8. The apparatus according to claim 1, further comprising:

an acquisition unit adapted to acquire, from an image storage unit, a plurality of images similar to the subject detected by said detection unit; and

a creation unit adapted to create, using the plurality of acquired images, the correction data sets to be used to perform the image correction concerning the subject.

9. The apparatus according to claim 1, further comprising:

a display unit adapted to display a plurality of results obtained by performing the image correction of the input image under a plurality of conditions; and

an acceptance unit adapted to accept selection of one of the plurality of results from a user.

11. A computer-readable storage medium storing a program that causes a computer to function as each unit of an image processing apparatus of claim 1.

12. An image capturing apparatus comprising:

an image capturing unit adapted to capture an image; and

an image processing apparatus of claim 1 adapted to process an input image that is the image captured by said image capturing unit.

10. An image processing method of an image processing apparatus including a model storage unit adapted to store model information representing a subject model belonging to a specific category, and a correction data set storage unit adapted to store, for a local region of the image, a plurality of correction data sets representing correspondence between a feature vector representing a feature before correction and a feature vector representing a feature after correction, the method comprising:

detecting the subject from an input image by referring to the model information;

determining a region for which an image correction is to be performed within a region occupied by the detected subject in the input image;

selecting, from the correction data set storage unit, at least one of the correction data sets to be used to correct a local region included the region determined to undergo the image correction; and

correcting the region determined to undergo the image correction using the selected correction data sets.