Method and device for extracting principal image subjects

• US 6,529,630 B1
• Filed: 03/02/1999
• Issued: 03/04/2003
• Est. Priority Date: 03/02/1998
• Status: Expired due to Term

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Kinjo, Naoto
• Primary Examiner(s): Lee, Thomas D.
• Application Number:
  US09/260,516
• Time in Patent Office: 1,463 Days
• Field of Search: 382/115, 382/117, 382/118, 382/173, 382/199, 382/203, 382/190, 382/195
• US Class Current: 382/190
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

The method for extracting a principal image subject performs a plurality of subject extractions having different characteristics, synthesizes extracted subjects and extracts the principal image subject from a result of a synthesis if it satisfies a specified criterion of judgment, but if it does not satisfy, further performs an auxiliary image subject extraction having a different characteristic and extracts the principal image subject using the result of the synthesis and a result of extraction by the auxiliary image subject extraction. The device for extracting a principal image subject has a cascated extracting units at least one of which comprises a plurality of extracting subunits processed in parallel and a condition evaluating section for evaluating conditions the subject extractions based on results in the respective subunits. The method and device can be used with photoprinters, both digital and analog alike, and yet principal image subjects can be extracted in high precision and with satisfactory efficiency in response to diverse states of images and principal subjects.

Claims

1. A method for extracting a principal image subject comprising the steps of:

performing a plurality of subject extractions having different characteristics;

synthesizing subjects extracted by the respective subject extractions; and

extracting the principal image subject from a result of a synthesis if the result of said synthesis satisfies a specified criterion of judgment;

but

if the result of said synthesis does not satisfy the criterion of judgment, performing an auxiliary image subject extraction having a different characteristic after said synthesizing and extracting the principal image subject using the result of said synthesis and a result of extraction by said auxiliary image subject extraction.

2. The method for extracting a principal image subject according to claim 1, wherein said criterion of judgment is at least one of a degree of overlap between regions that occurs when the subjects extracted by the respective subject extractions are synthesized and a weighting score applied when the subjects extracted by the respective subject extractions are synthesized.

3. The method for extracting a principal image subject according to claim 1, wherein if, in case of extracting the principal image subject from the result of said synthesis, an image characteristic quantity of the extracted principal image subject exceeds a predetermined threshold, said auxiliary image subject extraction is performed and the principal image subject is extracted using the result of said synthesis and the result of extraction by said auxiliary image subject extraction.

4. The method for extracting a principal image subject according to claim 1, wherein if, in case of extracting the principal image subject from the result of said synthesis, a plurality of principal image subjects are extracted and if the image characteristic quantities of said plurality of principal image subjects have differences in excess of a specified level, said auxiliary image subject extraction is performed and the principal image subject is extracted using the result of extraction by said auxiliary image subject extraction.

5. The method for extracting a principal image subject according to claim 1, wherein said subject extractions are at least two operations selected from among skin color extraction, face contour extraction and circular shape extraction, and said auxiliary image subject extraction is at least one operation selected from among hairs-on-head extraction, face'"'"'s internal structure extraction, body extraction, non-background extraction and circular shape extraction.

6. A device for extracting a principal image subject comprising:

a plurality of extracting units which are cascaded and at least one of which comprises;

a plurality of extracting subunits which are processed in parallel and which perform subject extractions having different characteristics; and

a condition evaluating section which evaluates conditions for said subject extractions based on results in the respective extracting subunits; and

wherein

said condition evaluating section performs weighting on respective extracting regions of subjects extracted by said extracting subunits, and/or evaluation of appropriateness of said subjects, as well as selection of at least one of said extracting units which is located downstream and to which extraction processing is applied based on the results thereof.

7. The device for extracting a principal image subject according to claim 6, wherein said extracting units comprise at least one unit which has one extracting subunit performing one subject extraction and the condition evaluating section which evaluates the conditions for said subject extraction based on the results in the extracting subunit, or at least one unit which has no extracting subunit but only the condition evaluating section which evaluates the conditions for the extracted subjects based on the results in the cascaded precedent extracting unit.

8. The device for extracting a principal image subject according to claim 6, wherein said extracting subunits comprises at least one section selected from the group consisting of a skin color extracting section, a face contour extracting section, a circular shape extracting section, a body extracting section, an eye extracting section, a face'"'"'s internal structure extracting section, a hairs-on-head extracting section and a non-background extracting section, as well as an extracting section for performing at least two of a skin color extraction, a face contour extraction, a circular shape extraction, a body extraction, an eye extraction, a face'"'"'s internal structure extraction, a hairs-on-head extraction and non-background extraction.

9. The device for extracting a principal image subject according to claim 6, wherein said condition evaluating section comprises a synthesis/result evaluating section which synthesizes candidate subjects having different characteristics as extracted in the plurality of extracting subunits processed in parallel, and which judges whether the results of the subject extractions are appropriate or not based on the extracting regions of said synthesized candidate subjects.

10. The device for extracting a principal image subject according to claim 6, wherein the condition evaluating section in the last one of the cascaded extracting units is a principal image subject evaluating section for extracting the principal image subject.

11. A method for extracting a principal image subject, comprising:

a plurality of extracting steps which are cascaded, at least one of said plurality of extracting steps comprising;

a plurality of extracting substeps processed in parallel for performing subject extractions having different characteristics; and

a condition evaluating substep for evaluating conditions for said subject extractions based on results in the respective extracting substeps; and

wherein

said condition evaluating substep is performing weighting on respective extracting regions of subjects extracted in said extracting substeps, and/or evaluation of appropriateness of said subjects, as well as selection of at least one of said extracting steps which is located downstream and to which extraction processing is applied based on the results thereof.

12. The method for extracting a principal image subject according to claim 11, wherein said plurality of extracting steps contain at least one step which has one extracting substep performing one subject extraction and the condition evaluating substep which evaluates the conditions for said subject extraction based on the results in the extracting substep, or at least one step which has no extracting substep but only the condition evaluating substep which evaluates the conditions for the extracted subjects based on the results in the cascaded precedent extracting step.

13. The method for extracting a principal image subject according to claim 11, wherein said extracting substeps comprises at least one selected from the group consisting of skin color extracting, face contour extracting, a circular shape extracting, body extracting, eye extracting, face'"'"'s internal structure extracting, hairs-on-head extracting, non-background extracting and extraction for performing at least two thereof.

14. The method for extracting a principal image subject according to claim 11, wherein said condition evaluating substep comprises synthesis/result evaluating substep synthesizing candidate subjects having different characteristics as extracted in the plurality of extracting substeps processed in parallel, and judging whether the results of the subject extractions are appropriate or not based on the extracting regions of said synthesized candidate subjects.

15. The method for extracting a principal image subject according to claim 11, wherein the condition evaluating substep in the last one of the cascaded extracting steps is a principal image subject evaluating substep for extracting the principal image subject.

16. A method of extracting a principal image subject, comprising the steps of:

preliminarily classifying a state of a principal subject in an image into a plurality of types;

setting an appropriate parameter for the extraction of the principal subject for each state of the principal subjects;

evaluating the similarity between two images; and

,

if similarity is found to exist between the image from which the principal subject is to be extracted and the image from which the principal subject has already been extracted, extracting the principal subject using a parameter for the extraction of the principal subject that corresponds to the state of the principal subject in the image that has already been subjected to the extraction of the principal subject.

17. The method of extracting a principal image subject according to claim 16, wherein the state of the principal subject in said image is at least one element of the group consisting of size, orientation, position in the image, color distribution and density distribution.

18. A method of extracting a principal image subject, comprising the steps of:

extracting a principal subject from an image;

using luminance data on shooting of the image to determine luminance relationship between a region containing the extracted principal subject and its surrounding region, as well as density relationship between the region containing the extracted principal subject and the surrounding region; and

judging for the match between said luminance and density relationships to see if a result of extracting said principal subject is appropriate or not.