Image extracting apparatus, image extracting method, and image extracting program

• US 7,796,785 B2
• Filed: 03/02/2006
• Issued: 09/14/2010
• Est. Priority Date: 03/03/2005
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kojima, Kenji, Takemoto, Fumito
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/365,945
• Publication Number:
  US 20060210165A1
• Time in Patent Office: 1,657 Days
• Field of Search: 382/118, 382/190, 382/209, 382/278, 382/296, 382/305
• US Class Current: 382/118
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: H04N 5/91 Television signal processing therefor

Abstract

The image extracting apparatus according to the present invention includes: a candidate image extracting section for extracting a moving image component including the facial image of a predetermined person from a moving image; and a representative image extracting section for extracting the moving image component including the facial image of the predetermined person which is more different from the facial image of the predetermined person with the usual facial expression among the plurality of moving image components extracted by the candidate image extracting section.

Claims

1. An image extracting apparatus for extracting a candidate image to be outputted as a representative image from a plurality of candidate images, comprising:

a candidate image extracting section for extracting at least one of candidate images including a predetermined facial image from the plurality of candidate images;

a first representative image extracting section for extracting the candidate image including the facial image of a predetermined person having a substantially different facial expression than a prepared facial image which includes a usual facial expression of the predetermined person among at least one of candidate images extracted by the candidate image extracting section;

an average facial information storage section for storing average facial information indicative of the facial image of the predetermined person in association with the predetermined person; and

a facial image comparison section for comparing the facial image included in the candidate image extracted by the candidate image extracting section, wherein

the first representative image extracting section extracts the candidate image including the facial image having a substantially different facial expression than the facial image indicated by the average facial information stored in the average facial information storage section among the candidate images extracted by the candidate image extracting section based on the comparison result by the facial image comparison section.

2. The image extracting apparatus according to claim 1 further comprising:

a facial image extracting section for extracting the facial image of the predetermined person from each of the plurality of candidate images extracted by the candidate image extracting section, respectively; and

an average facial information generating section for generating the average facial information on the predetermined person based on the plurality of facial images extracted by the facial image extracting section, wherein the average facial information storage section stores the average facial information generated by the average facial information generating section in association with the predetermined person.

3. The image extracting apparatus according to claim 1, wherein

the candidate image is a moving image component included in a moving image,

the candidate image extracting section extracts the moving image component including a predetermined person from the moving image,

the first representative image extracting section extracts the moving image component including the facial image of the predetermined person having a substantially different facial expression than the prepared facial image of the predetermined person among the plurality of moving image components extracted by the candidate image extracting section.

4. The image extracting apparatus according to claim 3 further comprising:

an average facial information storage section for storing the average facial information indicative of the facial image of the predetermined person in association with the predetermined person; and

a facial image comparison section for comparing each of the facial images included in the plurality of moving image components extracted by the candidate image extracting section with the facial image indicated by the average facial information stored in the average facial information storage section, wherein

the first representative image extracting section extracts the moving image component including the facial image having a substantially different facial expression than the facial image indicated by the average facial information stored in the average facial information storage section as a first representative image.

5. The image extracting apparatus according to claim 4 further comprising:

a facial image extracting section for extracting the facial image of the predetermined person from each of the plurality of moving image components extracted by the candidate image extracting section; and

an average facial information generating section for generating the average facial information on the predetermined person based on the plurality of facial images extracted by the facial image extracting section, wherein

the average facial information storage section stores the average facial information generated by the average facial information generating section in association with the predetermined person.

6. The image extracting apparatus according to claim 5 further comprising:

a part image extracting section for extracting a plurality of part images from each of the plurality of facial images extracted by the facial image extracting section; and

an average part shape calculating section for calculating the average shape for each part based on the plurality of part images extracted by the part image extracting section, wherein

the average facial information generating section generates the average facial information on the predetermined person based on the average shape for each part calculated by the average part shape calculating section.

7. The image extracting apparatus according to claim 3 further comprising a facial expression change calculating section for calculating the change of the facial image of the predetermined person in the plurality of moving image components extracted by the candidate image extracting section, wherein

the first representative image extracting section extracts a moving image component when the facial image of the predetermined person starts changing or a moving image component when the facial image of the predetermined person stops changing as the first representative image based on the change of the facial image of the predetermined person calculated by the facial expression change calculating section.

8. The image extracting apparatus according to claim 1 further comprising a template storage section for storing a template for an output area to previously define an image position and personal facial expression information indicative of the facial expression of the person included in the representative image to be laid out at the image position, wherein

the first representative image extracting section extracts the candidate image in which the facial expression of the person indicated by the personal facial expression information in association with the image position included in the template stored in the template storage section and the facial expression of the person included in the candidate image extracted by the candidate image extracting section are corresponded as the first representative image.

9. The image extracting apparatus according to claim 8, wherein

the template storage section stores the template for the output area to previously define the image position and the facial expression displacement information indicative of the amount of displacement of the facial expression of the person included in the first representative image to be laid out at the image position,

the first representative image extracting section extracts the candidate image in which the amount of displacement of the facial expression of the person indicated by the facial expression displacement information associated with the image position included in the template stored in the template storage section and the amount of displacement of the facial expression of the person included in the candidate image extracted by the candidate image extracting section are corresponded as the first representative image.

10. The image extracting apparatus according to claim 8, wherein

the template storage section stores the template for the output area to previously define the image position and variety of facial expression information indicative of the variety of the facial expression of the person included in the first representative image to be laid out at the image position, and

the first representative image extracting section extracts the candidate image in which the variety of the facial expression of the person indicated by the variety of facial expression information associated with the image position included in the template stored in the template storage section and the variety of the facial expression of the person included in the candidate image extracted by the candidate image extracting section are corresponded as the first representative image.

11. The image extracting apparatus according to claim 1 further comprising:

a template storage section for storing a template for an output area to previously define a first representative image position at which the first representative image should be laid out and a second representative image position at which a second representative image should be laid out;

a facial expression information determining section for determining personal facial expression information included in the second representative image to be laid out at the second representative image position according to the personal facial expression information indicative of the facial expression of the person included in the first representative image extracted by the first representative image extracting section; and

a second representative image extracting section for extracting the second representative image including the person with the facial expression indicated by the personal facial expression information determined by the facial expression information determining section from the candidate images extracted by the candidate image extracting section.

12. The image extracting apparatus according to claim 1, wherein

the usual facial expression is a face without expression.

13. An image extracting method for extracting a candidate image to be outputted as a representative image from a plurality of candidate images, comprising steps performed by an image extracting apparatus including a CPU, said steps including:

a first extracting step of extracting at least one of candidate images including a predetermined facial image from the plurality of candidate images;

a second extracting step of extracting the candidate image including the facial image of the predetermined person having a substantially different facial expression than a prepared facial image which includes a usual facial expression of the predetermined person among at least one of candidate images extracted in the candidate image extracting stem

storing average facial information indicative of the facial image of the predetermined person in association with the predetermined person; and

a comparing step of comparing the facial image included in the candidate image extracted by the candidate image extracting section, wherein

the second extracting step includes extracting the candidate image including the facial image having a substantially different facial expression than the facial image indicated by the average facial information stored in the average facial information storage section among the candidate images extracted in the first extracting step based on the comparison result of the comparing step.

14. A tangible computer readable recording medium encoded with a computer executable image extracting program for extracting a candidate image to be outputted as a representative image from a plurality of candidate images, the computer executable image extracting program when executed causes an image extracting apparatus to implement:

a candidate image extracting section for extracting at least one of candidate images including a predetermined facial image from the plurality of candidate images;

a first representative image extracting section for extracting the candidate image including the facial image of a predetermined person having a substantially different facial expression than a prepared facial image which includes a usual facial expression of the predetermined person among at least one of candidate images extracted by the candidate image extracting section;

an average facial information storage section for storing average facial information indicative of the facial image of the predetermined person in association with the predetermined person; and

a facial image comparison section for comparing the facial image included in the candidate image extracted by the candidate image extracting section, wherein

the first representative image extracting section extracts the candidate image including the facial image having a substantially different facial expression than the facial image indicated by the average facial information stored in the average facial information storage section among the candidate images extracted by the candidate image extracting section based on the comparison result by the facial image comparison section.