Automatically determining a name of a person appearing in an image

• US 9,020,212 B2
• Filed: 07/31/2012
• Issued: 04/28/2015
• Est. Priority Date: 07/31/2012
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zhang, Tong, Koutrika, Georgia, O'Brien-Strain, Eamonn
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/562,565
• Publication Number:
  US 20140037154A1
• Time in Patent Office: 1,001 Days
• Field of Search: 382/118, 382/155, 382/181, 707/755
• US Class Current: 382/118
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

A computer-implemented method of automatically determining a name of a person appearing in an image includes receiving a collection of web pages containing a plurality of images. For each of the images, a set of names associated with the image is identified based on a text analysis of at least one of the web pages. Face detection and clustering is performed on the plurality of images to generate a plurality of face clusters. For each of the face clusters, a label for the face cluster is identified based on the set of names associated with each image in the face cluster. A name of a first person appearing in at least one of the images is determined based on the identified label for one of the face clusters associated with the first person.

Claims

1. A computer-implemented method of automatically determining a name of a person appearing in an image, comprising:

receiving a collection of web pages containing a plurality of images;

for each of the images, identifying a set of names associated with the image based on a text analysis of at least one of the web pages;

performing face detection and clustering on the plurality of images to generate a plurality of face clusters, wherein each of the face clusters is associated with one person and includes a set of at least one image from the plurality of images in which a face of the one person was detected;

for each of the face clusters, identifying a label for the face cluster based on the set of names associated with each image in the face cluster;

determining a name of a first person appearing in at least one of the images based on the identified label for one of the face clusters associated with the first person; and

wherein the receiving, identifying a set of names, performing face detection and clustering, identifying a label, and determining a name are performed by at least one processor.

2. The computer-implemented method of claim 1, and further comprising:

for each of the face clusters, comparing the set of names associated with each image in the face cluster to determine an intersection set of names.

3. The computer-implemented method of claim 2, wherein the intersection set of names for each face cluster is used as the label for the face cluster.

4. The computer-implemented method of claim 1, and further comprising:

comparing the label for each face cluster to the labels for other ones of the face clusters to identify candidate face clusters for merging.

5. The computer-implemented method of claim 4, wherein two of the face clusters are identified as candidate face clusters if the two face clusters have matching labels.

6. The computer-implemented method of claim 4, and further comprising:

performing face matching on faces appearing in images of a pair of the candidate face clusters.

7. The computer-implemented method of claim 6, and further comprising:

merging the pair of candidate face clusters into a single face cluster if the face matching exceeds a specified threshold.

8. The computer-implemented method of claim 1, and further comprising:

identifying candidate face clusters for merging based on the labels for the face clusters;

performing face matching on faces appearing in images of the candidate face clusters; and

merging selected ones of the candidate face clusters based on results of the face matching, thereby generating an updated set of face clusters.

9. The computer-implemented method of claim 8, and further comprising:

for each of the face clusters in the updated set, identifying an updated label for the face cluster based on the set of names associated with each image in the face cluster.

10. The computer-implemented method of claim 1, and further comprising:

automatically adding the name of the first person to the at least one image in which the first person appears.

11. A computer-readable storage medium storing computer-executable instructions that when executed by at least one processor cause the at least one processor to perform a method within a computing system, the method comprising:

receiving a collection of web pages containing a plurality of images;

for each of the images, identifying a set of names associated with the image based on a text analysis of at least one of the web pages;

generating a plurality of face clusters based on face detection and clustering performed on the plurality of images, wherein each of the face clusters is associated with one person and includes a set of at least one image from the plurality of images in which a face of the one person was detected;

for each of the face clusters, identifying a label for the face cluster based on the set of names associated with each image in the face cluster; and

determining a name of a first person appearing in at least one of the plurality of images based on the identified label for one of the face clusters associated with the first person.

12. The computer-readable storage medium of claim 11, wherein the method further comprises:

for each of the face clusters, determining an intersection set of names based on a comparison of the set of names associated with each image in the face cluster, wherein the intersection set of names for each face cluster is used as the label for the face cluster.

13. The computer-readable storage medium of claim 11, wherein the method further comprises:

identifying candidate face clusters for merging based on a comparison of the label for each face cluster to the labels for other ones of the face clusters.

14. The computer-readable storage medium of claim 13, wherein two of the face clusters are identified as candidate face clusters if the two face clusters have matching labels.

15. The computer-readable storage medium of claim 13, wherein the method further comprises:

performing face matching on faces appearing in images of a pair of the candidate face clusters.

16. The computer-readable storage medium of claim 15, wherein the method further comprises:

merging the pair of candidate face clusters into a single face cluster if the face matching exceeds a specified threshold.

17. The computer-readable storage medium of claim 11, wherein the method further comprises:

identifying candidate face clusters for merging based on the labels for the face clusters;

performing face matching on faces appearing in images of the candidate face clusters; and

merging selected ones of the candidate face clusters based on results of the face matching, thereby generating an updated set of face clusters.

18. The computer-readable storage medium of claim 17, wherein the method further comprises:

for each of the face clusters in the updated set, identifying an updated label for the face cluster based on the set of names associated with each image in the face cluster.

19. A system for automatically determining a name of a person appearing in an image, comprising:

a memory to store web pages containing a plurality of images; and

a processor coupled to the memory to;

for each of the images, perform a text analysis of at least one of the web pages to identify a set of names associated with the image;

generate a plurality of face clusters by performing face detection and clustering on the plurality of images, wherein each of the face clusters is associated with one person and includes a set of at least one image from the plurality of images in which a face of the one person was detected;

for each of the face clusters, identify a label for the face cluster based on the set of names associated with each image in the face cluster; and

determine a name of a first person appearing in at least one of the plurality of images based on the identified label for one of the face clusters associated with the first person.

20. The system of claim 19, wherein the processor:

for each of the face clusters, compares the set of names associated with each image in the face cluster to determine an intersection set of names, wherein the intersection set of names for each face cluster is used as the label for the face cluster.