Identifying images using face recognition

• US 9,552,511 B2
• Filed: 05/21/2015
• Issued: 01/24/2017
• Est. Priority Date: 12/01/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Yagnik, Jay
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US14/718,212
• Publication Number:
  US 20160224826A1
• Time in Patent Office: 614 Days
• Field of Search: 382/118, 382/181
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method includes identifying a named entity, retrieving images associated with the named entity, and using a face detection algorithm to perform face detection on the retrieved images to detect faces in the retrieved images. At least one representative face image from the retrieved images is identified, and the representative face image is used to identify one or more additional images representing the at least one named entity.

Claims

1. A non-transitory computer readable storage medium storing instructions executable by one or more computers and upon such execution cause the one or more computers to perform operations comprising:

identifying, by at least one of the one or more computers, a representative face image from a set of three or more images by comparing each detected face in each of the images in the set of images to each of the other detected faces from the images to identify faces that match the representative face image associated with a named entity, at least one of the images in the set of images including a textual description that includes at least a portion of a name of the named entity;

determining, by at least one of the one or more computers, a number of faces in the images in the set of images that match the representative face image;

determining, by at least one of the one or more computers, that the number of faces that match the representative face image satisfies a threshold value;

in response to determining that the number of faces that match the representative face image satisfies the threshold value, building, by at least one of the one or more computers, a face model using the representative face image including identifying a training set that includes the representative face image and all other images that match the representative face image from the images in the set of images; and

using, by at least one of the one or more computers, the face model to identify one or more additional images representing the named entity from images not included in the set of images.

2. The computer readable storage medium of claim 1 wherein identifying the representative face image includes identifying the representative face image as the image that contains the detected face that matches a highest number of the other of the detected faces.

3. The computer readable storage medium of claim 2, the operations comprising determining the detected face in the images that matches a highest number of other detected faces in the images in the set of images.

4. The computer readable storage medium of claim 1, the operations comprising receiving a search query from a user including at least another portion of the name of the named entity.

5. The computer readable storage medium of claim 4, the operations comprising generating search results based on the search query, wherein the search results include the one or more additional images.

6. The computer readable storage medium of claim 1, wherein the named entity is a person.

7. The computer readable storage medium of claim 6, wherein identifying the named entity comprises identifying the name of the named entity from a query log or a web document.

8. The computer readable storage medium of claim 1, the operations comprising executing an image search to identify all images associated with the named entity, wherein identifying the representative face from the set of three or more images comprises identifying the representative face from the images retrieved from the execution of the image search.

9. The computer readable storage medium of claim 1, the operations comprising labeling the face model with at least a portion of the name of the named entity.

10. The computer readable storage medium of claim 1, wherein using the face model to identify the one or more additional images comprises comparing the face model to a collection of images to identify the one or more additional images.

11. The computer readable storage medium of claim 10, the operations comprising associating the one or more additional images with the named entity.

12. The computer readable storage medium of claim 1 wherein building a face model comprises constructing a composite face model built from a sum of the faces in the training set.

13. The computer readable storage medium of claim 1 wherein building the face model comprises building a complex face model.

14. The computer readable storage medium of claim 13 wherein building the complex face model comprises building the complex face model using oriented PCA or Kernal OPCA.

15. The computer readable storage medium of claim 1, the operations comprising:

identifying the named entity as an entity that is eligible for generation of a face model, the face model being used to identify un-tagged images of the named entity; and

collecting the set of images of the named entity including collecting one or more images from the Web, wherein the textual description of the image includes at least a portion of a name of the named entity.

16. The computer readable storage medium of claim 15, wherein collecting the set of images of the named entity including collecting one or more images from the Web comprises collecting the set of images of the named entity including collecting one or more images from the Web, of the at least one of the images including a textual description of the image derived from metadata specific to only that image.

17. The computer readable storage medium of claim 15, the operations comprising measuring eligibility for the named entity using one or more criteria selected from a group comprising popularity, ranking, search frequency or minimum number of occurrences in web documents.

18. The computer readable storage medium of claim 1 wherein identifying the representative face image from the set of three or more images by comparing each detected face in each of the images in the set of images to each of the other detected faces from the images to identify faces that match the representative face image associated with a named entity, the at least one of the images in the set of images including the textual description that includes at least the portion of the name of the named entity comprises identifying the representative face image from the set of three or more images, the at least one of the images in the set of images including metadata that includes at least the portion of the name of the named entity.

19. A system, comprising:

a data processing apparatus; and

a non-transitory computer readable storage medium in data communication with the data processing apparatus and storing instructions executable by the data processing apparatus and upon such execution cause the data processing apparatus to perform operations comprising;

identifying a representative face image from a set of three or more images by comparing each detected face in each of the images in the set of images to each of the other detected faces from the images to identify faces that match the representative face image associated with a named entity, at least one of the images in the set of images including a textual description that includes at least a portion of a name of the named entity;

determining a number of faces in the images in the set of images that match the representative face image;

determining that the number of faces that match the representative face image satisfies a threshold value;

in response to determining that the number of faces that match the representative face image satisfies the threshold value, building a face model using the representative face image including identifying a training set that includes the representative face image and all other images that match the representative face image from the images in the set of images; and

using the face model to identify one or more additional images representing the named entity from images not included in the set of images.

20. A method comprising:

identifying, by one or more computers, a representative face image from a set of three or more images by comparing each detected face in each of the images in the set of images to each of the other detected faces from the images to identify faces that match the representative face image associated with a named entity, at least one of the images in the set of images including a textual description that includes at least a portion of a name of the named entity;

determining, by at least one of the one or more computers, a number of faces in the images in the set of images that match the representative face image;

determining, by at least one of the one or more computers, that the number of faces that match the representative face image satisfies a threshold value;

in response to determining that the number of faces that match the representative face image satisfies the threshold value, building, by at least one of the one or more computers, a face model using the representative face image including identifying a training set that includes the representative face image and all other images that match the representative face image from the images in the set of images; and

using, by at least one of the one or more computers, the face model to identify one or more additional images representing the named entity from images not included in the set of images.

21. The method of claim 20 comprising storing the face model associated with the named entity in memory.