Computer-implemented method for performing similarity searches

• US 9,542,419 B1
• Filed: 03/20/2015
• Issued: 01/10/2017
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vanhoucke, Vincent O., Gokturk, Salih Burak, Anguelov, Dragomir, Lee, Kuang-chih, Shah, Munjal, Tengli, Ashwin
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US14/664,006
• Time in Patent Office: 662 Days
• Field of Search: 382/115, 382/118
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

A similarity search may be performed on the image of a person, using visual characteristics and information that is known about the person. The search identifies images of other persons that are similar in appearance to the person in the image.

Claims

1. A computer-implemented method, comprising:

identifying i) a person from an image and ii) biographical information associated with the person from the image;

determining a biographical classification based on the biographical information associated with the person from the image;

performing a similarity search for images based on the person of the image;

in response to the similarity search, identifying a plurality of images of one or more other persons that are similar in appearance to the person from the image;

filtering the identified plurality of images based on the biographical classification to identify a subset of the plurality of images; and

providing search results based on the subset of the plurality of images.

2. The method of claim 1, wherein the one or more classifications correspond to an age classification of the person from the image.

3. The method of claim 1, wherein identifying the biographical information associated with the person from the image includes identifying at least one of text or metadata associated with the image, the image associated with an account of the person of a social network.

4. The method of claim 3, wherein identifying the at least one of text or metadata associated with the image includes identifying a label or a tag provided with the image.

5. The method of claim 3, wherein identifying the at least one of text or metadata associated with the image includes identifying text contained in the image.

6. The method of claim 1, wherein identifying the person from the image includes detecting a face of the person from the image, including one or more features of the face.

7. The method of claim 6, wherein identifying the plurality of images includes formulating a search query to include image data that represents the one or more features of the face of the person from the image.

8. A system comprising one or more computers configured to perform operations comprising:

identifying i) a person from an image and ii) biographical information associated with the person from the image;

determining a biographical classification based on the biographical information associated with the person from the image;

performing a similarity search for images based on the person of the image;

in response to the similarity search, identifying a plurality of images of one or more other persons that are similar in appearance to the person from the image;

filtering the identified plurality of images based on the biographical classification to identify a subset of the plurality of images; and

providing search results based on the subset of the plurality of images.

9. The system of claim 8, wherein the one or more classifications correspond to an age classification of the person from the image.

10. The system of claim 8, wherein identifying the biographical information associated with the person from the image includes identifying at least one of text or metadata associated with the image, the image associated with an account of the person of a social network.

11. The system of claim 10, wherein identifying the at least one of text or metadata associated with the image includes identifying a label or a tag provided with the image.

12. The system of claim 10, wherein identifying the at least one of text or metadata associated with the image includes identifying text contained in the image.

13. The system of claim 8, wherein identifying the person from the image includes detecting a face of the person from the image, including one or more features of the face.

14. The system of claim 13, wherein identifying the plurality of images includes formulating a search query to include image data that represents the one or more features of the face of the person from the image.

15. Non-transitory computer storage media encoded with one or more computer programs, the one or more computer programs comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising

identifying i) a person from an image and ii) biographical information associated with the person from the image;

determining a biographical classification based on the biographical information associated with the person from the image;

performing a similarity search for images based on the person of the image;

in response to the similarity search, identifying a plurality of images of one or more other persons that are similar in appearance to the person from the image;

filtering the identified plurality of images based on the biographical classification to identify a subset of the plurality of images; and

providing search results based on the subset of the plurality of images.

16. The computer storage media of claim 15, wherein the one or more classifications correspond to an age classification of the person from the image.

17. The computer storage media of claim 15, wherein identifying the biographical information associated with the person from the image includes identifying at least one of text or metadata associated with the image, the image associated with an account of the person of a social network.