Computer-implemented method for performing similarity searches

• US 8,989,451 B2
• Filed: 09/14/2012
• Issued: 03/24/2015
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vanhoucke, Vincent, Gokturk, Salih Burak, Anguelov, Dragomir, Lee, Kuang-chih, Shah, Munjal, Tengli, Ashwin
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/618,226
• Publication Number:
  US 20130136319A1
• Time in Patent Office: 921 Days
• Field of Search: 382/115, 382/118
• US Class Current: 382/115
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

A similarity search may be performed on the image of a person, using visual characteristics and information that is known about the person. The search identifies images of other persons that are similar in appearance to the person in the image.

Claims

1. A computer-implemented method for determining face similarity, the method comprising:

identifying a person from an image associated with an account of a user of a social network;

identifying at least one of text or metadata associated with the image by the social network;

determining one or more or more classifications corresponding to an ethnicity, a gender or an age classification of the person in the image, using the at least one of text and metadata;

in response to the user initiating a search for other users of the social network, performing a similarity search using the person of the image, in order to identify images of other persons that are similar in appearance to the person in the image and that share the one or more classifications of the person in the image; and

providing, to the user, a search result based on at least one identified image from the similarity search.

2. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes determining biographical information about the person in the image using the text and metadata.

3. The method of claim 1, wherein determining one or more of the classifications further comprises using image data from the image of the person to determine one or more of the classifications corresponding to the ethnicity, the gender or the age classification.

4. The method of claim 1, wherein identifying a person from an image includes detecting a face of the person from the image, including one or more features of the face.

5. The method of claim 1, wherein performing the similarity search includes formulating a search query to include image data that represents one or more visual characteristics of a face of the person.

6. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes identifying a text contained in a file that includes the image of the person.

7. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes identifying a label or a tag provided with the image.

8. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes identifying a text contained in the image.

10. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes determining biographical information about the person in the image using the text and metadata.

11. The method of claim 1, wherein determining one or more of the classifications further comprises using image data from the image of the person to determine one or more of the classifications corresponding to the ethnicity, the gender or the age classification.

12. The method of claim 1, wherein identifying a person from an image includes detecting a face of the person from the image, including one or more features of the face.

13. The method of claim 1, wherein performing the similarity search includes formulating a search query to include image data that represents one or more visual characteristics of a face of the person.

14. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes identifying a text contained in a file that includes the image of the person.

15. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes identifying a label or a tag provided with the image.

16. The method of claim 1, wherein identifying at least one of text or metadata associated with the image includes identifying a text contained in the image.

9. A system comprising one or more computers configured to perform operations comprising:

identifying a person from an image associated with an account of a user of a social network;

identifying at least one of text or metadata associated with the image by the social network;

determining one or more or more classifications corresponding to an ethnicity, a gender or an age classification of the person in the image, using the at least one of text and metadata;

in response to the user initiating a search for other users of the social network, performing a similarity search using the person of the image, in order to identify images of other persons that are similar in appearance to the person in the image and that share the one or more classifications of the person in the image; and

providing, to the user, a search result based on at least one identified image from the similarity search.

17. A non-transitory computer-readable medium for determining face similarity, the computer-readable medium comprising instructions, that when executed by one or more processors, cause the one or more processors to perform steps comprising:

identifying a person from an image associated with an account of a user of a social network;

identifying at least one of text or metadata associated with the image by the social network;

determining one or more or more classifications corresponding to an ethnicity, a gender or an age classification of the person in the image, using the at least one of text and metadata;

in response to the user initiating a search for other users of the social network, performing a similarity search using the person of the image, in order to identify images of other persons that are similar in appearance to the person in the image and that share the one or more classifications of the person in the image; and

providing, to the user, a search result based on at least one identified image from the similarity search.