Facial recognition with social network aiding

• US 9,208,177 B2
• Filed: 02/20/2014
• Issued: 12/08/2015
• Est. Priority Date: 08/07/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Petrou, David, Rabinovich, Andrew, Adam, Hartwig
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US14/185,392
• Publication Number:
  US 20140172881A1
• Time in Patent Office: 656 Days
• Field of Search: 382115-118, 382/103.115, 348/169, 348/170, 348/171, 348/172, 348/352, 707/748, 707/769
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics. Finally, at least one person identifier from the list is sent to the requester.

Claims

1. A computer-implemented method comprising:

obtaining a query image that was submitted to an image-based search system by a requesting user;

obtaining one or more images that are identified as matching the query image;

identifying people that are associated with at least one of the one or more images that are identified as matching the query image;

for each person that is associated with at least one of the one or more images that are identified as matching the query image, obtaining a score that reflects a level of social connectivity between the person and the requesting user that submitted the query image to the image-based search system;

selecting one or more of the people that are associated with at least one of the one or more images that are identified as matching the query image based on the scores that reflect the level of social connectivity between each person and the requesting user that submitted the query image to the image-based search system; and

providing a respective identifier of one or more of the selected people to the requesting user.

2. The computer-implemented method of claim 1, further comprising determining that at least a portion of the query image includes one or more faces.

3. The computer-implemented method of claim 2, wherein identifying the people that are associated with at least one of the one or more images that are identified as matching the query image further comprises:

identifying one or more additional images associated with the requesting user that include one or more additional faces;

comparing the face of the query image with the one or more additional faces of the one more additional images;

based on comparing the face of the query image with the one or more additional faces of the one more additional images, determining that the face of the query image matches at least one of the one or more additional faces of the one or more additional images; and

identifying an additional identifier tag of the at least one of the one or more additional faces of the one or more additional images.

4. The computer-implemented method of claim 1, further comprising:

determining that the query image includes a plurality of faces; and

receiving, from the requesting user, a selection of one of the plurality of faces,

wherein obtaining the one or more images that are identified as matching the query image further comprises obtaining the one or more images based on the selected face.

5. The computer-implemented method of claim 1, further comprising:

identifying user information associated with the requesting user; and

identifying one or more image sources based on the identified user information of the requesting user,

wherein obtaining the one or more images that are identified as matching the query image further comprises obtaining the one or more images from the one or more image sources.

6. The computer-implemented method of claim 5, wherein identifying the user information associated with the requesting user further comprises identifying one or more social networking profiles associated with the user.

7. The computer-implemented method of claim 1, further comprising:

identifying, for one or more of the selected people, a social networking profile associated with the selected person; and

providing, for one or more of the selected people, a portion of the social networking profile of the selected person to the requesting user.

8. The computer-implemented method of claim 7, further comprising:

ranking each selected person that is associated with the images that are identified as matching the query image based on the score associated with each selected person,

wherein providing, for one or more of the selected people, the portion of the social networking profile of the selected person to the requesting user further comprises providing, for one or more of the selected people, the portion of the social networking profile of the selected person to the requesting user based on a rank of the selected person.

9. The computer-implemented method of claim 8, wherein providing, for one or more of the selected people, the portion of the social networking profile of the selected person to the requesting user based on the rank of the selected person further comprises increasing a quantity of content of the social networking profile of each selected person that is provided to the requesting user based on an increasing rank of each selected user.

10. The computer-implemented method of claim 1, further comprising:

identifying, for one or more of the selected people, social networking profile information associated with the selected person;

comparing, for one or more of the selected people, the social networking profile information associated with the selected person with social networking profile information associated with the requesting user;

based on the comparing, for one or more of the selected people, identifying common social networking profile information to the selected person and the requesting user; and

providing, for one or more of the selected people, the identified common social networking profile information associated with the selected person and the requesting user.

11. The computer-implemented method of claim 1, further comprising providing the query image to one or more of the selected people that are associated with the images that are identified as matching the query image.

12. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

obtaining a query image that was submitted to an image-based search system by a requesting user;

obtaining one or more images that are identified as matching the query image;

identifying people that are associated with at least one of the one or more images that are identified as matching the query image;

for each person that is associated with at least one of the one or more images that are identified as matching the query image, obtaining a score that reflects a level of social connectivity between the person and the requesting user that submitted the query image to the image-based search system;

selecting one or more of the people that are associated with at least one of the one or more images that are identified as matching the query image based on the scores that reflect the level of social connectivity between each person and the requesting user that submitted the query image to the image-based search system; and

providing a respective identifier of one or more of the selected people to the requesting user.

13. The system of claim 12, the operations further comprising determining that at least a portion of the query image includes one or more faces.

14. The system of claim 12, the operations further comprising:

determining that the query image includes a plurality of faces; and

receiving, from the requesting user, a selection of one of the plurality of faces,

wherein obtaining the one or more images that are identified as matching the query image further comprises obtaining the one or more images based on the selected face.

15. The system of claim 12, the operations further comprising:

identifying user information associated with the requesting user; and

identifying one or more image sources based on the identified user information of the requesting user,

wherein obtaining the one or more images that are identified as matching the query image further comprises obtaining the one or more images from the one or more image sources.

16. The system of claim 15, wherein identifying the user information associated with the requesting user further comprises identifying one or more social networking profiles associated with the user.

17. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

obtaining a query image that was submitted to an image-based search system by a requesting user;

obtaining one or more images that are identified as matching the query image;

identifying people that are associated with at least one of the one or more images that are identified as matching the query image;

for each person that is associated with at least one of the one or more images that are identified as matching the query image, obtaining a score that reflects a level of social connectivity between the person and the requesting user that submitted the query image to the image-based search system;

selecting one or more of the people that are associated with at least one of the one or more images that are identified as matching the query image based on the scores that reflect the level of social connectivity between each person and the requesting user that submitted the query image to the image-based search system; and

providing a respective identifier of one or more of the selected people to the requesting user.

18. The computer-readable medium of claim 17, the operations further comprising determining that at least a portion of the query image includes one or more faces.

19. The computer-readable medium of claim 17, the operations further comprising:

determining that the query image includes a plurality of faces; and

receiving, from the requesting user, a selection of one of the plurality of faces,

wherein obtaining the one or more images that are identified as matching the query image further comprises obtaining the one or more images based on the selected face.

20. The computer-readable medium of claim 17, the operations further comprising:

identifying user information associated with the requesting user; and

identifying one or more image sources based on the identified user information of the requesting user,

wherein obtaining the one or more images that are identified as matching the query image further comprises obtaining the one or more images from the one or more image sources.