Face recognition based on spatial and temporal proximity

• US 9,317,530 B2
• Filed: 03/29/2011
• Issued: 04/19/2016
• Est. Priority Date: 03/29/2011
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Papakipos, Phaedra, Papakipos, Matthew Nicholas
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/074,743
• Publication Number:
  US 20120250950A1
• Time in Patent Office: 1,848 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/535 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

In one embodiment, a social networking system determines one or more individuals matching one or more faces in an image file of a still image or a video sequence, associated with a first user based on the one or more individuals'"'"' spatial and temporal proximity to the image file, and presents the matched individuals to the first user.

Claims

1. A method comprising:

by one or more computing devices, accessing an image file associated with a first user;

by one or more computing devices, detecting a face in an image corresponding to the image file;

by one or more computing devices, accessing a social-networking system associated with the first user and a plurality of second users, wherein;

the social-networking system comprises a graph that comprises a plurality of nodes and edges connecting the nodes;

at least one of the nodes corresponds to the first user; and

for each of the second users, at least one of the nodes corresponds to the second user; and

by one or more computing devices, determining one or more candidates among the second users to be matched to the face, wherein the determination is based at least in part on;

a comparison of temporal and spatial proximity of the second users, as indicated by the social-networking system, to a time and place associated with the image with a threshold temporal and spatial proximity; and

a comparison of degrees of separation between the second users and the first user indicated by the social-networking system with a threshold degrees of separation.

2. The method of claim 1, further comprising presenting the candidates to the first user for the first user to match to the face.

3. The method of claim 1, wherein determining one or more candidates among the second users to be matched to the face is further based on application of a facial-recognition algorithm to the face.

4. The method of claim 2, wherein presenting the candidates to the first user for the first user to match to the face comprises:

accessing one or more data stores for information associated with one or more of the candidates;

generating one or more frames comprising information associated with the one or more of the candidates; and

displaying one or more of the frames at least partially overlying the image.

5. The method of claim 4, wherein one or more of the frames comprises one or more selectable icons.

6. The method of claim 1, wherein the image file comprises:

a photograph and associated metadata in digital form;

or

a video and associated metadata in digital form.

7. The method of claim 3, wherein determining one or more candidates among the second users to be matched to the face comprises:

identifying a subset of the second users based on;

a comparison of temporal and spatial proximity of the second users, as indicated by the social-networking system, to the time and place associated with the image with a threshold temporal and spatial proximity; and

a comparison of degrees of separation between the second users and the first user indicated by the social-networking system with a threshold degrees of separation; and

by the facial-recognition algorithm, comparing the face in the image with faces of the second users in the subset.

8. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, wherein the processors are operable when executing the instructions to;

access an image file associated with a first user;

detect a face in an image corresponding to the image file;

access a social-networking system associated with the first user and a plurality of second users, wherein;

the social-networking system comprises a graph that comprises a plurality of nodes and edges connecting the nodes;

at least one of the nodes corresponds to the first user; and

for each of the second users, at least one of the nodes corresponds to the second user; and

determine one or more candidates among the second users to be matched to the face, wherein the determination is based at least in part on;

a comparison of temporal and spatial proximity of the second users, as indicated by the social-networking system, to a time and place associated with the image with a threshold temporal and spatial proximity; and

a comparison of degrees of separation between the second users and the first user indicated by the social-networking system with a threshold degrees of separation.

9. The system of claim 8, wherein the processors are further operable when executing the instructions to present the candidates to the first user for the first user to match to the face.

10. The system of claim 8, wherein determining one or more candidates among the second users to be matched to the face is further based on application of a facial-recognition algorithm to the face.

11. The system of claim 9, wherein presenting the candidates to the first user for the first user to match to the face comprises:

accessing one or more data stores for information associated with one or more of the candidates;

generating one or more frames comprising information associated with the one or more of the candidates; and

displaying one or more of the frames at least partially overlying the image.

12. The system of claim 11, wherein one or more of the frames comprises one or more selectable icons.

13. The system of claim 8, wherein the image file comprises:

a photograph and associated metadata in digital form;

or

a video and associated metadata in digital form.

14. The system of claim 10, wherein determining one or more candidates among the second users to be matched to the face comprises:

identifying a subset of the second users based on;

a comparison of temporal and spatial proximity of the second users, as indicated by the social-networking system, to the time and place associated with the image with a threshold temporal and spatial proximity; and

a comparison of degrees of separation between the second users and the first user indicated by the social-networking system with a threshold degrees of separation; and

by the facial-recognition algorithm, comparing the face in the image with faces of the second users in the subset.

15. One or more non-transitory computer-readable storage media embodying software that is operable when executed to:

access an image file associated with a first user;

detect a face in an image corresponding to the image file;

access a social-networking system associated with the first user and a plurality of second users, wherein;

the social-networking system comprises a graph that comprises a plurality of nodes and edges connecting the nodes;

at least one of the nodes corresponds to the first user; and

for each of the second users, at least one of the nodes corresponds to the second user; and

determine one or more candidates among the second users to be matched to the face, wherein the determination is based at least in part on;

a comparison of temporal and spatial proximity of the second users, as indicated by the social-networking system, to a time and place associated with the image with a threshold temporal and spatial proximity; and

a comparison of degrees of separation between the second users and the first user indicated by the social-networking system with a threshold degrees of separation.

16. The media of claim 15, wherein the software is further operable when executed to present the candidates to the first user for the first user to match to the face.

17. The media of claim 15, wherein determining one or more candidates among the second users to be matched to the face is further based on application of a facial-recognition algorithm to the face.

18. The media of claim 16, wherein presenting the candidates to the first user for the first user to match to the face comprises:

accessing one or more data stores for information associated with one or more of the candidates;

generating one or more frames comprising information associated with the one or more of the candidates; and

displaying one or more of the frames at least partially overlying the image.

19. The media of claim 18, wherein one or more of the frames comprises one or more selectable icons.

20. The media of claim 17, wherein determining one or more candidates among the second users to be matched to the face comprises:

identifying a subset of the second users based on;

a comparison of temporal and spatial proximity of the second users, as indicated by the social-networking system, to the time and place associated with the image with a threshold temporal and spatial proximity; and

a comparison of degrees of separation between the second users and the first user indicated by the social-networking system with a threshold degrees of separation; and

by the facial-recognition algorithm, comparing the face in the image with faces of the second users in the subset.

21. The media of claim 15, wherein the image file comprises:

a photograph and associated metadata in digital form;

or

a video and associated metadata in digital form.