Face Recognition Based on Spatial and Temporal Proximity

• US 20120250950A1
• Filed: 03/29/2011
• Published: 10/04/2012
• Est. Priority Date: 03/29/2011
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Papakipos, Matthew Nicholas, Papakipos, Phaedra
• Granted Patent: US 9,317,530 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/535 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/179 metadata assisted face recognition

Abstract

In one embodiment, a social networking system determines one or more individuals matching one or more faces in an image file of a still image or a video sequence, associated with a first user based on the one or more individuals'"'"' spatial and temporal proximity to the image file, and presents the matched individuals to the first user.

Claims

1. A method, comprising:

accessing an image file associated with a first user;

identifying one or more faces in an image corresponding to the image file;

generating a list of candidates comprising one or more individuals in temporal and spatial proximity to spatio-temporal data associated with the image file;

for at least a first face in the one or more identified faces in the image, determining one or more individuals from the list of candidates as a match to the first face; and

presenting the one or more matched individuals to the first user.

2. The method of claim 1 wherein the list of candidates is further limited to individuals within a threshold social proximity to the first user.

3. The method of claim 1 wherein the determining one or more individuals from the list of candidates as a match to the first face further comprises:

applying a facial recognition algorithm to the first face and a set of face images associated with the list of candidates to compute a matching score between the first face and each face image in the set of face images; and

determining one or more individuals corresponding to the set of face images as a match to the first face based on the matching scores.

4. The method of claim 1 wherein the presenting the one or more matched individuals to the first user further comprises:

accessing one or more data stores for information associated with the one or more matched individuals;

for each identified face with one or more matched individuals, generating a frame comprising information associated with the one or more individuals matching the identified face; and

displaying the one or more frames overlaying the image.

5. The method of claim 4 wherein each frame of the one or more frames further comprises one or more selectable icons.

6. The method of claim 1 wherein the image file further comprises a photographic picture and associated metadata in digital form.

7. The method of claim 1 wherein the image file further comprises a moving picture and associated metadata in digital form.

8. A system comprising:

a memory;

one or more processors; and

a non-transitory, storage medium storing computer-readable instructions operative, when executed, to cause the one or more processors to;

access an image file associated with a first user;

identify one or more faces in an image corresponding to the image file;

generate a list of candidates comprising one or more individuals in temporal and spatial proximity to spatio-temporal data associated with the image file;

for at least a first face in the one or more identified faces in the image, determine one or more individuals from the list of candidates as a match to the first face; and

present the one or more matched individuals to the first user.

9. The system of claim 8 wherein the list of candidates is further limited to individuals within a threshold social proximity to the first user.

10. The system of claim 8 wherein to determine one or more individuals from the list of candidates as a match to the first face further comprises instructions operable to cause the one or more processors to:

apply a facial recognition algorithm to the first face and a set of face images associated with the list of candidates to compute a matching score between the first face and each face image in the set of face images; and

determine one or more individuals corresponding to the set of face images as a match to the first face based on the matching scores.

11. The system of claim 8 wherein to present the one or more matched individuals to the first user further comprises instructions operable to cause the one or more processors to:

access one or more data stores for information associated with the one or more matched individuals;

for each identified face with one or more matched individuals, generate a frame comprising information associated with the one or more individuals matching the identified face; and

display the one or more frames overlaying the image.

12. The system of claim 11 wherein each frame of the one or more frames further comprises one or more selectable icons.

13. The system of claim 8 wherein the image file further comprises a photographic picture and associated metadata in digital form.

14. The system of claim 8 wherein the image file further comprises a moving picture and associated metadata in digital form.

15. One or more computer-readable tangible storage media embodying software operable when executed by one or more computing devices to:

access an image file associated with a first user;

identify one or more faces in an image corresponding to the image file;

generate a list of candidates comprising one or more individuals in temporal and spatial proximity to spatio-temporal data associated with the image file;

for at least a first face in the one or more identified faces in the image, determine one or more individuals from the list of candidates as a match to the first face; and

present the one or more matched individuals to the first user.

16. The media of claim 15 wherein the list of candidates is further limited to individuals within a threshold social proximity to the first user.

17. The media of claim 15 wherein to determine one or more individuals from the list of candidates as a match to the first face further comprises software operable when executed by the one or more computing devices to:

apply a facial recognition algorithm to the first face and a set of face images associated with the list of candidates to compute a matching score between the first face and each face image in the set of face images; and

determine one or more individuals corresponding to the set of face images as a match to the first face based on the matching scores.

18. The media of claim 15 wherein to present the one or more matched individuals to the first user further comprises software operable when executed by the one or more computing devices to:

access one or more data stores for information associated with the one or more matched individuals;

for each identified face with one or more matched individuals, generate a frame comprising information associated with the one or more individuals matching the identified face; and

display the one or more frames overlaying the image.

19. The media of claim 18 wherein each frame of the one or more frames further comprises one or more selectable icons.

20. The media of claim 15 wherein the image file further comprises a moving picture and associated metadata in digital form.