Auto-tagging in geo-social networking system

• US 9,292,518 B2
• Filed: 07/11/2014
• Issued: 03/22/2016
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Tseng, Erick
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): YANG, WEI WEN
• Application Number:
  US14/328,722
• Publication Number:
  US 20140337341A1
• Time in Patent Office: 620 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

In one embodiment, a social networking system automatically tags one or more users to an image file by creating a list of potential matches, and selecting a subset of potential matches based on location, asking a first user to confirm the subset of potential matches, and tagging one or more matched users to the image file.

Claims

1. A method comprising:

by a computing device, accessing a social-networking information item and a media file associated with a first user, the social-networking information item associated with a social graph of a social-networking system,

wherein the social graph comprises a plurality of nodes and edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between the nodes,

wherein at least one node of the social graph corresponds to the first user, and

wherein attributes of the social-networking information item comprise a list identifying one or more second users associated with the social-networking information item, the one or more second users each associated with at least one node of the social graph having a limited degree of separation with the first user;

by the computing device, determining based on a facial-recognition algorithm a matching score of each of one or more of the second users to each of one or more faces in the media file;

by the computing device, for each of the second users with a matching score above a threshold value to a face in the media file;

comparing first spatio-temporal information of the media file with second spatio-temporal information of the second user; and

adjusting the matching score of the second user to the face based on the comparison;

by the computing device, determining one or more potential matches between the second users and the faces in the media file based on the adjusted matching scores; and

by the computing device, tagging the media file with information identifying the social-networking information item based on one or more of the potential matches between the second users and the faces in the media file.

2. The method of claim 1 wherein the media file is a photograph with associated metadata in digital form or a video with associated metadata in digital form.

3. The method of claim 1 wherein the second spatio-temporal information of the second user comprises:

one or more location data with time stamps as provided by a client computing device of the second user;

one or more calendar entries of the second user;

or

one or more geo-social networking data of the second user.

4. The method of claim 1 wherein the first spatio-temporal information of the media file comprises:

metadata associated with the media file comprising location data with a time stamp;

or

geo-social networking data of the first user associated with the media file.

5. The method of claim 1 further comprising:

presenting one or more of the potential matches to the first user for confirmation.

6. The method of claim 1 wherein the social-networking information item comprises:

a calendar entry, wherein the calendar entry was sent to the second users;

or

a check-in, wherein the second users are associated with the check-in.

7. The method of claim 1 further comprising, by the computing device, tagging the media file with spatio-temporal information of the social-networking information item.

8. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

access a social-networking information item and a media file associated with a first user, the social-networking information item associated with a social graph of a social-networking system,

wherein the social graph comprises a plurality of nodes and edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between the nodes,

wherein at least one node of the social graph corresponds to the first user, and

wherein attributes of the social-networking information item comprise a list identifying one or more second users associated with the social-networking information item, the one or more second users each associated with at least one node of the social graph having a limited degree of separation with the first user;

determine based on a facial-recognition algorithm a matching score of each of one or more of the second users to each of one or more faces in the media file;

for each of the second users with a matching score above a threshold value to a face in the media file;

compare first spatio-temporal information of the media file with second spatio-temporal information of the second user; and

adjust the matching score of the second user to the face based on the comparison;

determine one or more potential matches between the second users and the faces in the media file based on the adjusted matching; and

tag the media file with information identifying the social-networking information item based on one or more of the potential matches between the second users and the faces in the media file.

9. The system of claim 8 wherein the media file is a photograph with associated metadata in digital form or a video with associated metadata in digital form.

10. The system of claim 8 wherein the second spatio-temporal information of the second user comprises:

one or more location data with time stamps as provided by a client computing device of the second user;

one or more calendar entries of the second user;

or

one or more geo-social networking data of the second user.

11. The system of claim 8 wherein the first spatio-temporal information of the media file comprises:

metadata associated with the media file comprising location data with a time stamp;

or

geo-social networking data of the first user associated with the media file.

12. The system of claim 8 wherein the processors are further operable when executing the instructions to:

present one or more of the potential matches to the first user for confirmation.

13. The system of claim 8 wherein the social-networking information item comprises:

a calendar entry, wherein the calendar entry was sent to the second users;

or

a check-in, wherein the second users are associated with the check-in.

14. The system of claim 8 wherein the processors are further operable when executing the instructions to tag the media file with spatio-temporal information of the social-networking information item.

15. One or more computer-readable non-transitory storage media embodying software that is operable when executed by one or more computing devices to:

access a social-networking information item and a media file associated with a first user, the social-networking information item associated with a social graph of a social-networking system,

wherein the social graph comprises a plurality of nodes and edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between the nodes,

wherein at least one node of the social graph corresponds to the first user, and

wherein attributes of the social-networking information item comprise a list identifying one or more second users associated with the social-networking information item, the one or more second users each associated with at least one node of the social graph having a limited degree of separation with the first user;

determine based on a facial-recognition algorithm a matching score of each of one or more of the second users to each of one or more faces in the media file;

for each of the second users with a matching score above a threshold value to a face in the media file;

compare first spatio-temporal information of the media file with second spatio-temporal information of the second user; and

adjust the matching score of the second user to the face based on the comparison;

determine one or more potential matches between the second users and the faces in the media file based on the adjusted matching scores; and

tag the media file with information identifying the social-networking information item based on one or more of the potential matches between the second users and the faces in the media file.

16. The media of claim 15 wherein the media file is a photograph with associated metadata in digital form or a video with associated metadata in digital form.

17. The media of claim 15 wherein the second spatio-temporal information of the second user comprises:

one or more location data with time stamps as provided by a client computing device of the second user;

one or more calendar entries of the second user;

or

one or more geo-social networking data of the second user.

18. The media of claim 15 wherein the first spatio-temporal information of the media file comprises:

metadata associated with the media file comprising location data with a time stamp;

or

geo-social networking data of the first user associated with the media file.

19. The media of claim 15 wherein the software is further operable when executed by the computing devices to:

present one or more of the potential matches to the first user for confirmation.

20. The media of claim 15 wherein the social-networking information item comprises:

a calendar entry, wherein the calendar entry was sent to the second users;

or

a check-in, wherein the second users are associated with the check-in.