Auto tagging in geo-social networking system

• US 10,176,199 B2
• Filed: 12/18/2015
• Issued: 01/08/2019
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Tseng, Erick
• Primary Examiner(s): Yang, Weiwen
• Application Number:
  US14/975,354
• Publication Number:
  US 20160103852A1
• Time in Patent Office: 1,117 Days
• Field of Search: None
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

In one embodiment, a social networking system automatically tags one or more users to an image file by creating a list of potential matches, and selecting a subset of potential matches based on location, asking a first user to confirm the subset of potential matches, and tagging one or more matched users to the image file.

Claims

1. A method comprising:

by a computing device associated with a social-networking system, accessing a social-networking information item and a media file associated with a first user, wherein attributes of the social-networking information item comprise a list identifying one or more second users associated with the social-networking information item and a relationship between each of the one or more second users and the first user;

by the computing device, determining based on a facial-recognition algorithm a matching score of each of one or more of the second users to each of one or more faces in the media file;

by the computing device, for each of the second users with a matching score above a threshold value to a face in the media file;

comparing first spatio-temporal information of the media file with second spatio-temporal information of the second user, the second spatio-temporal information being determined based at least in part on one or more attributes of an event that the second user is registered as attending, wherein;

the event is created and configured by a user using an application supported by the social-networking system;

the event is stored in an event database associated with the social-networking system; and

the attributes of the event comprise a time and date of the event and a location of the event specified by the user; and

adjusting the matching score of the second user to the face based on the comparison and the relationship between the second user and the first user;

by the computing device, determining one or more potential matches between the second users and the faces in the media file based on the adjusted matching scores; and

by the computing device, tagging the media file with information identifying the social-networking information item based on one or more of the potential matches between the second users and the faces in the media file.

2. The method of claim 1 wherein the media file is a photograph with associated metadata in digital form or a video with associated metadata in digital form.

3. The method of claim 1 wherein the second spatio-temporal information of the second user comprises:

one or more location data with time stamps as provided by a client computing device of the second user;

one or more calendar entries of the second user;

or

one or more geo-social networking data of the second user.

4. The method of claim 1 wherein the first spatio-temporal information of the media file comprises:

metadata associated with the media file comprising location data with a time stamp;

or

geo-social networking data of the first user associated with the media file.

5. The method of claim 1 further comprising:

presenting one or more of the potential matches to the first user for confirmation.

6. The method of claim 1 wherein the social-networking information item comprises:

a calendar entry, wherein the calendar entry was sent to the second users;

or

a check-in, wherein the second users are associated with the check-in.

7. The method of claim 1 further comprising, by the computing device, tagging the media file with spatio-temporal information of the social-networking information item.

8. A system comprising:

one or more processors associated with a social-networking system; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

access a social-networking information item and a media file associated with a first user, wherein attributes of the social-networking information item comprise a list identifying one or more second users associated with the social-networking information item and a relationship between each of the one or more second users and the first user;

determine based on a facial-recognition algorithm a matching score of each of one or more of the second users to each of one or more faces in the media file;

for each of the second users with a matching score above a threshold value to a face in the media file;

compare first spatio-temporal information of the media file with second spatio-temporal information of the second user, the second spatio-temporal information being determined based at least in part on one or more attributes of an event that the second user is registered as attending, wherein

the event is created and configured by a user using an application supported by the social-networking system;

the event is stored in an event database associated with the social-networking system; and

the attributes of the event comprise a time and date of the event and a location of the event specified by the user; and

adjust the matching score of the second user to the face based on the comparison and the relationship between the second user and the first user;

determine one or more potential matches between the second users and the faces in the media file based on the adjusted matching; and

tag the media file with information identifying the social-networking information item based on one or more of the potential matches between the second users and the faces in the media file.

9. The system of claim 8 wherein the media file is a photograph with associated metadata in digital form or a video with associated metadata in digital form.

10. The system of claim 8 wherein the second spatio-temporal information of the second user comprises:

one or more location data with time stamps as provided by a client computing device of the second user;

one or more calendar entries of the second user;

or

one or more geo-social networking data of the second user.

11. The system of claim 8 wherein the first spatio-temporal information of the media file comprises:

metadata associated with the media file comprising location data with a time stamp;

or

geo-social networking data of the first user associated with the media file.

12. The system of claim 8 wherein the processors are further operable when executing the instructions to:

present one or more of the potential matches to the first user for confirmation.

13. The system of claim 8 wherein the social-networking information item comprises:

a calendar entry, wherein the calendar entry was sent to the second users;

or

a check-in, wherein the second users are associated with the check-in.

14. The system of claim 8 wherein the processors are further operable when executing the instructions to tag the media file with spatio-temporal information of the social-networking information item.

15. One or more computer-readable non-transitory storage media associated with a social-networking system embodying software that is operable when executed by one or more computing devices to:

access a social-networking information item and a media file associated with a first user, wherein attributes of the social-networking information item comprise a list identifying one or more second users associated with the social-networking information item and a relationship between each of the one or more second users and the first user;

determine based on a facial-recognition algorithm a matching score of each of one or more of the second users to each of one or more faces in the media file;

for each of the second users with a matching score above a threshold value to a face in the media file;

compare first spatio-temporal information of the media file with second spatio-temporal information of the second user, the second spatio-temporal information being determined based at least in part on one or more attributes of an event that the second user is registered as attending, wherein

the event is created and configured by a user using an application supported by the social-networking system;

the event is stored in an event database associated with the social-networking system; and

the attributes of the event comprise a time and date of the event and a location of the event specified by the user; and

adjust the matching score of the second user to the face based on the comparison and the relationship between the second user and the first user;

determine one or more potential matches between the second users and the faces in the media file based on the adjusted matching scores; and

tag the media file with information identifying the social-networking information item based on one or more of the potential matches between the second users and the faces in the media file.

16. The media of claim 15 wherein the media file is a photograph with associated metadata in digital form or a video with associated metadata in digital form.

17. The media of claim 15 wherein the second spatio-temporal information of the second user comprises:

one or more location data with time stamps as provided by a client computing device of the second user;

one or more calendar entries of the second user;

or

one or more geo-social networking data of the second user.

18. The media of claim 15 wherein the first spatio-temporal information of the media file comprises:

metadata associated with the media file comprising location data with a time stamp;

or

geo-social networking data of the first user associated with the media file.

19. The media of claim 15 wherein the software is further operable when executed by the computing devices to:

present one or more of the potential matches to the first user for confirmation.

20. The media of claim 15 wherein the social-networking information item comprises:

a calendar entry, wherein the calendar entry was sent to the second users;

or

a check-in, wherein the second users are associated with the check-in.