Tag Suggestions for Images on Online Social Networks

• US 20130262588A1
• Filed: 05/30/2013
• Published: 10/03/2013
• Est. Priority Date: 03/20/2008
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Barak, Dan, Taigman, Yaniv N., Hirsch, Gil, van Dijk, Jorn Martinus Johannes, Stoop, Dirk John
• Granted Patent: US 9,143,573 B2
• Time in Patent Office: Days
• US Class Current: 709/204
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/179 metadata assisted face recognition
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/306 User profiles
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

In one embodiment, a method includes accessing an image portraying at least a first person, accessing a social graph, determining a social-graph affinity for a first set of users, determining a facial-recognition scores for the first set of users based on the social-graph affinity for each user and a facial-representation associated with each user, where the facial-representation for each user is compared with the image, and generating one or more tag suggestions for the first person portrayed in the image based on the facial-recognition scores.

Claims

1. A method comprising, by a computing device:

accessing an image associated with an online social network, wherein the image portrays at least a first person;

accessing a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between them, the nodes comprising;

a first node corresponding to the image; and

a plurality of second nodes corresponding to a plurality of users associated with the online social network, respectively;

determining a social-graph affinity for each user from a first set of users of the plurality of users;

determining, for each user in the first set of users, a facial-recognition score with respect to the first person portrayed in the image, wherein the facial-recognition score is based at least in part on;

the social-graph affinity determined for each user of the first set of users; and

a facial-representation associated with each user of the first set of users, wherein the facial-representation associated with each user is compared with the image; and

generating one or more tag suggestions for the first person portrayed in the image based on the determined facial-recognition scores, wherein each tag suggestion corresponds to a particular user of the first set of users.

2. The method of claim 1, further comprising receiving one or more characters of a character string as a first user at a client system enters the character string into a graphical user interface of a webpage associated with the image.

3. The method of claim 2, further comprising searching the social graph for one or more second nodes that substantially match the character string.

4. The method of claim 2, wherein determining the facial-recognition score is further based at least in part on the character string.

5. The method of claim 2, further comprising sending to the client system one or more of the tag suggestions for display on the client system as the first user enters the character string into the graphical user interface, wherein the tag suggestions are displayed to the user in a drop-down menu on the graphical user interface, the display of the tag suggestions enabling the first user to select one of the tag suggestions for automatic entry into the graphical user interface.

6. The method of claim 1, wherein generating the one or more tag suggestions comprises identifying each user in the first set of users having a facial-recognition score greater than a threshold facial-recognition score, wherein each identified user is generated as one of the tag suggestions.

7. The method of claim 1, further comprising:

receiving, from a first user associated with the online social network, a request to view the image; and

sending the image to a client system for display to the first user.

8. The method of claim 1, further comprising sending the tag suggestions for display on a page associated with the image, wherein the tag suggestions are displayed in a drop-down menu on the page, and wherein the drop-down menu enables a first user select one of the tag suggestions.

9. The method of claim 8, further comprising:

receiving from a first user a selection of one of the tag suggestions; and

generating a tag corresponding to the particular user corresponding to the selected tag suggestion.

10. The method of claim 1, further comprising rendering the image with a tag-label for the tag, wherein the tag-label comprises a description of the particular user corresponding to the tag.

11. The method of claim 1, wherein the facial-recognition score for each user measure a probability that the user matches the first person portrayed in the image.

12. The method of claim 1, wherein the image comprises one or more tags corresponding to one or more second users of the plurality of users, respectively, and wherein the social-graph affinity for each user from the first set of users is based at least in part on the degree of separation between the second node corresponding to the user from the first set of users and the one or more second nodes corresponding to the one or more second users, respectively.

13. The method of claim 1, wherein the image is associated with a first set of images, and wherein the social-graph affinity for each user from the first set of users is based at least in part on a tag-history information of the other images in the first set of images.

14. The method of claim 1, wherein the social-graph affinity for each user from the first set of users is based at least in part on a tag-history information of the image.

15. The method of claim 14, wherein tag-history information comprises one or more of the users tagged in the image;

the user who created each tag in the image;

the time since each tag was created;

the history of particular users being tagged together;

a geographic location associated with the image;

or any combination thereof.

16. The method of claim 1, wherein the social-graph affinity for each user from the first set of users is based at least in part on time-decay information.

17. The method of claim 1, wherein the social-graph affinity for each user from the first set of users is based at least in part on a degree of separation between the second node corresponding to the user and the first node.

18. The method of claim 1, wherein the image is a digital photograph.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

access an image associated with an online social network, wherein the image portrays at least a first person;

access a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between them, the nodes comprising;

a first node corresponding to the image; and

a plurality of second nodes corresponding to a plurality of users associated with the online social network, respectively;

determine a social-graph affinity for each user from a first set of users of the plurality of users;

determine, for each user in the first set of users, a facial-recognition score with respect to the first person portrayed in the image, wherein the facial-recognition score is based at least in part on;

the social-graph affinity determined for each user of the first set of users; and

a facial-representation associated with each user of the first set of users, wherein the facial-representation associated with each user is compared with the image; and

generate one or more tag suggestions for the first person portrayed in the image based on the determined facial-recognition scores, wherein each tag suggestion corresponds to a particular user of the first set of users.

20. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to;

access an image associated with an online social network, wherein the image portrays at least a first person;

access a social graph comprising a plurality of nodes and a plurality of edges connecting the nodes, each of the edges between two of the nodes representing a single degree of separation between them, the nodes comprising;

a first node corresponding to the image; and

a plurality of second nodes corresponding to a plurality of users associated with the online social network, respectively;

determine a social-graph affinity for each user from a first set of users of the plurality of users;

determine, for each user in the first set of users, a facial-recognition score with respect to the first person portrayed in the image, wherein the facial-recognition score is based at least in part on;

the social-graph affinity determined for each user of the first set of users; and

a facial-representation associated with each user of the first set of users, wherein the facial-representation associated with each user is compared with the image; and

generate one or more tag suggestions for the first person portrayed in the image based on the determined facial-recognition scores, wherein each tag suggestion corresponds to a particular user of the first set of users.