Method and system for the contextual display of image tags in a social network

• US 9,195,679 B1
• Filed: 08/11/2012
• Issued: 11/24/2015
• Est. Priority Date: 08/11/2011
• Status: Active Grant

• Current Assignee: Ikorongo Technology, LLC (Concert Technology Corporation)
• Original Assignee: Ikorongo Technology, LLC (Concert Technology Corporation)
• Inventors: Svendsen, Hugh Blake
• Primary Examiner(s): Christensen, Scott B
• Assistant Examiner(s): CONCANNON, SEAN D
• Application Number:
  US13/572,618
• Time in Patent Office: 1,200 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

Methods and systems for the contextual display of image tags in a social network are disclosed. According to an aspect, a method includes obtaining text for display with an image. The method also includes displaying the image in a first display area. Further, for a portion of the text, the method includes determining whether the portion is designated as a recommended tag or not. In response to determining that the portion is designated as the recommended tag, displaying the portion in a second display area and indicating that the portion is the recommended tag, and in response to determining that the portion is not designated as the recommended tag, displaying the portion in the second display area and indicating that the portion is not the recommended tag.

Claims

1. A method comprising:

receiving, by a server from a first device of a plurality of devices, a first image;

receiving, by a server from at least one of the plurality of devices, a plurality of annotations, each annotation comprising a multi-word comment associated with the first image;

identifying, by the server, a plurality of tags occurring in the annotations by;

receiving, by the server, information identifying a first word in the plurality of annotations as a current tag for the first image,

identifying, by the server, a target subject face corresponding to a target user in the first image,

receiving, by the server, target subject information, the target subject information comprising target identification information identifying the target user within a social network associated with the plurality of devices,

identifying, by the server based on the target subject information, a plurality of second images wherein the target user has been identified as a subject face in each of the plurality of second images,

deriving, by the server, a plurality of candidate tags from the plurality of second images,

determining, by the server, a plurality of candidate tag probabilities for the plurality of candidate tags derived from the plurality of second images based on a proximity between the first image and each of the plurality of second images, and

selecting, by the server as a second word occurring in the plurality of annotations, at least one of the plurality of candidate tags to apply as a recommended tag to the first image based on the candidate tag probabilities;

sending, by the server to a second device of the plurality of devices, the first image to present in a first area of a display of the second device;

sending, by the server to the second device of the plurality of devices, the plurality of annotations to present in a second area of the display of the second device, wherein,

the first word in the plurality of annotations is configured to be displayed in a first visual style and applied as a first hyperlink,

the second word in the plurality of annotations is configured to be displayed in a second visual style and applied as a second hyperlink, and

remaining words in the plurality of annotations, which are neither current tags nor recommended tags, are configured to be displayed in a third visual style distinct from the first visual style and the second visual style;

receiving, by the server from the second device, a first input selection of the first hyperlink;

changing, by the server based on the first input, a first state of the first word from the current tag to one from a group consisting of;

a remaining word and a recommended tag;

receiving, by the server from the second device, a second input selection of the second hyperlink; and

changing, by the server based on the second input, a second state of the second word from the recommended tag to a current tag.

2. The method of claim 1 wherein the server is a photosharing server.

3. The method of claim 1 wherein the server is a social network server.

4. The method of claim 1 further comprising:

receiving, by the server from the first device of the plurality of devices, information indicating a permission setting, the permission setting defining a right of the second device of the plurality of devices to apply the recommended tag based on a social distance between the first device and the second device, and

based on the permission setting, determining by the server the second visual style,

wherein ones of the plurality of annotations are received from the second device, the first device being associated with a first user and the second device being associated with a second user, the first user and second user separated by the social distance in the social network.

5. The method of claim 4 further comprising:

determining, by the server based on the social distance based permission setting, the second visual style.

6. The method of claim 1 wherein sending the plurality of annotations to present in the second area of the display of the second device further comprises:

configuring for presentation user identity information for each of the plurality of annotations, the user identity information identifying a user associated with a device of the plurality of devices having contributed the respective annotation.

7. The method of claim 6 wherein the user identity information is a user name associated with the user.

8. The method of claim 6 wherein the user identity information is a user image, the user image representing the user.

9. The method of claim 1 further comprising:

receiving, from the first device of the plurality of devices, information indicating a permission setting to be applied to the recommended tag, the permission setting defining a right of the second device of the plurality of devices to apply the recommended tag to the first image.

10. The method of claim 1 wherein at least one of the plurality of annotations is a sentence.

11. The method of claim 1 wherein selecting the second word in the plurality of annotations as the recommended tag further comprises:

determining the plurality of candidate tags based on an analysis of the plurality of annotations;

ranking the plurality of candidate tags based on their inverse occurrence rates;

modifying a candidate tag probability of at least one of the plurality of candidate tags based on at least one factor selected from the group consisting of;

social distance between an annotation contributor and a first user of the first device,

size of a subject face of the annotation contributor appearing in the first image,

geographical proximity of the annotation contributor to capture location of the first image at time of capture, and

social distance between the annotation contributor and one or more subjects appearing in first image.

12. The method of claim 1 wherein selecting the second word in the plurality of annotations as the recommended tag based on proximity further comprises:

determining the proximity based on one or more factors selected from the group consisting of;

organizational proximity, temporal proximity, geographical proximity, compositional proximity, event proximity, and subject face association.

13. The method of claim 1 further comprising:

receiving annotation rights information from the first device of the plurality of devices indicating whom may contribute annotations to the first image, the annotation rights information indicating at least one group of users selected from the group consisting of;

friends of a first user associated with the first device of the plurality of devices,

friends of the friends of the first user associated with the first device of the plurality of devices,

subjects appearing in the first image,

friends of the subjects appearing in the first image,

social network users geographically and temporally present at an event where the first image was captured, and

friends of social network users geographically and temporally present at an event where the first image was captured.

14. The method of claim 1 further comprising:

storing, in a database associated with the server, the first image and the plurality of second images;

storing, in the database associated with the server, the plurality of annotations;

storing, in the database associated with the server, the recommended tag; and

storing, in the database associated with the server, the current tag.

15. The method of claim 1 wherein determining the plurality of candidate tag probabilities for the plurality of candidate tags further comprises:

identifying a target annotation from the plurality of annotations associated with the plurality of second images;

identifying at least one candidate tag of the plurality of candidate tags from the target annotation;

adjusting the at least one candidate tag probability based on an inverse occurrence frequency of the at least one candidate tag; and

adjusting the at least one candidate tag probability based on a social distance between the target user and a user providing the first image from the first device.

16. The method of claim 1 wherein determining the proximity between the first image and each of the plurality of second images comprises:

determining an organizational proximity distance between the first image and each of the plurality of second images, wherein the organizational proximity distance is a function of a number of hops in a hierarchical storage structure between storage locations of the first image and each of the plurality of second images;

receiving an organizational proximity threshold value;

performing a comparison of the organizational proximity distance and the organizational proximity threshold value for each of the plurality of second images; and

based on the comparison, adjusting at least one candidate tag probability of at least one candidate tag.

17. The method of claim 1 wherein determining the proximity between the first image and each of the plurality of second images comprises:

determining a geographical proximity distance between the first image and each of the plurality of second images, wherein the geographical proximity distance is a function of a geographical distance between capture locations of the first image and each of the plurality of second images;

receiving a geographical proximity threshold value;

performing a comparison of the geographical proximity distance and the geographical proximity threshold value for each of the plurality of second images; and

based on the comparison, adjusting at least one candidate tag probability of at least one candidate tag.

18. The method of claim 1 wherein determining the proximity between the first image and each of the plurality of second images comprises:

determining a temporal proximity distance between the first image and each of the plurality of second images, wherein the temporal proximity distance is a function of a temporal distance between capture times of the first image and each of the plurality of second images;

receiving a temporal proximity threshold value;

performing a comparison of the temporal proximity distance and the temporal proximity threshold value for each of the plurality of second images; and

based on the comparison, adjusting at least one candidate tag probability of at least one candidate tag.

19. The method of claim 1 wherein determining the proximity between the first image and each of the plurality of second images comprises:

determining a compositional proximity distance between the first image and each of the plurality of second images, wherein the compositional proximity distance is a function of a number of commonly occurring subject faces between the first image and each of the plurality of second images;

receiving a compositional proximity threshold value;

performing a comparison of the compositional proximity distance and the compositional proximity threshold value for each of the plurality of second images; and

based on the comparison, adjusting at least one candidate tag probability of at least one candidate tag.

20. The method of claim 1 wherein determining the proximity between the first image and each of the plurality of second images comprises:

determining an event proximity distance between the first image and each of the plurality of second images, wherein the event proximity distance is a function of a temporal distance between capture times of the first image and each of the plurality of second images and a geographical distance between capture locations of the first image and each of the plurality of second images;

receiving an event proximity threshold value;

performing a comparison of the event proximity distance and the event proximity threshold value for each of the plurality of second images; and

based on the comparison, adjusting at least one candidate tag probability of at least one candidate tag.

21. The method of claim 1 wherein the first device and the second device are selected from the group consisting of:

the same device and two separate devices.

22. A computer program product, the computer program product stored on a non-transitory computer-readable storage medium and including instructions configured to cause a processor to:

receive, by a server from a first device of a plurality of devices, a first image;

receive, by a server from at least one of the plurality of devices, a plurality of annotations, each annotation comprising a multi-word comment associated with the first image;

where in order to identify, by the server, a plurality of tags occurring in the annotations, the processor is further operable to;

receive, by the server, information identifying a first word in the plurality of annotations as a current tag for the first image,

identify, by the server, a target subject face corresponding to a target user in the first image,

receive, by the server, target subject information, the target subject information comprising target identification information identifying the target user within a social network associated with the plurality of devices,

identify, by the server based on the target subject information, a plurality of second images wherein the target user has been identified as a subject face in each of the plurality of second images,

derive, by the server, a plurality of candidate tags from the plurality of second images,

determine, by the server, a plurality of candidate tag probabilities for the plurality of candidate tags derived from the plurality of second images based on a proximity between the first image and each of the plurality of second images, and

select, by the server as a second word occurring in the plurality of annotations, at least one of the plurality of candidate tags to apply as a recommended tag to the first image based on the candidate tag probabilities;

send, by the server to a second device of the plurality of devices, the first image to present in a first area of a display of the second device;

send, by the server to the second device of the plurality of devices, the plurality of annotations to present in a second area of the display of the second device, wherein,

the first word in the plurality of annotations is configured to be displayed in a first visual style and applied as a first hyperlink,

the second word in the plurality of annotations is configured to be displayed in a second visual style and applied as a second hyperlink, and

remaining words in the plurality of annotations, which are neither current tags nor recommended tags, are configured to be displayed in a third visual style distinct from the first visual style and the second visual style;

receive, by the server from the second device, a first input selection of the first hyperlink;

change, by the server based on the first input, a first state of the first word from the current tag to one from a group consisting of;

a remaining word and a recommended tag;

receive, by the server from the second device, a second input selection of the second hyperlink; and

change, by the server based on the second input, a second state of the second word from the recommended tag to a current tag.

23. A server comprising:

a communication interface facilitating communication between the server and a plurality of devices;

at least one processor operably in communication with the communication interface; and

memory storing software executable by the at least one processor, whereby the server is operable to;

receive, by a server from a first device of a plurality of devices, a first image;

receive, by a server from at least one of the plurality of devices, a plurality of annotations, each annotation comprising a multi-word comment associated with the first image;

where in order to identify, by the server, a plurality of tags occurring in the annotations, the processor is further operable to;

receive, by the server, information identifying a first word in the plurality of annotations as a current tag for the first image,

identify, by the server, a target subject face corresponding to a target user in the first image,

receive, by the server, target subject information, the target subject information comprising target identification information identifying the target user within a social network associated with the plurality of devices,

identify, by the server based on the target subject information, a plurality of second images wherein the target user has been identified as a subject face in each of the plurality of second images,

derive, by the server, a plurality of candidate tags from the plurality of second images,

determine, by the server, a plurality of candidate tag probabilities for the plurality of candidate tags derived from the plurality of second images based on a proximity between the first image and each of the plurality of second images, and

select, by the server as a second word occurring in the plurality of annotations, at least one of the plurality of candidate tags to apply as a recommended tag to the first image based on the candidate tag probabilities;

send, by the server to a second device of the plurality of devices, the first image to present in a first area of a display of the second device;

send, by the server to the second device of the plurality of devices, the plurality of annotations to present in a second area of the display of the second device, wherein,

the first word in the plurality of annotations is configured to be displayed in a first visual style and applied as a first hyperlink,

the second word in the plurality of annotations is configured to be displayed in a second visual style and applied as a second hyperlink, and

remaining words in the plurality of annotations, which are neither current tags nor recommended tags, are configured to be displayed in a third visual style distinct from the first visual style and the second visual style;

receive, by the server from the second device, a first input selection of the first hyperlink;

change, by the server based on the first input, a first state of the first word from the current tag to one from a group consisting of;

a remaining word and a recommended tag;

receive, by the server from the second device, a second input selection of the second hyperlink; and

change, by the server based on the second input, a second state of the second word from the recommended tag to a current tag.