Media tag recommendation technologies

• US 8,239,333 B2
• Filed: 03/03/2009
• Issued: 08/07/2012
• Est. Priority Date: 03/03/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yang, Linjun, Wu, Lei, Hua, Xian-Sheng
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): OLUDE AFOLABI, OLATOYOSI
• Application Number:
  US12/396,885
• Publication Number:
  US 20100228691A1
• Time in Patent Office: 1,253 Days
• Field of Search: 706/12, 382/224, 704/9
• US Class Current: 706/12
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Technologies for recommending relevant tags for the tagging of media based on one or more initial tags provided for the media and based on a large quantity of other tagged media. Sample media as candidates for recommendation are provided by a set of weak rankers based on corresponding relevance measures in semantic and visual domains. The various samples provided by the weak rankers are then ranked based on relative order to provide a list of recommended tags for the media. The weak rankers provide sample tags based on relevance measures including tag co-occurrence, tag content correlation, and image-conditioned tag correlation.

Claims

1. A tag recommendation system comprising:

a computing device;

a tag selection element implemented at least in part by the computing device and configured for selecting a plurality of candidate media/tag pairs wherein a candidate tag of each of the candidate media/tag pairs exhibits raw co-occurrence with a target tag of a target media/tag pair;

a tag co-occurrence normalization element configured for processing each of the plurality of candidate media/tag pairs against the target media/tag pair resulting in corresponding tag co-occurrence (“

TC”

) relevance measures wherein the TC relevance measures are in a semantic domain;

a visual language modeling element configured for generating a candidate visual language model (“

VLM”

) for each candidate tag of the plurality of candidate media/tag pairs and a target VLM for a target tag of the target media/tag pair, and further configured for processing each of the candidate VLMs against the target VLM resulting in corresponding tag content correlation (“

TCC”

) relevance measures wherein such relevance measures are in a visual domain;

a visual correlation element configured for processing each of the candidate VLMs against the target VLM resulting in corresponding image-conditioned tag correlation (“

ITC”

) relevance measures wherein the ITC relevance measures are in a visual domain, wherein the ITC relevance measures are based at least in part on a distance between the target tag and at least one candidate tag, and wherein the distance is an asymmetric distance or a symmetric distance, and wherein the asymmetric distance defined as

2. The system of claim 1 further comprising a table generation element configured for generating an inverted tag table wherein the inverted tag table is comprised of the candidate tags wherein each candidate tag is associated with media instances listed as attributes of the candidate tag.

3. The system of claim 1 wherein the ITC relevance measures include a probability that the target tag is descriptive of a target media of the target tag/media pair.

4. The system of claim 1 wherein the ITC relevance measures indicate a visual similarly between a target tag of the target tag/media pair and candidate tag of the plurality of candidate media/tag pairs.

5. The system of claim 1 wherein the ITC relevance measures are based at least in part on a unigram visual language model of the target media/tag pair.

6. The system of claim 1 wherein the ITC relevance measures are based at least in part on a bigram visual language model of the target media/tag pair.

7. The system of claim 1 wherein the ITC relevance measures are based at least in part on a trigram visual language model of the target media/tag pair.

8. The system of claim 1 wherein the distance is defined as the inter-product of two likelihood vectors.

9. A method for generating an image-conditioned tag correlation (“

ITC”

) relevance measure, the method comprising;

generating, by a computing device, a target visual language model (“

VLM”

) for a target tag t_i of a target media/tag pair;

generating a candidate VLM for a candidate tag t_j of a plurality of candidate media/tag pairs wherein the candidate tag t_j co-occurs with the target tag t_i of the target media/tag pair;

defining a similarity between a target media it stance of the target media/tag pair and a candidate media instance of the candidate media/tag pair as likelihood ;

defining a distance between t_i and t_j that is the inter-product of two likelihood vectors of likelihood between a target media of the target media/tag pair and tags t_i and t_j, wherein the distance is an asymmetric distance or a symmetric distance, and wherein the asymmetric distance defined as

10. The method of claim 9 wherein the likelihood is defined as (x)∝

P(x|VLM_t^m) where x is the target media instance, and where t is a tag associated with x, and where VLM is a VLM generated based on t, and where m indicates a form of the VLM, and where P is the probability that tag t is descriptive of the target media instance x.

11. A method for combining multi-domain relevance measures provided by a plurality of weak rankers, the method comprising:

defining an instance space that is comprised of a plurality of sets, each set being a set of instances provided by a distinct weak ranker of the plurality of the weak rankers, each instance of a set of instances being a sample tag from a media/tag pair, wherein a first weak ranker of the plurality of the weak rankers provides a first set of instances based on tag co-occurrence (“

TC”

) relevance measures, and a second weak ranker of the plurality of the weak rankers provides a second set of instances based on tag content correlation (“

TCC”

) relevance measures, and a third weak ranker of the plurality of the weak rankers provides a third set of instances based on image-conditioned tad correlation (“

ITC”

) relevance measures, wherein the ITC relevance measures are based at least in part on a distance between a target tag and at least one candidate tag, and wherein the distance is an asymmetric distance or a symmetric distance, and wherein the asymmetric distance defined as

12. The method of claim 11 wherein the mapping establishes a relative ordering between the instances in the ranking space R.

13. The method of claim 12 wherein the relative ordering forms an ordered recommendation of tags for the target media/tag pair.

14. The method of claim 11 wherein at least one of the plurality of the weak rankers provides one set of instances in a semantic domain and a second one of the plurality of the weak rankers provides a second set of instances in a visual domain.

15. The method of claim 14 wherein the at least one of the plurality of the weak rankers provides a first ranking of the one set of instances in a semantic domain and a second ranking of the second one of the plurality of the weak rankers provides a second set of instances in a visual domain.

16. The method of claim 11 wherein the first weak ranker of the plurality of the weak rankers provides a first ranking of the first set of instances based on the tag co-occurrence (“

TC”

) relevance measures, and the second weak ranker of the plurality of the weak rankers provides a second ranking of the second set of instances based on the tag content correlation (“

TCC”

) relevance measures, and the third weak ranker of the plurality of the weak rankers provides a third ranking of the third set of instances based on the image-conditioned tag correlation (“

ITC”

) relevance measures.