Media Tag Recommendation Technologies

• US 20100228691A1
• Filed: 03/03/2009
• Published: 09/09/2010
• Est. Priority Date: 03/03/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hua, Xian-Sheng, Wu, Lei, Yang, Linjun
• Granted Patent: US 8,239,333 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Technologies for recommending relevant tags for the tagging of media based on one or more initial tags provided for the media and based on a large quantity of other tagged media. Sample media as candidates for recommendation are provided by a set of weak rankers based on corresponding relevance measures in semantic and visual domains. The various samples provided by the weak rankers are then ranked based on relative order to provide a list of recommended tags for the media. The weak rankers provide sample tags based on relevance measures including tag co-occurrence, tag content correlation, and image-conditioned tag correlation.

Claims

1. A tag recommendation system comprising:

a tag selection element operable to select a plurality of candidate media/tag pairs wherein a candidate tag of each of the candidate media/tag pairs exhibits raw co-occurrence with a target tag of a target media/tag pair;

a tag co-occurrence normalization element operable to process each of the plurality of candidate media/tag pairs against the target media/tag pair resulting in corresponding tag co-occurrence (“

TC”

) relevance measures wherein the TC relevance measures are in a semantic domain;

a visual language modeling element operable to generate a candidate visual language model (“

VLM”

) for each candidate tag of the plurality of candidate media/tag pairs and a target VLM for a target tag of the target media/tag pair, and further operable to process each of the candidate VLMs against the target VLM resulting in corresponding tag content correlation (“

TCC”

) relevance measures wherein such relevance measures are in a visual domain;

a visual correlation element operable to process each of the candidate VLMs against the target VLM resulting in corresponding image-conditioned tag correlation (“

ITC”

) relevance measures wherein the ITC relevance measures are in a visual domain; and

a multi-domain combining element operable to combine one or more TC tag lists indicated by the TC relevance measures with one or more TCC tag lists indicated by the TCC relevance measures and with one or more ITC tag lists indicated by the ITC relevance measures, and operable to form a list of recommended tags from the combined one or more TC tags and the one or more TCC tags and the one or more ITC tags.

2. The system of claim 1 further comprising a table generation element operable to generate an inverted tag table wherein the inverted tag table is comprised of the candidate tags wherein each candidate tag is associated with media instances listed as attributes of the candidate tag.

3. The system of claim 1 wherein the ITC relevance measures include a probability that the target tag is descriptive of a target media of the target tag/media pair.

4. The system of claim 1 wherein the ITC relevance measures indicate a visual similarly between a target tag of the target tag/media pair and candidate tag of the plurality of candidate media/tag pairs.

5. The system of claim 1 wherein the ITC relevance measures are based at least in part on a unigram visual language model of the target media/tag pair.

6. The system of claim 1 wherein the ITC relevance measures are based at least in part on a bigram visual language model of the target media/tag pair.

7. The system of claim 1 wherein the ITC relevance measures are based at least in part on a trigram visual language model of the target media/tag pair.

8. The system of claim 1 wherein the ITC relevance measures are based at least in part on a distance between the target tag and at least one candidate tag.

9. The system of claim 8 wherein the distance is defined as the inter-product of two likelihood vectors.

10. A method performed in a computing environment of generating an image-conditioned tag correlation (“

ITC”

) relevance measure, the method comprising;

generating a target visual language model (“

VLM”

) for a target tag t_i of a target media/tag pair;

generating a candidate VLM for a candidate tag t_j of a plurality of candidate media/tag pairs wherein the candidate tag t_j co-occurs with the target tag t_i of the target media/tag pair;

defining a similarity between a target media instance of the target media/tag pair and a candidate media instance of the candidate media/tag pair as likelihood

defining a distance between t_i and t_j that is the inter-product of two likelihood vectors of likelihood between a target media of the target media/tag pair and tags t_i and t_j; and

defining the ITC relevance measure as the inverse of the distance wherein the ICT relevance measure is in a visual domain.

11. The method of claim 10 wherein the likelihood is defined as (x)∝

P(x|VLM_t^m) where x is the target media instance, and where t is a tag associated with x, and where VLM is a VLM generated based on t, and where m indicates a form of the VLM, and where P is the probability that tag t is descriptive of the target media instance x.

12. The method of claim 10 wherein the distance is an asymmetric distance defined as

13. The method of claim 10 wherein the distance is a symmetric distance defined as

14. A method performed in a computing environment of combining multi-domain relevance measures provided by a plurality of weak rankers, the method comprising:

defining an instance space that is comprised of a plurality of sets, each set being a set of instances provided by a distinct weak ranker of a plurality of the weak rankers, each instance of a set of instances being a sample tag from a media/tag pair;

generating a mapping function for each weak ranker, each mapping function operable to map a corresponding instance from the instance space to a ranking space R; and

mapping each instance using the corresponding mapping function from the instance space to a ranking space R.

15. The method of claim 14 wherein the mapping establishes a relative ordering between the instances in the ranking space R.

16. The method of claim 15 wherein the relative ordering forms an ordered recommendation of tags for a target media/tag pair.

17. The method of claim 14 wherein at least one of the plurality of the weak rankers provides one set of instances in a semantic domain and a second one of the plurality of the weak rankers provides a second set of instances in a visual domain.

18. The method of claim 17 wherein the at least one of the plurality of the weak rankers provides a first ranking of the one set of instances in a semantic domain and a second ranking of the second one of the plurality of the weak rankers provides a second set of instances in a visual domain.

19. The method of claim 14 wherein a first weak ranker of the plurality of the weak rankers provides a first set of instances based on tag co-occurrence (“

TC”

) relevance measures, and a second weak ranker of the plurality of the weak rankers provides a second set of instances based on tag content correlation (“

TCC”

) relevance measures, and a third weak ranker of the plurality of the weak rankers provides a third set of instances based on image-conditioned tag correlation (“

ITC”

) relevance measures.

20. The method of claim 14 wherein a first weak ranker of the plurality of the weak rankers provides a first ranking of the first set of instances based on tag co-occurrence (“

TC”

) relevance measures, and a second weak ranker of the plurality of the weak rankers provides a second ranking of the second set of instances based on tag content correlation (“

TCC”

) relevance measures, and a third weak ranker of the plurality of the weak rankers provides a third ranking of the third set of instances based on image-conditioned tag correlation (“

ITC”

) relevance measures.