VIDEO SEARCH RE-RANKING VIA MULTI-GRAPH PROPAGATION
First Claim
1. A computer-implemented process for ranking the relevance of video returned in response to a search, comprising:
- inputting search results of video shots with text-based relevance scores received in response to a text string search query;
creating a set of hierarchical graphs based on different semantic concepts, with the video shots as vertices and hyperlinks, that exploit conceptual similarity and visual similarity between the video shots, as edges;
applying a topic-sensitive ranking procedure to propagate the text-based relevance scores of the video shots through the hyperlinks in each hierarchical graph of the set of hierarchical graphs; and
aggregating the results of the topic-sensitive ranking procedure from the set of hierarchical graphs to determine the final ranking of the video shot search results.
2 Assignments
0 Petitions
Accused Products
Abstract
A video search re-ranking via multi-graph propagation technique employing multimodal fusion in video search is presented. It employs not only textual and visual features, but also semantic and conceptual similarity between video shots to rank or re-rank the search results received in response to a text-based search query. In one embodiment, the technique employs an object-sensitive approach to query analysis to improve the baseline result of text-based video search. The technique then employs a graph-based approach to text-based search result ranking or re-ranking. To better exploit the underlying relationship between video shots, the re-ranking scheme simultaneously leverages textual relevancy, semantic concept relevancy, and low-level-feature-based visual similarity. The technique constructs a set of graphs with the video shots as vertices, and the conceptual and visual similarity between video shots as hyperlinks. A modified topic-sensitive PageRank algorithm is then applied to these graphs to determine the overall relevancy ranking.
-
Citations
20 Claims
-
1. A computer-implemented process for ranking the relevance of video returned in response to a search, comprising:
-
inputting search results of video shots with text-based relevance scores received in response to a text string search query; creating a set of hierarchical graphs based on different semantic concepts, with the video shots as vertices and hyperlinks, that exploit conceptual similarity and visual similarity between the video shots, as edges; applying a topic-sensitive ranking procedure to propagate the text-based relevance scores of the video shots through the hyperlinks in each hierarchical graph of the set of hierarchical graphs; and aggregating the results of the topic-sensitive ranking procedure from the set of hierarchical graphs to determine the final ranking of the video shot search results. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computer-implemented process for ranking the relevance of video shots returned in response to a search, comprising:
-
inputting video shot search results with text-based relevance scores received in response to a text string search query; determining a first expansion of query terms by expanding the number of query terms by segmenting the test string search query and computing modified text-based relevance scores using the first expansion of the number of query terms; determining a second expansion of query terms by expanding the number of query terms by performing name entity generalization; further modifying the modified text-based relevance scores by identifying targeted objects in the text string search query and the first and second expansions of query terms by assigning greater weight to video shot search results of query terms that represent the targeted objects; and using the further modified text-based relevance scores and the first and second expansion of query terms to determine the final ranking of the video shot search results. - View Dependent Claims (14, 15, 16, 17, 20)
-
-
18. A system for ranking the results of video data returned in response to a search query, comprising:
-
a general purpose computing device; a computer program comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to, input a ranked set of video shot search results received in response to a text-based search query; using the ranked set of video shot search results, construct a set of graphs based on semantic similarity with video shots as vertices and semantic concept similarity and visual similarity between video shots as hyperlinks; and apply a topic sensitive ranking procedure to the set of graphs to re-rank the ranked set of video shots. - View Dependent Claims (19)
-
Specification