Tool for automatically mapping multimedia annotations to ontologies
First Claim
1. An interactive multimedia delivery system dynamically linking contextual information with multimedia documents, said system retrieving said contextual information by searching an ontology and one or more databases over a network, said ontology comprising one or more nodes, said system comprising:
- a. a learning data preparation component accessing mappings of annotations in said ontology and fusing annotations mapped in each of said nodes to form learning instances;
b. an intelligent inverted index creating a data structure based on the following calculated statistics for said learning instances;
term frequency (tf), inverse document frequency (idf), and contribution frequency (cf);
c. a retriever receiving a request for new annotations associated with multimedia documents, said retriever utilizing said inverted index to retrieve and rank most relevant nodes for said received new annotations, said ranking determined based upon a weight, wtij, contributed to a particular node in said ontology by the occurrence of a word i in a learning instance j;
d. an information retriever extracting information related to said requested annotations from said most relevant nodes and said one or more databases over said network, and e. a contextual information linker linking multimedia content with said extracted information.
1 Assignment
0 Petitions
Accused Products
Abstract
A tool for learning to relate annotations and transcript of a multimedia sequence to nodes in a formally or semi-formally represented ontology covering a broad range of possible multimedia documents. The device includes learning data preparation that involves certain special techniques for deriving data from the past mappings of annotations to nodes in an ontology, building inverted indices maintaining certain special statistics and a retriever that exploits these special statistics to rank the relevance of the nodes in an ontology for a given a set of new annotations.
-
Citations
32 Claims
-
1. An interactive multimedia delivery system dynamically linking contextual information with multimedia documents, said system retrieving said contextual information by searching an ontology and one or more databases over a network, said ontology comprising one or more nodes, said system comprising:
-
a. a learning data preparation component accessing mappings of annotations in said ontology and fusing annotations mapped in each of said nodes to form learning instances;
b. an intelligent inverted index creating a data structure based on the following calculated statistics for said learning instances;
term frequency (tf), inverse document frequency (idf), and contribution frequency (cf);
c. a retriever receiving a request for new annotations associated with multimedia documents, said retriever utilizing said inverted index to retrieve and rank most relevant nodes for said received new annotations, said ranking determined based upon a weight, wtij, contributed to a particular node in said ontology by the occurrence of a word i in a learning instance j;
d. an information retriever extracting information related to said requested annotations from said most relevant nodes and said one or more databases over said network, and e. a contextual information linker linking multimedia content with said extracted information. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method for searching an ontology of mapped multimedia annotations for appropriate annotations for one more multimedia documents, said ontology comprising one or more nodes, said method comprising the steps of:
-
a. receiving a request for searching and extracting one or more annotations related to said multimedia documents from said ontology;
b. identifying nodes in said ontology that are relevant to said multimedia documents, said nodes further comprising fused learning instances formed by fusing annotations in each of said nodes, said identification based upon using special statistics including term frequency, inverse document frequency and contribution frequency;
c. extracting information from said identified relevant nodes, and d. dynamically linking said extracted information with said multimedia documents. - View Dependent Claims (8, 9, 10)
-
-
11. A method for retrieving contextual information by searching an ontology and one or more databases, said method comprising:
-
receiving a request for contextual information;
retrieving from an ontology, with automatically mapped annotations, said requested contextual information using information retrieval statistics;
retrieving said requested contextual information from one or more databases, and rendering an integrated presentation comprising audio, video, or graphics and said retrieved contextual information. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for retrieving contextual information by searching for a selected multimedia representation, said system comprising:
-
a server, said server receiving requests for contextual information for a selected multimedia representation;
one or more databases associated with said server, wherein said server retrieves both from its own ontology, said ontology having automatically mapped annotations, and from said one or more databases said requested contextual information, and renders said retrieved information as an integrated presentation comprising said multimedia and said retrieved contextual information. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26)
-
-
27. A method for automatically mapping annotations to ontologies, said method comprising the steps of:
-
extracting annotations from a multimedia document segment;
mapping said extracted multimedia document segment to an appropriate node in said ontology;
comparing to other related content mapped to said appropriate node, and integrating said related content with said extracted multimedia document segment. - View Dependent Claims (28, 29, 30, 31)
-
-
32. An article of manufacture comprising a computer usable medium having computer readable program code embodied therein which searches an ontology of mapped multimedia annotations for appropriate annotations for one more multimedia documents, said ontology comprising one or more nodes, said article comprising:
-
a. computer readable program code receiving a request for searching and extracting one or more annotations related to said multimedia documents from said ontology;
b. computer readable program code identifying nodes in said ontology that are relevant to said multimedia documents, said nodes further comprising fused learning instances formed by fusing annotations in each of said nodes, said identification based upon using special statistics including term frequency, inverse document frequency and contribution frequency;
c. computer readable program code extracting information from said identified relevant nodes, and d. computer readable program code dynamically linking said extracted information with said multimedia documents.
-
Specification