Context and content based information processing for multimedia segmentation and indexing
First Claim
1. A data processing device for processing an information signal comprising:
- at least one stage, wherein a first stage includes, a first layer having first plurality of nodes for extracting content attributes from the information signal; and
a second layer having at least one node for determining context information for the at least one node using the content attributes of selected nodes in an other layer or a next stage, and for integrating certain ones of the content attributes and the context information at the at least one node.
2 Assignments
0 Petitions
Accused Products
Abstract
Method and system are disclosed for information processing, for example, for multimedia segmentation, indexing and retrieval. The method and system includes multimedia, for example audio/visual/text (A/V/T), integration using a probabilistic framework. Both, multimedia content and context information are represented and processed via the probabilistic framework. This framework is represented, for example, by a Bayesian network and hierarchical priors, which is graphically described by stages, each having a set of layers with each layer including a number of nodes representing content or context information. At least the first layer of the first stage is processes multimedia content information such as objects in the A/V/T domains, or combinations of thereof. The other layers of the various stages describe multimedia context information, as further described below. Each layer is a Bayesian network, wherein nodes of each layer explain certain characteristics of the next “lower” layer and/or “lower” stages. Together, the nodes and connections there between form an augmented Bayesian network. Multimedia context is the circumstance, situation, underlying structure of the multimedia information (audio, visual, text) being processed. The multimedia information (both content and context) is combined at different levels of granularity and level of abstraction within the layers and stages.
-
Citations
35 Claims
-
1. A data processing device for processing an information signal comprising:
at least one stage, wherein a first stage includes, a first layer having first plurality of nodes for extracting content attributes from the information signal; and
a second layer having at least one node for determining context information for the at least one node using the content attributes of selected nodes in an other layer or a next stage, and for integrating certain ones of the content attributes and the context information at the at least one node. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34)
-
15. A method for processing an information signal comprising the steps of:
segmenting and indexing the information signal using a probabilistic framework, said framework including at least one stage having a plurality of layers with each layer having a plurality of nodes, wherein the segmenting and indexing includes, extracting content attributes from the information signal for each node of a first layer;
determining context information, in a second layer, using the content attributes of selected nodes in an other layer or a next stage; and
integrating certain content attributes and the context information for at lest one node in the second layer.
-
23. A computer-readable memory medium including code for processing an information signal, the code comprising:
framework code said framework including at least one stage having a plurality of layers with each layer having a plurality of nodes, wherein the segmenting and indexing includes, feature extracting code to extract content attributes from the information signal for each node of a first stage;
probability generating code to determine context information, in a node of a stage, using the content attributes of selected nodes in other layers or context information of a next stage; and
integrating code to combine certain content attributes and the context information for a node.
-
35. An apparatus for processing an information signal, the apparatus comprising:
-
a memory which stores process steps; and
a processor which executes the process steps stored in the memory so as (i) to use at least one stage with a plurality of layers with at least one node in each layer, (ii) extract content attributes from the information signal for each node of a first layer, (iii) to determine context information, in a second layer, using the content attributes of selected nodes in an other layer or context information of a next stage; and
(iv) to combine certain content attributes and the context information for a node.
-
Specification