Multimedia database for use over networks

• US 5,983,218 A
• Filed: 06/30/1997
• Issued: 11/09/1999
• Est. Priority Date: 06/30/1997
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Syeda-Mahmood, Tanveer F.
• Primary Examiner(s): Kulik, Paul V.
• Assistant Examiner(s): Corrielus, Jean M.
• Application Number:
  US08/886,052
• Time in Patent Office: 862 Days
• Field of Search: 707/104, 707/100, 707/5, 707/102, 707/1, 707/2, 707/3, 707/4, 706/45, 395/200.32
• US Class Current: 707/769
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: Y10S 707/913 Multimedia
• CPC Class Codes: Y10S 707/915 Image
• CPC Class Codes: Y10S 707/917 Text
• CPC Class Codes: Y10S 707/956 Hierarchical
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Disclosed is a multimedia database for use in distributed network environments. A query processing module transforms user queries into search transformations that can be used for indexing; an attentional selection module records salient information represented in images by a hierarchy of feature maps, saliency maps and combined saliency maps; a declarative memory comprises active modules that update the representations acquired over time for the same images, as well as across images, to cluster, categorize and organize information extracted from different images by the attentional selection module; and an indexing mechanism utilizes the search transformations from the query processing module to search the declarative memory for an answer to a user query. The multimedia database is used in distributed network environments wherein a server can select and coordinate information flow between multiple database sites and user sites, provide effective multimedia representation, create indexes on multimedia content of said database sites to derive a set of target multimedia database sites and, enable database queries descriptions and final search results to be conveyed to a user, records information needed for database selection, generates useful query patterns based on memorization of responses patterns from various databases based on user queries, and directs interactive dialogue between a user and multimedia databases.

Claims

1. A database using text and multimedia, the database containing at least one image with salient information represented therein, comprising:

a) a query processing module for transforming user queries into search transformations that are usable in both searching and indexing information in the database;

b) an attentional selection module for extracting the salient information represented in the at least one image contained within the multimedia database, said attentional selection module automatically inferring the salient information represented in the at least one image;

c) a declarative memory, communicating with said attentional selection module, for categorizing, organizing, and recording the salient information extracted from the at least one image by said attentional selection module;

d) an indexer, communicating with both said query processing module and said declarative memory, for using the search transformations from said query processing module to search said declarative memory for an answer to a user query wherein answers to said queries are conveyed back to a user who can confirm to what extent said answer is relevant to said user'"'"'s query and a newly formed representation is conveyed back to said declarative memory and combined with existing representations.

2. The database of claim 1, wherein said declarative memory further comprises active modules for updating representations acquired over time for the at least one image.

3. The database of claim 1, in which media data is retrieved, wherein the retrieved media data is reprocessed by said attentional selection module to find salient information pertaining to at least one of the user queries.

4. The database of claim 1, in which the database undergoes creation, wherein said attentional selection module extracts information content during database creation and during indexing.

5. The database of claim 1, wherein the salient information is configured within the at least one image to facilitate said automatic inferring.

6. The database of claim 5, wherein the salient information includes feature maps, saliency maps and combined saliency maps.

7. The database of claim 6, wherein the feature maps, the saliency maps, and combined saliency maps are arranged hierarchically.

8. A method for use with a database containing at least one image with salient information represented therein and including a declarative memory, comprising:

a) transforming user queries into search transformations that are usable in both searching and indexing information in the database;

b) extracting the salient information from the at least one image, said extracting including automatically inferring the salient information represented in the at least one image;

c) categorizing, organizing and recording the extracted salient information for the declarative memory; and

d) using the search transformations to search the declarative memory for an answer to a user query wherein answers to said queries are conveyed back to a user who can confirm to what extent said answer is relevant to said user'"'"'s query.

9. The method of claim 8, further comprising updating representations acquired over time for the at least one image.

10. The method of claim 8, further comprising configuring the salient information represented in the at least one image as a hierarchy of feature maps, saliency maps and combined saliency maps.

11. The method of claim 8, in which the database undergoes creation, further comprising extracting information during both database creation and said (d).