Method and a system for managing multimedia databases

• US 20030126147A1
• Filed: 10/11/2002
• Published: 07/03/2003
• Est. Priority Date: 10/12/2001
• Status: Active Grant

• Current Assignee: Commissariat a L'Energie Atomique (AES Corporation)
• Original Assignee: Commissariat a L'Energie Atomique (AES Corporation)
• Inventors: Adam, Pascal, Essafi, Hassane, Pic, Marc, Gayrard, Michel-Pierre
• Granted Patent: US 6,925,475 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

The management system comprises a set of specialized operational units with an acquisition module, a multimedia database, indexing and search agents that can be subdivided into subagents, an agent dedicated to managing information relating to the structure of documents, a module for analyzing and structuring document components, a user interface including a document indexing interface and a document search interface, a module for creating a multimedia database, and a module for opening a session. The management system also comprises a set of specialized functional units with an agent for arranging the various units of the system, agents for driving document indexing and searching, and an identifier manager. A platform provides interconnection and intercommunication between the specialized operational functional units.

Claims

1. / A system for managing a multimedia database, the system being characterized in that it comprises:

a) a set of specialized operational units comprising at least;

a1) an acquisition module;

a2) a multimedia database;

a3) at least one indexing agent that can be subdivided into at least two subagents each using a specific mode of processing;

a4) at least one search agent that can be subdivided into at least two subagents each using a specific mode of processing;

a5) an agent dedicated to managing information relating to document structure;

a6) a module for analyzing the content of an input multimedia document and for extracting and structuring components from the document;

a7) a user interface having;

a7-1) a document indexing interface; and

a7-2) a document search interface;

a8) a module for creating, starting, and destroying a multimedia database; and

a9) a module enabling a user to open a session and control the module for creating, starting, and destroying a multimedia database;

b) a set of specialized functional units comprising at least;

b1) an agent for arranging various units of the system;

b2) an agent for driving document indexing;

b3) an agent for driving document searching; and

b4) an identifier manager; and

c) an interconnection and intercommunication platform for the specialized operational units and the specialized functional units.

2. / A system according to claim 1, characterized in that said indexing agent that can be subdivided into at least two subagents is an agent serving to index objects of image type, and in that said search agent that can be subdivided into at least two subagents is an agent serving to search for objects of image type.

3. / A system according to claim 1, characterized in that said module for analyzing the content of an input multimedia document and for extracting the structure of document components comprises a first unit for processing and analyzing an image or a signal, for statistical analysis, for modelling, and for classification to group together the elements of an input document into homogeneous components, and a unit for producing the structure of the document or an object graph establishing the relationships between the various components of the document.

4. / A system according to claim 1, characterized in that it includes an indexing agent for indexing objects of text type and a search agent for searching for objects of text type.

5. / A system according to claim 1, characterized in that it includes an indexing agent for indexing objects of sound type and a search agent for searching for objects of sound type.

6. / A system according to claim 1, characterized in that it includes an analysis agent for extracting image objects, sound objects, and objects characterizing the dynamic behavior thereof, an indexing agent for indexing video type objects, and a search agent for searching for video type objects.

7. / A system according to claim 1, characterized in that the multimedia database comprises both references to documents to be archived and composite signatures constituted by characterizing components of archived documents and by descriptions of the structure of archived documents identifying the relationships between the components of archived documents.

8. / A system according to claim 1, characterized in that it comprises a plurality of modules for analyzing the content of an input multimedia document and for extracting and structuring components of the document, each of these modules for analyzing contents, extraction, and structuring being adapted to a specific type of document identified by the nature of its content (image, text, sound, video).

9. / A method of managing a multimedia database, the method being characterized in that it comprises the following steps:

creating an identifier allocator agent for associating a unique reference with each representative element of a multimedia document at each step of the processing;

acquiring a multimedia document of heterogeneous content comprising information belonging to at least one of the following categories;

image, video, sound, text, animated vector graphics;

indexing the multimedia document, where indexing comprises;

a1) analyzing the multimedia document both to extract both homogeneous components each belonging to a category of the same type, and also to extract relationships between said homogeneous components;

b1) characterizing the content of each of the homogeneous components and establishing relationships between the elements of the content of each of said homogeneous components;

c1) establishing the relationships, if any, between the elements belonging to the contents of different homogeneous components; and

d1) producing a composite signature combining the indices of each of the homogeneous components, the previously identified relationships, and all of the previously acquired identifiers;

storing the indexed multimedia document together with its composite signature; and

searching a multimedia document as previously indexed and stored on the basis of a composite request, searching comprising;

a2) analyzing the composite request to extract firstly homogeneous requests referring to a category of a single type, and secondly user constraint relationships expressed in the composite request;

b2) processing homogeneous requests;

c2) formulating results of processing homogeneous requests while taking user constraint relationships into account; and

d2) establishing a pertinence order for the results and presenting the results.

10. / A method according to claim 9, characterized in that while searching for a previously indexed and stored multimedia document, during the processing of homogeneous requests, the response returned by a search agent is constituted by a sequence of triplets each comprising:

a similarity score after comparing the composite signatures of requests with the composite signatures prepared during the indexing stage;

the identifier of the object associated during the indexing stage with the original document; and

the identifier of a component of the homogeneous request.