Concept-based system for representing and processing multimedia objects with arbitrary constraints

• US 20030041047A1
• Filed: 08/09/2001
• Published: 02/27/2003
• Est. Priority Date: 08/09/2001
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Natsev, Apostol Ivanov, Chang, Yuan-Chi, Smith, John R., Li, Chung-Sheng
• Application Number:
  US09/925,397
• Publication Number:
  US 20030041047A1
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Disclosed is a method and apparatus for querying stored multimedia data in a computer system, comprising receiving into an intermediate level a high-level concept from a user describing data to be retrieved, translating, in said intermediate level, said high-level concept into low-level queries by using system pre-defined high-level concepts, transferring said low-level queries to a low level comprising one or more search engines; said one or more search engines performing a query of the stored multimedia information using said low-level queries. A matching algorithm for use with the invention is also disclosed.

Claims

1. A method for querying stored multimedia data in a computer system, comprising:

receiving into an intermediate level a high-level concept from a user describing data to be retrieved;

translating, in said intermediate level, said high-level concept into low-level queries by using system pre-defined high-level concepts; and

transferring said low-level queries to a low level comprising one or more search engines;

said one or more search engines performing a query of the stored multimedia information using said low-level queries.

2. The method of claim 1 wherein said intermediate level comprises:

a set of library modules, said set of library modules comprising;

a concept library module for storing concepts;

one or more library modules adapted to store said data from said one or more data sources;

a cataloger module adapted to construct a new concept from said high-level concept using data from said concept library and library modules, thereby creating a concept construct, and to pass said concept construct to said concept library module for storage as a concept; and

an interpreter module adapted to translate said high-level concept into low-level queries using said concepts stored in said construct library and to pass said low-level queries to said one or more search engines.

3. The method of claim 2 wherein said set of library modules further comprises at least one library module selected from the group comprising:

a feature library module adapted to store multimedia features;

a matching algorithm library module adapted to store matching algorithms; and

a constraint library module adapted to store feature constraints.

4. The method of claim 3 wherein each said library module further comprises an application program interface to receive said data from a said data source.

5. The method of claim 3 wherein said cataloger module further performs the steps of:

selecting a set of concept features from said feature library module;

selecting a set of concepts from said concept library module for use as child-concepts; and

selecting a set of constraints on said child concepts from said constraint library module.

6. The method of claim 1 wherein said each said concept comprises a triplet of a set of child-concepts, a set of features, and a set of relationships.

7. The method of claim 6 wherein said concepts comprise a hierarchical fuzzy graph data tree-structure comprising nodes, aggregation edges, and association edges and wherein:

said nodes correspond to said concepts and said features;

said aggregation edges correspond to parent-child relationships; and

said association edges correspond to said constraints.

8. The method of claim 7 wherein said edges are weighted.

9. The method of claim 3 further comprising a matching algorithm comprising GetNextMatch( ), AssignNextMatch( ), and ShiftNextMatch( ) procedures, wherein:

11. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for querying stored multimedia data, said method steps comprising:

receiving into an intermediate level a high-level concept from a user describing data to be retrieved;

translating, in said intermediate level, said high-level concept into low-level queries by using system pre-defined high-level concepts;

transferring said low-level queries to a low level comprising one or more search engines;

said one or more search engines performing a query of the stored multimedia information using said low-level queries.

12. The apparatus of claim 11 wherein said intermediate level comprises:

a set of library modules, said set of library modules comprising;

a concept library module for storing concepts; and

one or more library modules adapted to store said data from said one or more data sources;

a cataloger module adapted to construct a new concept from said high-level concept using data from said concept library and library modules, thereby creating a concept construct, and to pass said concept construct to said concept library module for storage as a concept; and

an interpreter module adapted to translate said high-level concept into low-level queries using said concepts stored in said construct library and to pass said low-level queries to said one or more search engines.

13. The apparatus of claim 12 wherein said set of library modules further comprises at least one library module selected from the group comprising:

a feature library module adapted to store multimedia features;

a matching algorithm library module adapted to store matching algorithms; and

a constraint library module adapted to store feature constraints.

14. The apparatus of claim 13 wherein each said library module further comprises an application program interface to receive said data from a said data source.

15. The apparatus of claim 13 wherein said cataloger module further performs the steps of:

selecting a set of concept features from said feature library module;

selecting a set of concepts from said concept library module for use as child-concepts; and

selecting a set of constraints on said child concepts from said constraint library module.

16. The apparatus of claim 11 wherein said each said concept comprises a triplet of a set of child-concepts, a set of features, and a set of relationships.

17. The apparatus of claim 16 wherein said concepts comprise a hierarchical fuzzy graph data tree-structure comprising nodes, aggregation edges, and association edges and wherein:

said nodes correspond to said concepts and said features;

said aggregation edges correspond to parent-child relationships; and

said association edges correspond to said constraints.

18. The apparatus of claim 17 wherein said edges are weighted.

19. The apparatus of claim 13 further comprising a matching algorithm comprising GetNextMatch( ), AssignNextMatch( ), and ShiftNextMatch( ) procedures, wherein: