Using meta-descriptors to represent multimedia information

• US 6,411,724 B1
• Filed: 07/02/1999
• Issued: 06/25/2002
• Est. Priority Date: 07/02/1999
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Vaithilingam, Gandhimathi, Abdel-Mottaleb, Mohamed S.
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US09/347,214
• Time in Patent Office: 1,089 Days
• Field of Search: 382/190, 382/100, 382/225, 382/305, 382/306, 707/104.1
• US Class Current: 382/100
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

Multimedia information retrieval is performed using meta-descriptors in addition to descriptors. A “descriptor” is a representation of a feature, a “feature” being a distinctive characteristic of multimedia information, while a “meta-descriptor” is information about the descriptor. Meta-descriptors are generated for multimedia information in a repository (10, 12, 14, 16, 18, 20, 22, 24) by extracting the descriptors from the multimedia information (111), clustering the multimedia information based on the descriptors (112), assigning meta-descriptors to each cluster (113), and attaching the meta-descriptors to the multimedia information in the repository (114). The multimedia repository is queried by formulating a query using query-by-example (131), acquiring the descriptor/s and meta-descriptor/s for a repository multimedia item (132), generating a query descriptor/s if none of the same type has been previously generated (133, 134), comparing the descriptors of the repository multimedia item and the query multimedia item (135), and ranking and displaying the results (136, 137).

Claims

1. A method of representing a plurality of multimedia information, comprising:

acquiring descriptors for the multimedia information;

generating clusters of the descriptors;

generating at least one meta-descriptors for the clusters; and

attaching the at least one meta-descriptor to the multimedia information, including respectively attaching the meta-descriptors for the clusters to items of the multimedia information described by the descriptors in the clusters,

wherein the meta-descriptor generating step comprises generating respective groups of data elements for each of the clusters indicating relevancy of the descriptors therein,

wherein at least some of the descriptors are representations of features of an item of multimedia information belonging to a category of multimedia content, the features comprising an ordered set of features including color, texture, shape and sketch, and the category of multimedia content being still image; and

the meta-descriptor generating step comprises generating respective binary vectors for each of the clusters indicating relevancy of the descriptors therein.

2. A method of representing a plurality of multimedia information, comprising:

acquiring descriptors for the multimedia information;

generating clusters of the descriptors;

generating at least one meta-descriptors for the clusters; and

attaching the at least one meta-descriptor to the multimedia information, including respectively attaching the meta-descriptors for the clusters to items of the multimedia information described by the descriptors in the clusters,

wherein the meta-descriptor generating step comprises generating respective groups of data elements for each of the clusters indicating relevancy of the descriptors therein, and

wherein at least some of the descriptors are representations of features of an item of multimedia information belonging to a category of multimedia content, the features comprising an ordered set of features including color, texture, shape and sketch, and the category of multimedia content being still image; and

the meta-descriptor generating step comprises generating respective groups of weight values for each of the clusters indicating respective weights for the descriptors therein.

3. A method of representing a plurality of multimedia information, comprising:

acquiring descriptors for the multimedia information;

generating clusters of the descriptors;

generating at least one meta-descriptors for the clusters; and

attaching the at least one meta-descriptor to the multimedia information, including respectively attaching the meta-descriptors for the clusters to items of the multimedia information described by the descriptors in the clusters,

wherein the meta-descriptor generating step comprises generating respective groups of data elements for each of the clusters indicating relevancy of the descriptors therein, and wherein;

at least some of the descriptors are representations of features of an item of multimedia information belonging to a category of multimedia content, the features comprising an ordered set of features including color, texture, shape and sketch, and the category of multimedia content being still image; and

the meta-descriptor generating step comprises generating respective character strings for each of the clusters identifying at least one relevant feature having a predetermined set of representation types, and at least one of the representation types from the predetermined set of representation types.

4. A method of representing a plurality of multimedia information, comprising:

acquiring descriptors for the multimedia information;

generating at least one meta-descriptors for the descriptors and a group of data elements indicating relevancy of the descriptors therein; and

attaching the at least one meta-descriptor to the multimedia information wherein

at least some of the descriptors are representations of features of an item of multimedia information belonging to a category of multimedia content, the features comprising an ordered set of features including color, texture, shape and sketch, and the category of multimedia content being still image; and

the meta-descriptor generating step comprises generating binary vectors indicating relevancy of the descriptors therein.

5. A method of representing a plurality of multimedia information, comprising:

acquiring descriptors for the multimedia information;

generating at least one meta-descriptors for the descriptors and a group of data elements indicating relevancy of the descriptors therein; and

attaching the at least one meta-descriptor to the multimedia information wherein

at least some of the descriptors are representations of features of an item of multimedia information belonging to a category of multimedia content, the features comprising an ordered set of features including color, texture, shape and sketch, and the category of multimedia content being still image; and

the meta-descriptor generating step comprises generating a group of weight values indicating weights for the descriptors therein.

6. A method of representing a plurality of multimedia information, comprising:

acquiring descriptors for the multimedia information;

generating at least one meta-descriptors for the descriptors and a group of data elements indicating relevancy of the descriptors therein; and

attaching the at least one meta-descriptor to the multimedia information wherein

at least some of the descriptors are representations of features of an item of multimedia information belonging to a category of multimedia content, the features comprising an ordered set of features including color, texture, shape and sketch, and the category of multimedia content being still image; and

the meta-descriptor generating step comprises generating a character string identifying at least one relevant feature having a predetermined set of representation types, and at least one of the representation types from the predetermined set of representation types.