Meta-descriptor for multimedia information

• US 7,340,458 B2
• Filed: 05/06/2002
• Issued: 03/04/2008
• Est. Priority Date: 07/02/1999
• Status: Expired due to Term

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Vaithilingam, Gandhimathi, Abdel-Mottaleb, Mohamed S.
• Primary Examiner(s): Truong; Cam Y T
• Application Number:
  US10/139,177
• Publication Number:
  US 20020159640A1
• Time in Patent Office: 2,129 Days
• Field of Search: 707/3, 707/104.1, 707/45, 707/6, 707/4, 382/305, 382/100, 382/190
• US Class Current: 1/1
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

Multimedia information retrieval is performed using meta-descriptors in addition to descriptors. A “descriptor” is a representation of a feature, a “feature” being a distinctive characteristic of multimedia information, while a “meta-descriptor” is information about the descriptor. Meta-descriptors are generated for multimedia information in a repository (10, 12, 14, 16, 18, 20, 22, 24) by extracting the descriptors from the multimedia information (111), clustering the multimedia information based on the descriptors (112), assigning meta-descriptors to each cluster (113), and attaching the meta-descriptors to the multimedia information in the repository (114). The multimedia repository is queried by formulating a query using query-by-example (131), acquiring the descriptor/s and meta-descriptor/s for a repository multimedia item (132), generating a query descriptor/s if none of the same type has been previously generated (133, 134), comparing the descriptors of the repository multimedia item and the query multimedia item (135), and ranking and displaying the results (136, 137).

Claims

1. A method of searching multimedia information in a repository described by descriptors using a query multimedia information item, comprising executing operations in a data processing device, the operations comprising acts of:

acquiring meta-descriptors of repository descriptors;

selecting query multimedia information;

extracting at least one query descriptor from the query multimedia information based on the meta-descriptors;

comparing the query descriptor with the repository descriptors;

ranking at least some of the multimedia information in the repository in accordance with the comparing act; and

providing an output indicating the ranking, wherein the repository descriptors are instances of a plurality of different descriptor types, and where the act of extracting at least one descriptor from the query multimedia information comprises extracting a plurality of descriptors based on the meta-descriptors to obtain a plurality of query descriptors, the query descriptors being respective instances of the descriptor types; and

the comparing act comprises comparing the query descriptors with respective repository descriptors having same descriptor types.

2. The method of claim. 1, wherein at least one of the query descriptors is a value repreuentng a descriptor type.

3. The method of claim 1, wherein meta-descriptors are included in the repository descriptors.

4. The method of claim 3, wherein the comparing act ccanpz ices comparing query descriptors to stored meta-descriptors.

5. The method of claim 1, wherein at least one of the query descriptors is a value representing a descriptor type.

6. A method of retrieving multimedia information from a repository, comprising executing operations in a data processing device, the operations comprising acts of:

extracting repository descriptors from the multimedia information in the repository;

generating clusters of the repository descriptors;

indexing the repository descriptors to the multimedia information in the repository;

generating meta-descriptors for the clusters;

attaching the meta-descriptors for the clusters to respective multimedia information in the clusters;

selecting query multimedia information;

extracting at least one descriptor from the query multimedia information based on the meta-descriptors to obtain at least one query descriptor;

comparing the query descriptor with the repository descriptors;

ranking at least some of the multimedia information in the repository in accordance with the comparing act; and

providing an output indicating the ranking.

7. A method as in claim 6 wherein the repository descriptors are instances of a plurality of different descriptor types, and wherein:

the act of extracting at least one descriptor from the query multimedia information comprises extracting a plurality of descriptors based on the meta-descriptors to obtain a plurality of query descriptors, the query descriptors being respective instances of the descriptor types; and

the comparing act comprises comparing the query descriptors with respective repository descriptors having same descriptor types.

8. The method of claim 6, wherein meta-descriptors are included in the repository descriptors.

9. The method of claim 8, wherein the comparing act comprises comparing query descriptors to stored meta-descriptors.

10. A method comprising executing operations in at least one data processing device, the operations comprising acts of:

maintaining a database of multimedia information, the database comprising at least one respective descriptor and at least one respective meta-descriptor attached to each piece of the multimedia information;

extracting at least one query descriptor from query multimedia information based on meta-descriptors of the multimedia information; and

comparing the query descriptor with descriptors of the multimedia information in said database;

ranking at least some of the multimedia information in the database in accordance with the comparing act; and

providing an output indicating the ranking, wherein the repository descriptors are instances of a plurality of different descriptor types, and where the act of extracting at least one descriptor from the query multimedia information comprises extracting a plurality of descriptors based on the meta-descriptors to obtain a plurality of query descriptors, the query descriptors being respective instances of the descriptor types;

and the comparing act comprises comparing the query descriptors with respective repository descriptors having same descriptor types.

11. Apparatus for searching a database of multimedia information in a repository, the apparatus comprising:

means for storing the database; and

at least one data processing device performs operations comprising the acts of;

acquiring meta-descriptors of repository descriptors;

selecting query multimedia information;

extracting at least one query descriptor from the query multimedia information based on the meta-descriptors to obtain at least one query descriptor;

comparing the query descriptor with the repository descriptors;

ranking at least some of the multimedia information in the repository in accordance with the comparing act; and

providing an output indicating the ranking, wherein the repository descriptors are instances of a plurality of different descriptor types, and where the act of extracting at least one descriptor from the query multimedia information comprises extracting a plurality of descriptors based on the meta-descriptors to obtain a plurality of query descriptors, the query descriptors being respective instances of the descriptor types;

and the comparing act comprises comparing the query descriptors with respective repository descriptors having same descriptor types.

12. Apparatus for retrieving information from a database of multimedia information, the apparatus comprising:

means for storing the database; and

at least one data processing device performs operations comprising acts of;

extracting repository descriptors from the multimedia information in the repository;

generating clusters of the repository descriptors;

indexing the repository descriptors to the multimedia information in the repository;

generating meta-descriptors for the clusters;

attaching the meta-descriptors for the clusters to the respective multimedia information in the clusters;

selecting query multimedia information;

extracting at least one descriptor from the query multimedia information based on the meta-descriptors to obtain at least one query descriptor;

comparing the query descriptor with the repository descriptors; and

ranking at least some of the multimedia information in the repository in accordance with the comparing act; and

providing an output indicating the ranking.

13. The apparatus of claim 12, wherein the act of extracting the at least one descriptor from the query multimedia information comprises extracting a plurality of descriptors based on the meta-descriptors to obtain a plurality of query descriptors, the query descriptors being respective instances of the descriptor types;

and the comparing act comprises comparing the query descriptors with respective repository descriptors having same descriptor types.