Multimedia content description framework

• US 6,564,263 B1
• Filed: 12/03/1999
• Issued: 05/13/2003
• Est. Priority Date: 12/04/1998
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bergman, Lawrence David, Kim, Michelle Yoonk Yung, Li, Chung-Sheng, Mohan, Rakesh, Smith, John Richard
• Primary Examiner(s): Dinh, Dung C.
• Assistant Examiner(s): JOHNSON, MARLON B
• Application Number:
  US09/456,031
• Time in Patent Office: 1,257 Days
• Field of Search: 709/231, 709/232, 707/101, 707/104.1, 707/500.1, 725/53, 725/135, 725/136, 725/137
• US Class Current: 709/231
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A framework is provided for describing multimedia content and a system in which a plurality of multimedia storage devices employing the content description methods of the present invention can interoperate. In accordance with one form of the present invention, the content description framework is a description scheme (DS) for describing streams or aggregations of multimedia objects, which may comprise audio, images, video, text, time series, and various other modalities. This description scheme can accommodate an essentially limitless number of descriptors in terms of features, semantics or metadata, and facilitate content-based search, index, and retrieval, among other capabilities, for both streamed or aggregated multimedia objects.

Claims

1. A method for describing a multimedia content source including at least one terminal object, said at least one terminal object including one or more modalities, each of said one or more modalities having one or more fidelities associated therewith, said method comprising the steps of:

generating a distribution of one or more modalities and fidelities, said distribution corresponding to an audience of said multimedia content source;

grouping said multimedia content source into one or more target modalities and target fidelities according to said distribution;

generating a modality-fidelity dependency representation for a terminal object in said multimedia content source, said dependency representation including a description scheme comprising predetermined transformation rules for describing at least one of a relationship between two modalities and a relationship between two fidelities;

decomposing the multimedia content source according to said description scheme to create an InfoPyramid representation of each modality;

transforming said multimedia content source according to said modality-fidelity transformation rules;

generating annotations for each object in said multimedia content source; and

repeating said decomposing step, said transforming step and said step of generating annotations until every terminal object in said multimedia content source has been processed.

2. A method for creating a multimedia content source including at least one terminal object, said at least one terminal object including one or more modalities, each of said one or more modalities having one or more fidelities associated therewith, said method comprising the steps of:

generating a distribution of one or more modalities and fidelities, said distribution corresponding to an audience of said multimedia content source;

selecting one or more source modalities and associated source fidelities, said source modalities and source fidelities being selected according to a union of said distribution;

generating a modality-fidelity dependency representation for a terminal object in said multimedia content source, said dependency representation including a description scheme comprising predetermined transformation rules for describing at least one of a relationship between two modalities and a relationship between two fidelities;

synthesizing said multimedia content source according to the description scheme and including predetermined intra-object and inter-object relationships;

transforming said multimedia content source according to said modality-fidelity transformation rules;

generating-annotations for each object in said multimedia content source; and

repeating said synthesizing step, said transforming step and said step of generating annotations until every terminal object in said multimedia content source has been processed.