Multimedia archive description scheme

• US 6,941,325 B1
• Filed: 02/01/2000
• Issued: 09/06/2005
• Est. Priority Date: 02/01/1999
• Status: Expired due to Fees

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University), International Business Machines Corporation
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University), International Business Machines Corporation
• Inventors: Jaimes, Alejandro, Seungyup, Paek, Chang, Shih-Fu, Smith, John R., Li, Chung-Sheng, Benitez, Ana B.
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): Veillard, Jacques
• Application Number:
  US09/889,859
• Time in Patent Office: 2,044 Days
• Field of Search: 707 1- 6, 707100-102, 707/103.R, 707/104.1, 707/200, 709/219, 709/224, 706/45, 706/52, 706/55, 345/440, 345/763
• US Class Current: 1/1
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A multimedia archive description scheme is provided for characterizing a multimedia archive having records and associated record descriptions. The multimedia archive description scheme provides a data structure which relates records by similarity measures. The principle data structure in the multimedia archive description scheme is a cluster. A cluster includes one or more attributes of the records in the archive and can include one or more cluster relationships. Cluster attributes can include feature space attributes, semantic attributes, media attributes and meta attributes of the records in the archive. The cluster relationships can relate records to clusters or clusters to clusters. Cluster relationships can include feature space (syntactic) relationships, semantic relationships, media relationships and meta relationships. The multimedia archive description scheme provides an efficient form for describing a collection of records.

Claims

1. A method of describing the content of a multimedia archive, having records and record descriptions associated with the records, comprising:

evaluating the record descriptions to identify cluster attributes and cluster relationships in a plurality of records in the archive;

wherein the cluster attributes are selected from the group including feature space attributes, semantic attributes, media attributes and meta attributes; and

generating an archive description record including at least one cluster describing at least one common cluster attribute and at least one cluster relationship in at least two records.

2. The method of describing the content of a multimedia archive of claim 1, further comprising the step of indexing, the cluster attributes are indexed in accordance with an information-based hierarchy.

3. The method of describing the content of a multimedia archive of claim 2, wherein feature space attributes are indexed above semantic attributes in the information-based hierarchy.

4. The method of describing the content of a multimedia archive of claim 2, wherein the feature space attributes are selected from the group consisting of type/technique attributes, global distribution attributes, local structure attributes and global composition attributes.

5. The method of describing the content of a multimedia archive of claim 2, wherein the semantic attributes are selected from the group consisting of generic object attributes, generic scene attributes, specific object attributes, specific scene attributes, abstract object attributes and abstract scene attributes.

6. The method of describing the content of a multimedia archive of claim 2, wherein the information-based hierarchy is a ten level index structure having a plurality of feature space levels and a plurality of semantic attribute levels, wherein the feature space attribute levels include a type/technique attribute level, a global distribution attribute level, a local structure attribute level and a global composition attribute level and wherein the semantic attribute levels include generic object attribute level, a generic scene attribute level, a specific object attribute level, a specific scene attribute level, an abstract object attribute level and an abstract scene attribute level.

7. The method of describing the content of a multimedia archive of claim 1, wherein the cluster includes at least one cluster relationship.

8. The method of describing the content of a multimedia archive of claim 7 wherein the cluster relationships are selected from the group consisting of feature space relationships, semantic relationships, media relationships and meta relationships.

9. The method of describing the content of a multimedia archive of claim 8 wherein the feature space relationships are selected from the group consisting of spatial relationships, temporal relationships and visual relationships.

10. The method of describing the content of a multimedia archive of claim 8 wherein the semantic relationships are selected from the group consisting of lexical relationships and predictive relationships.

11. A system for generating a multimedia archive description Comprising:

a digital storage subsystem for storing multimedia records and descriptions of the records in accordance with a media description scheme;

a computer processor operatively coupled to the digital storage subsystem, the computer processor accessing the record descriptions and identifying at least one cluster from the multimedia record descriptions of at least two media records, extracting at least one cluster attribute and at least one cluster relationship from the at least one cluster, the processor generating an archive description record including said at least one cluster, said at least one cluster attribute and said at least one cluster relationship, wherein the cluster attributes are selected from the group including feature space attributes, semantic attributes, media attributes and meta attributes; and

archive description storage operative coupled to the computer processor for storing the archive description record.

12. The system for generating a multimedia archive description of claim 11, wherein the at least one cluster attribute is indexed in accordance with a multi-level indexing hierarchy.

13. The system for generating a multimedia archive description of claim 12, wherein the multi-level indexing hierarchy is a ten level index structure having a plurality of feature space levels and a plurality of semantic attribute levels, wherein the feature space attribute levels include a type/technique attribute level, a global distribution attribute level, a local structure attribute level and a global composition attribute level and wherein the semantic attribute levels include generic object attribute level, a generic scene attribute level, a specific object attribute level, a specific scene attribute level, an abstract object attribute level and an abstract scene attribute level.

14. The system for generating, a multimedia archive description of claim 11, wherein the feature space attributes are selected from the group including type/technique attributes, global distribution attributes, local structure attributes and global composition attributes.

15. The system for generating a multimedia archive description of claim 11, wherein the semantic attributes are selected from the group consisting of generic object attributes, generic scene attributes, specific object attributes, specific scene attributes, abstract object attributes and abstract scene attributes.

16. The system for generating a multimedia archive description of claim 11, wherein the at least one cluster relationship is selected from the group consisting of feature space relationships, semantic relationships, media relationships and meta relationships.

17. The system for generating a multimedia archive description of claim 16, wherein the feature space relationships are selected from the group consisting of spatial relationships, temporal relationships, and visual relationships.

18. The system for generating a multimedia archive description of claim 16, wherein the semantic relationships are selected from the group consisting of lexical relationships and predictive relationships.

19. The system for generating a multimedia archive description of claim 16, wherein the feature space relationships are selected from the group including generic relationships and specific relationships.

20. The system for generating a multimedia archive description of claim 16, wherein the semantic relationships are selected from the group including generic relationships, specific relationships and abstract relationships.

21. The system for generating a multimedia archive of claim 11, wherein the digital storage subsystem includes local computer readable storage for multimedia records, multimedia record descriptions and the archive description record.

22. The system of claim 11, wherein the digital storage subsystem includes a plurality of storage devices interconnected by a computer network.

23. The system for generating a multimedia archive description of claim 11, wherein the media attributes are selected from the group including format, storage, file identification and file location attributes.

24. The system for generating a multimedia archive description of claim 11, wherein the meta attributes are selected from the group including creation, representative icon and intellectual property right attributes.

25. A media archive system comprising:

a computer readable storage system for storing a plurality of media records;

at least one media description engine, said media description engine accessing a media record stored in the computer readable storage system and generating a media description record corresponding thereto;

a cluster processor operatively coupled to the computer readable storage system, the cluster processor accessing the media description records and generating an archive description record, the archive description record including at least one cluster which relates at least two records in the storage system each said at least one cluster being defined at least in part by a cluster attribute and a cluster relationship of the media description records, wherein the cluster attributes are selected from the group consisting of feature space attributes, semantic attributes, media attributes and meta attributes; and

a query processor operatively coupled to the cluster processor, the query processor receiving archive search parameters from a user and providing a search query to the cluster processor.

26. The media archive system of claim 25, wherein the cluster attributes are indexed in accordance with an information-based hierarchy.

27. The media archive system of claim 26, wherein feature space attributes are indexed above semantic attributes in the information-based hierarchy.

28. The media archive system of claim 27, wherein the feature space attributes are selected from the group consisting of type/technique attributes, global distribution attributes, local structure attributes and global composition attributes.

29. The media archive system of claim 27, wherein the semantic attributes are selected from the group consisting of generic object attributes, generic scene attributes, specific object attributes, specific scene attributes, abstract object attributes and abstract scene attributes.

30. The media archive system of claim 27, wherein the information-based hierarchy is a ten level index structure having a plurality of feature space levels and a plurality of semantic attribute levels, wherein the feature space attribute levels include a type/technique attribute level, a global distribution attribute level, a local structure attribute level and a global composition attribute level and wherein the semantic attribute levels include generic object attribute level, a generic scene attribute level, a specific object attribute level, a specific scene attribute level, an abstract object attribute level and an abstract scene attribute level.

31. The media archive system of claim 25, wherein the cluster relationships are selected from the group consisting of feature space relationships, semantic relationships, media relationships and meta relationships.

32. The media archive system of claim 31, wherein the feature space relationships are selected from the group including spatial relationships, temporal relationships, and visual relationships.

33. The media archive system of claim 31, wherein the semantic relationships are selected from the group consisting of lexical relationships and predictive relationships.

34. An archive description file defined in computer readable media for describing the content of a multimedia archive, having records and record descriptions associated with the records, comprising:

at least one cluster attribute and at least one cluster relationship describing at least one cluster in the record descriptions, wherein the cluster attributes are selected from the group consisting of feature space attributes, semantic attributes, media attributes and meta attributes; and

a collection index structure relating the clusters to the records.

35. The archive description file for describing the content of a multimedia archive of claim 34, wherein the cluster attributes are indexed in accordance with an information-based hierarchy.

36. The archive description file for describing the content of a multimedia archive of claim 35, wherein feature space attributes are indexed above semantic attributes in the information-based hierarchy.

37. The archive description file for describing the content of a multimedia archive of claim 35, wherein the feature space attributes are selected from the group including type/technique attributes, global distribution attributes, local structure attributes and global composition attributes.

38. The archive description file for describing the content of a multimedia archive of claim 35, wherein the semantic attributes are selected from the group consisting of generic object attributes, generic scene attributes, specific object attributes, specific scene attributes, abstract object attributes and abstract scene attributes.

39. The archive description file for describing the content of a multimedia archive of claim 35, wherein the information-based hierarchy is a ten level index structure having a plurality of feature space levels and a plurality of semantic attribute levels, wherein the feature space attribute levels include a type/technique attribute level, a global distribution attribute level, a local structure attribute level and a global composition attribute level and wherein the semantic attribute levels include generic object attribute level, a generic scene attribute level, a specific object attribute level, a specific scene attribute level, an abstract object attribute level and an abstract scene attribute level.

40. The archive description file for describing the content of a multimedia archive of claim 39, wherein the cluster relationships are selected from the group including feature space relationships, semantic relationships, media relationships and meta relationships.

41. The archive description file for describing the content of a multimedia archive of claim 40, wherein the feature space relationships are selected from the group including spatial relationships, temporal relationships, and visual relationships.

42. The archive description file for describing the content of a multimedia archive of claim 40, wherein the semantic relationships are selected from the group including lexical relationships and predictive relationships.