Synthetic audiovisual description scheme, method and system for MPEG-7

• US 6,593,936 B1
• Filed: 02/01/2000
• Issued: 07/15/2003
• Est. Priority Date: 02/01/1999
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Ostermann, Joern, Rajendran, Raj Kumar, Huang, Qian, Puri, Atul
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/495,171
• Time in Patent Office: 1,260 Days
• Field of Search: 345/619, 345/419, 345/473, 345/474, 345/475, 345/620
• US Class Current: 345/619
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax

Abstract

A method and system for description of synthetic audiovisual content makes it easier for humans, software components or devices to identify, manage, categorize, search, browse and retrieve such content. For instance, a user may wish to search for specific synthetic audiovisual objects in digital libraries, Internet web sites or broadcast media; such a search is enabled by the invention. Key characteristics of synthetic audiovisual content itself such as the underlying 2d or 3d models and parameters for animation of these models are used to describe it. More precisely, to represent features of synthetic audiovisual content, depending on the description scheme to be used, a number of descriptors are selected and assigned values. The description scheme instantiated with descriptor values is used to generate the description, which is then stored for actual use during query/search. Typically, a user, to search for a needed synthetic audiovisual content initiates a query that is passed on to a search engine that then retrieves the candidate content from one or more databases whose description closely matches the query criteria specified by the user.

Claims

1. A method of generating a description of synthetic audiovisual data, the method comprising:

extracting at least one feature of the synthetic audiovisual data;

representing the extracted at least one feature using at least one descriptor selected according to a synthetic audiovisual description scheme;

assigning at least one descriptor value to the at least one descriptor based on the extracted at least one feature; and

generating the synthetic audiovisual data description based on the at least one descriptor values and the synthetic audiovisual description scheme,

wherein the synthetic audiovisual description scheme specifies a structure and semantics of relationships between its components, which include descriptors.

2. The method of claim 1, wherein the components of the synthetic audiovisual description scheme also include other constituent description schemes.

3. The method of claim 2, wherein the constituent synthetic audiovisual description schemes include an animation event description scheme corresponding to dynamic characteristics of the synthetic audiovisual data.

4. The method of claim 2, wherein the constituent synthetic audiovisual description schemes include an animation object description scheme corresponding to static characteristics of the synthetic audiovisual data.

5. The method of claim 2, wherein the constituent synthetic audiovisual description schemes include an animation event/object relational description scheme.

6. The method of claim 1, wherein the at least one feature is a feature of a model used to generate the synthetic audiovisual data.

7. The method of claim 6, wherein the model is a two-dimensional or three-dimensional model.

8. The method of claim 1, wherein the generated description describes the synthetic audiovisual data in such a way that an audiovisual scene is partitioned into synthetic audiovisual objects and their spatial and temporal relationships.

9. The method of claim 8, wherein each synthetic audiovisual object has one or more associated synthetic features and is associated with an animation stream.

10. The method of claim 9, wherein synthetic features of a synthetic audiovisual object are grouped together as being visual, audio or a relationship on semantic or media.

11. The method of claim 1, wherein the descriptor value is an instantiation of a descriptor corresponding to the synthetic audiovisual data.

12. The method of claim 1, wherein the steps of extracting, representing, assigning and generating are performed off-line.

13. The method of claim 1, wherein the steps of extracting, representing, assigning and generating are performed using extensible Markup Language.

14. The method of claim 1, wherein the synthetic audiovisual description scheme includes at least one document type definition that provides a list of example descriptors used during the step of representing.

15. The method of claim 14, wherein the at least one document type definition specifies a set of rules for a structure of the synthetic audiovisual data.

16. The method of claim 1, wherein the generated description includes the at least one descriptor value and is associated with the synthetic audiovisual description scheme.

17. The method of claim 16, wherein the synthetic audiovisual description scheme is incorporated in the generated description.

18. The method of claim 16, wherein the synthetic audiovisual description scheme is stored at a remote location and the generated description is linked to the synthetic audiovisual description scheme.

19. The method of claim 18, wherein the remote location is an Internet site.

20. The method of claim 1, wherein the synthetic audiovisual description scheme is compatible with MPEG-7.

21. A system that generates a description of synthetic audiovisual data, the system comprising:

a feature extractor that extracts at least one feature of the synthetic audiovisual data;

a feature representor that represents the extracted at least one feature using at least one descriptor selected according to a synthetic audiovisual description scheme and assigns at least one descriptor value to the at least one descriptor based on the extracted at least one feature; and

a synthetic audiovisual data description generator that generates the synthetic audiovisual data description based on the at least one descriptor values and the synthetic audiovisual description scheme,

wherein the synthetic audiovisual description scheme specifies a structure and semantics of relationships between its components, which include descriptors.

22. The system of claim 21, wherein the synthetic audiovisual description scheme includes at least one document type definition that provides a list of example descriptors used during the step of representing.

23. The system of claim 22, wherein the at least one document type definition specifies a set of rules for a structure of the synthetic audiovisual data.

24. The system of claim 21, wherein the at least one feature is a feature of a model used to generate the synthetic audiovisual data.

25. The system of claim 24, wherein the model is a two-dimensional or three-dimensional model.

26. The system of claim 21, wherein the generated description describes the synthetic audiovisual data in such a way that an audiovisual scene is partitioned into synthetic audiovisual objects and their spatial and temporal relationships.

27. The system of claim 26, wherein each synthetic audiovisual object has one or more associated synthetic features and is associated with an animation stream.

28. The system of claim 27, wherein synthetic features of a synthetic audiovisual object are grouped together as being visual, audio or a relationship on semantic or media.

29. The system of claim 21, wherein the descriptor value is an instantiation of a descriptor corresponding to the synthetic audiovisual data.

30. A system for accessing and processing synthetic audiovisual data, the system comprising:

at least one server including the system that generates a description of synthetic audiovisual data recited in claim 21, coupled to a synthetic audiovisual data description database and a synthetic audiovisual data database; and

at least one client terminal coupled to the at least server via a network and including a synthetic audiovisual data query interface, search tools and a synthetic audiovisual data browser,

wherein, a synthetic audiovisual data query input through the query interface is used by the search tools to map the query to pseudo features, these pseudo features being transmitted to the at least one server over the network, and wherein, the at least one server maps the pseudo features received over the network to pseudo description that is used by the search engine to access synthetic audiovisual data descriptions stored in the description database.

31. The system of claim 30, wherein the search engine retrieves synthetic audiovisual data stored in the synthetic audiovisual data database based on at least one description.

32. The system of claim 31, wherein the search engine transmits the retrieved synthetic audiovisual content to the client terminal via the network.

33. The system of claim 31, wherein the at least one description is stored in the synthetic audiovisual description database.