Synthetic audiovisual description scheme, method and system for MPEG-7

• US 8,881,009 B2
• Filed: 12/29/2009
• Issued: 11/04/2014
• Est. Priority Date: 02/01/1999
• Status: Expired due to Fees

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Huang, Qian, Ostermann, Joern, Puri, Atul, Rajendran, Raj Kumar
• Primary Examiner(s): SAX, STEVEN PAUL
• Application Number:
  US12/648,988
• Publication Number:
  US 20100106722A1
• Time in Patent Office: 1,771 Days
• Field of Search: 715/717, 715719-726, 715/760, 345/619, 345/419, 345473-475, 382/107
• US Class Current: 715/717
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax

Abstract

A method and system for description of synthetic audiovisual content makes it easier for humans, software components or devices to identify, manage, categorize, search, browse and retrieve such content. For instance, a user may wish to search for specific synthetic audiovisual objects in digital libraries, Internet web sites or broadcast media; such a search is enabled by the invention. Key characteristics of synthetic audiovisual content itself such as the underlying 2d or 3d models and parameters for animation of these models are used to describe it. More precisely, to represent features of synthetic audiovisual content, depending on the description scheme to be used, a number of descriptors are selected and assigned values. The description scheme instantiated with descriptor values is used to generate the description, which is then stored for actual use during query/search. Typically, a user, to search for a needed synthetic audiovisual content initiates a query that is passed on to a search engine that then retrieves the candidate content from one or more databases whose description closely matches the query criteria specified by the user.

Claims

1. A method comprising:

in an online mode, downloading audiovisual data;

representing a synthetic feature of the audiovisual data having a defined animation parameter as a descriptor, wherein the descriptor is selected according to a synthetic audiovisual description scheme that specifies a structure and semantics of relationships between components of the audiovisual data;

assigning a value to the descriptor based on the synthetic feature; and

at an arbitrary time after the downloading of the audiovisual data, in an offline mode, generating a description based on the value of the descriptor, wherein the description comprises spatial and temporal relationships of audiovisual objects in a scene.

2. The method of claim 1, wherein the components comprise other constituent description schemes.

3. The method of claim 2, wherein the other constituent description schemes comprise an animation event description scheme corresponding to dynamic characteristics of the audiovisual data.

4. The method of claim 2, wherein the constituent description schemes comprise an animation object description scheme corresponding to static characteristics of the audiovisual data in the scene.

5. The method of claim 2, wherein the constituent description schemes comprise an animation event and a relational description scheme.

6. The method of claim 1, wherein the synthetic feature is a model feature used to generate the audiovisual data.

7. The method of claim 6, wherein the model feature comprises one of a two-dimensional model and a three-dimensional model.

8. The method of claim 1, wherein each audiovisual object is associated with synthetic features and an animation stream.

9. The method of claim 8, wherein the synthetic features of an audiovisual object are grouped together as being one of visual, audio, and a relationship of media.

10. The method of claim 1, wherein the value is an instantiation of the descriptor corresponding to the audiovisual data.

11. The method of claim 1, wherein representing, assigning and generating are performed using extensible markup language.

12. The method of claim 1, wherein the synthetic audiovisual description scheme comprises a document type definition that provides a list of example descriptors used during the representing.

13. The method of claim 12, wherein the document type definition specifies a set of rules for a structure of the audiovisual data.

14. The method of claim 1, wherein the description comprises the value and is associated with the synthetic audiovisual description scheme.

15. The method of claim 14, wherein the synthetic audiovisual description scheme is incorporated in the description.

16. The method of claim 14, wherein the synthetic audiovisual description scheme is stored at a remote location and the description is linked to the synthetic audiovisual description scheme.

17. The method of claim 16, wherein a device at the remote location is accessible over a network.

18. The method of claim 1, wherein the synthetic audiovisual description scheme is compatible with MPEG-7.

19. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, result in the processor performing operations comprising;

in an online mode, downloading audiovisual data;

representing a synthetic feature of the audiovisual data having a defined animation parameter as a descriptor, wherein the descriptor is selected according to a synthetic audiovisual description scheme that specifies a structure and semantics of relationships between components of the audiovisual data;

assigning a value to the descriptor based on the synthetic feature; and

at an arbitrary time after the downloading of the audiovisual data, in an offline mode, generating a description based on the value of the descriptor, wherein the description comprises spatial and temporal relationships of audiovisual objects in a scene.

20. The system of claim 19, wherein the components comprise other constituent description schemes.

21. The system of claim 20, wherein the constituent description schemes comprise the synthetic animation event description scheme corresponding to dynamic characteristics of the audiovisual data.

22. The system of claim 21, wherein the synthetic audiovisual description scheme comprises a document type definition that provides a list of example descriptors used during the representing.

23. The system of claim 22, wherein the document type definition specifies a set of rules for a structure of the audiovisual data.

24. The system of claim 19, wherein the synthetic feature is a feature of a model used to generate the audiovisual data.

25. The system of claim 24, wherein the model is one of a two-dimensional and a three-dimensional model.

26. The system of claim 25, wherein each audiovisual object is associated with synthetic features and an animation stream.

27. The system of claim 26, wherein audiovisual object features are grouped together as being one of visual, audio, a relationship on semantic, and a relationship on media.

28. The system of claim 19, wherein the value is an instantiation of the descriptor corresponding to the audiovisual data.

29. A computer readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

in an online mode, downloading audiovisual data;

representing a synthetic feature of the audiovisual data having a defined animation parameter as a descriptor, wherein the descriptor is selected according to a synthetic audiovisual description scheme that specifies a structure and semantics of relationships between components of the audiovisual data;

assigning a value to the descriptor based on the synthetic feature; and

at an arbitrary time after the downloading of the audiovisual data, in an offline mode, generating a description based on the value of the descriptor, wherein the description comprises spatial and temporal relationships of audiovisual objects in a scene.

30. The computer readable device of claim 29, wherein the components comprise other constituent description schemes.