Systems and methods for playing, browsing and interacting with MPEG-4 coded audio-visual objects

• US 9,641,897 B2
• Filed: 09/14/2012
• Issued: 05/02/2017
• Est. Priority Date: 01/27/1998
• Status: Expired due to Fees

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Haskell, Barin Geoffry, Puri, Atul, Schmidt, Robert Lewis
• Primary Examiner(s): Patel, Manglesh M
• Application Number:
  US13/616,038
• Publication Number:
  US 20130013989A1
• Time in Patent Office: 1,691 Days
• Field of Search: 715200-203
• CPC Class Codes: H04N 19/25 with scene description coding, e.g. binary format for scenes [BIFS] compression
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A number of novel configurations for MPEG-4 playback, browsing and user interaction are disclosed. MPEG-4 playback systems are not simple extensions of MPEG-2 playback systems, but, due to object based nature of MPEG-4, present new opportunities and challenges in synchronized management of independent coded objects as well as scene composition and presentation. Therefore, these configurations allow significantly new and enhanced multimedia services and systems. In addition, MPEG-4 aims for an advanced functionality, called Adaptive Audio Visual Session (AAVS) or MPEG-J. Adaptive Audio Visual Session (AAVS) (i.e., MPEG-AAVS, MPEG-Java or MPEG-J) requires, in addition to the definition of configurations, a definition of an application programming interface (API) and its organization into Java packages. Also disclosed are concepts leading to definition of such a framework.

Claims

1. A method comprising:

receiving, via a device, user input associated with an audiovisual object demultiplexer and a binary format for a scene browser;

decoding, composing, and rendering the user input, to yield MPEG-4 data comprising a scene description graph, the scene description graph indicating how objects appear within a scene and comprising a video object node, an audio source node and an image texture node; and

receiving additional user interaction, via the device and a script, with the audiovisual object demultiplexer and the binary format for the scene browser, to modify the MPEG-4 data by adjusting one or more of the decoding, the composing, and the rendering of the MPEG-4 data according to the additional user interaction, such that a new object is added to the scene description graph.

2. The method of claim 1, wherein the MPEG-4 data comprises one of audio media, visual media, and synthetic media.

3. The method of claim 1, wherein receiving additional user interaction occurs via a JavaScript interpreter.

4. The method of claim 1, further comprising interpreting the MPEG-4 data to yield an object which interacts with a user.

5. The method of claim 1, wherein decoding and composing further comprise:

connecting a decoder to a corresponding object programmer interface; and

connecting the decoder to a scene compositor.

6. The method of claim 5, wherein the scene compositor is connected to the audiovisual object demultiplexer and the binary format for the scene browser.

7. The method of claim 5, wherein the scene compositor is connected to the script.

8. A system comprising:

a processor; and

a computer-readable storage device having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving, via a device, user input associated with an audiovisual object demultiplexer and a binary format for a scene browser;

decoding, composing, and rendering the user input, to yield MPEG-4 data comprising a scene description graph, the scene description graph indicating how objects appear within a scene and comprising a video object node, an audio source and an image texture node; and

receiving additional user interaction, via the device and a script, with the audiovisual object demultiplexer and the binary format for the scene browser, to modify the MPEG-4 data by adjusting one or more of the decoding, the composing, and the rendering of the MPEG-4 data according to the additional user interaction, such that a new object is added to the scene description graph.

9. The system of claim 8, wherein the MPEG-4 data comprises one of audio media, visual media, and synthetic media.

10. The system of claim 8, wherein receiving additional user interaction occurs via a JavaScript interpreter.

11. The system of claim 8, the computer-readable storage device having additional instructions which result in the operations further comprising interpreting the MPEG-4 data to yield an object which interacts with a user.

12. The system of claim 8, wherein decoding and composing further comprise:

connecting a decoder to a corresponding object programmer interface; and

connecting the decoder to a scene compositor.

13. The system of claim 12, wherein the scene compositor is connected to the audiovisual object demultiplexer and the binary format for the scene browser.

14. The system of claim 12, wherein the scene compositor is connected to the script.

15. A computer-readable storage device having instructions stored which, when executed on a computing device, cause the computing device to perform operations comprising:

receiving, via a device, user input associated with an audiovisual object demultiplexer and a binary format for a scene browser;

decoding, composing, and rendering the user input, to yield MPEG-4 data comprising a scene description graph, the scene description graph indicating how objects appear within a scene and comprising a video object node, an audio source node and an image texture node; and

receiving additional user interaction, via the device and a script, with the audiovisual object demultiplexer and the binary format for the scene browser, to modify the MPEG-4 data by adjusting one or more of the decoding, the composing, and the rendering of the MPEG-4 data according to the additional user interaction, such that a new object is added to the scene description graph.

16. The computer-readable storage device of claim 15, wherein the MPEG-4 data comprises one of audio media, visual media, and synthetic media.

17. The computer-readable storage device of claim 15, wherein receiving additional user interaction occurs via a JavaScript interpreter.

18. The computer-readable storage device of claim 15 having additional instructions which result in the operations further comprising interpreting the MPEG-4 data to yield an object which interacts with a user.

19. The computer-readable storage device of claim 15, wherein decoding and composing further comprise:

connecting a decoder to a corresponding object programmer interface; and

connecting the decoder to a scene compositor.

20. The computer-readable storage device of claim 19, wherein the scene compositor is connected to the audiovisual object demultiplexer and the binary format for the scene browser.