Object-based audio-visual terminal and bitstream structure

• US 8,115,873 B2
• Filed: 06/10/2009
• Issued: 02/14/2012
• Est. Priority Date: 02/14/1997
• Status: Expired due to Fees

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Eleftheriadis, Alexandros, Kalya, Hari
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): DESIR, JEAN WICEL
• Application Number:
  US12/482,292
• Publication Number:
  US 20090245389A1
• Time in Patent Office: 979 Days
• Field of Search: 348/578, 348/584, 348/598, 348/722, 348/423.1, 382/284, 345/630, 715/723
• US Class Current: 348/578
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G10L 19/002 Dynamic bit allocation for perceptual audio coders G10L19/032
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/835 Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

As information to be processed at an object-based video or audio-visual (AV) terminal, an object-oriented bitstream includes objects, composition information, and scene demarcation information. Such bitstream structure allows on-line editing, e.g. cut and paste, insertion/deletion, grouping, and special effects. In the interest of ease of editing, AV objects and their composition information are transmitted or accessed on separate logical channels (LCs). Objects which have a lifetime in the decoder beyond their initial presentation time are cached for reuse until a selected expiration time. The system includes a de-multiplexer, a controller which controls the operation of the AV terminal, input buffers, AV objects decoders, buffers for decoded data, a composer, a display, and an object cache.

Claims

1. A method for processing object-based audiovisual/video data at a receiver to generate a composed scene, comprising:

(a) receiving in a data bit stream, over time, at least one audiovisual/video object and composition information for the object;

(b) locally storing the at least one object;

(c) processing the received composition information to compose a scene using the at least one stored object; and

(d) generating the composed scene.

2. The method of claim 1, wherein the at least one audiovisual/video object includes at least one audio object.

3. The method of claim 1, further comprising:

outputting the composed scene.

4. The method of claim 3, wherein outputting the composed scene includes presenting the composed scene.

5. The method of claim 4, wherein presenting the composed scene includes playing audio.

6. The method of claim 1, wherein the at least one audiovisual/video object includes a plurality of audiovisual/video objects.

7. The method of claim 1, wherein the at least one audiovisual/video object includes a single audiovisual/video object.

8. The method of claim 1, wherein the composition information is relative to a single object.

9. The method of claim 8, wherein the composition information is selected from the group consisting of the location, rotation angle, and zooming factor of the single object.

10. The method of claim 1, wherein the composition information includes information for composing audio objects.

11. The method of claim 1, wherein locally storing the at least one object includes storing the at least one object in a memory.

12. The method of claim 11, wherein the memory is selected from the group consisting of a cache memory, decoder buffer, and input buffer.

13. The method of claim 1, further comprising:

removing the at least one audiovisual/video object from local storage prior to its initial presentation time.

14. An apparatus for processing object-based audiovisual/video data, comprising:

(a) a receiver circuit configured to receive in a data bit stream, over time, at least one audiovisual/video object and composition information for the object;

(b) local storage configured to store the at least one object;

(c) a composer circuit configured to process the received composition information to compose a scene using the at least one object; and

(d) an scene generating device configured to generate the composed scene.

15. The apparatus of claim 14, further comprising:

an output device configured to output the composed scene.

16. The apparatus of claim 15, wherein the output device is configured to present the scene.

17. The apparatus of claim 15, wherein the output device is further configured to present the scene by playing audio.

18. The apparatus of claim 14, wherein the receiver circuit is configured to receive a plurality of audiovisual/video objects.

19. The apparatus of claim 14, wherein the receiver circuit is configured to receive a single audiovisual/video object.

20. The apparatus of claim 14, wherein the composer circuit is configured to process composition information that relates to a single object.

21. The apparatus of claim 20, wherein the composer circuit is configured to process composition information selected from the group consisting of the location, rotation angle, and zooming factor of the single object.

22. The apparatus of claim 14, wherein the composer circuit is configured to process composition information related to an audio object.

23. The apparatus of claim 14, wherein the local storage is selected from the group consisting of a memory, cache memory, decoder buffer, and input buffer.