Object-based audio-visual terminal and bitstream structure

• US 8,917,357 B2
• Filed: 08/01/2013
• Issued: 12/23/2014
• Est. Priority Date: 02/14/1997
• Status: Expired due to Fees

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Eleftheriadis, Alexandros, Kalva, Hari
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): DESIR, JEAN WICEL
• Application Number:
  US13/956,860
• Publication Number:
  US 20130314602A1
• Time in Patent Office: 509 Days
• Field of Search: 348/578, 348/584, 348/598, 348/722, 348/423.1, 382/284, 345/630, 715/723, 375/240.26, 375/240.01
• US Class Current: 348/584
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G10L 19/002 Dynamic bit allocation for perceptual audio coders G10L19/032
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/2541 Rights Management protecting software against unauthorised usage in a vending or licensing environment G06F21/10; security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/835 Generation of protective data, e.g. certificates protecting software against unauthorised usage in a vending or licensing environment G06F21/10
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

As information to be processed at an object-based video or audio-visual (AV) terminal, an object-oriented bitstream includes objects, composition information, and scene demarcation information. Such bitstream structure allows on-line editing, e.g. cut and paste, insertion/deletion, grouping, and special effects. In the interest of ease of editing, AV objects and their composition information are transmitted or accessed on separate logical channels (LCs). Objects which have a lifetime in the decoder beyond their initial presentation time are cached for reuse until a selected expiration time. The system includes a de-multiplexer, a controller which controls the operation of the AV terminal, input buffers, AV objects decoders, buffers for decoded data, a composer, a display, and an object cache.

Claims

1. A method for processing video data, comprising:

receiving video information and composition information, wherein said video information includes one or more objects, each object being individually encoded, wherein said composition information includes decoding time information and presentation time information corresponding to the one or more objects;

processing said composition information to compose a single video from said video information, wherein the individually encoded one or more objects are composed within a presentation frame of the single video; and

outputting said single video.

2. The method of claim 1, wherein said one or more objects has immediate visual relevancy.

3. The method of claim 1, wherein said one or more objects represents one or more corresponding physical objects.

4. The method of claim 1, further comprising:

storing said video information in memory.

5. The method of claim 4, wherein said memory is a cache memory.

6. The method of claim 4, further comprising:

storing said video information beyond its initial presentation time.

7. The method of claim 1, wherein outputting said single video further comprises one or more of displaying said single video, storing said single video, or transmitting said single video.

8. A system for processing video data, comprising:

a controller circuit for receiving video information and composition information, wherein said video information includes one or more objects, each object being individually encoded, wherein said composition information includes decoding time information and presentation time information corresponding to the one or more objects;

a composer circuit for processing said composition information to compose a single video from said video information, wherein the individually encoded one or more objects are composed within a presentation frame of the single video; and

an output circuit for outputting said single video.

9. The system of claim 8, wherein said one or more objects has immediate visual relevancy.

10. The system of claim 8, wherein said one or more objects represents physical objects.

11. The system of claim 8, further comprising:

a storage circuit for storing said video information in memory.

12. The system of claim 11, wherein said memory is a cache memory.

13. The system of claim 11, wherein the storage circuit is further configured to store said video information beyond its initial presentation time.

14. A non-transitory computer-readable medium including one or more instructions that cause a computer processor to perform the steps comprising:

receiving video information and composition information, wherein said video information includes one or more objects, each object being individually encoded, wherein said composition information includes decoding time information and presentation time information corresponding to the one or more objects;

processing said composition information to compose a single video from said video information, wherein the individually encoded one or more objects are composed within a presentation frame of the single video; and

outputting said single video.

15. The computer-readable medium of claim 14, wherein said one or more objects has immediate visual relevancy.

16. The computer-readable medium of claim 14, wherein said one or more objects represents physical objects.

17. The computer-readable medium of claim 14, further comprising:

storing said video information in memory.

18. The computer-readable medium of claim 17, wherein said memory is a cache memory.

19. The computer-readable medium of claim 17, further comprising:

storing said video information beyond its initial presentation time.

20. The computer-readable medium of claim 14, wherein outputting said single video further comprises one or more of displaying said single video, storing said single video, or transmitting said single video.