Audio/video system with auxiliary data

• US 6,801,575 B1
• Filed: 05/27/1998
• Issued: 10/05/2004
• Est. Priority Date: 06/09/1997
• Status: Expired due to Fees

• Current Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Crinon, Regis J.
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US09/085,393
• Time in Patent Office: 2,323 Days
• Field of Search: 375/240.26, 375/240.24, 375/240.09, 375/240.01, 375/240, 348/414.1, 348/420.1, 348/429.1, 348/423.1, 348/554, 382/242, 382/243, 343/397.1, 345/302, 345/157, 345/719, 707/526, 707/101, 709/217, 725/105, 725/142, 725/131
• US Class Current: 375/240.26
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/643 Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A video system includes a plurality of frames of video each of which is defined by a plurality of scene elements. The scene elements for a respective frame together define an image of the frame. First auxiliary data is descriptive of a first scene element of the frame and second auxiliary data is descriptive of a second scene element of the frame. A sending device sends the frame of video, including its scene elements, the first auxiliary data, and the second auxiliary data, from the sending device to a receiving device.

Claims

1. A video system comprising:

(a) a plurality of frames of video each of which is defined by a plurality of scene elements, where said plurality of scene elements for a respective frame together define an image of said respective frame, wherein each of said scene elements is a separately encoded image with respect to other scene elements for said respective frame, wherein said image is created by composing said separately encoded scene elements within a respective frame based upon location information defining the position of said scene elements within said respective frame by a receiving device of said video system;

(b) first auxiliary data descriptive of the content of a first scene element of said plurality of scene elements for said respective frame;

(c) second auxiliary data descriptive of the content of a second scene element of said plurality of scene elements for said respective frame; and

(d) a sending device that sends a bitstream representing said respective frame of said video, including respective said separately encoded scene elements and said location information, said first auxiliary data, and said second auxiliary data, from said sending device to said receiving device, wherein said separately encoded scene elements are sent from said sending device to said receiving device using at least two elementary streams.

2. The video system of claim 1 wherein said video includes audio information.

3. The video system of claim 1 wherein said scene elements are overlapping.

4. The video system of claim 1 wherein said first auxiliary data is a set of descriptors and parameter values resulting from said first scene element.

5. The video system of claim 1 wherein said first auxiliary data is not directly representative of an image.

6. The video system of claim 1 wherein said first auxiliary data is viewable on a display device by selecting said first scene element.

7. A video system comprising:

(a) a plurality of frames of video each of which defines an image of said respective frame, each of said frames defined by a plurality of pixels;

(b) first auxiliary data descriptive of the content of a first portion of said pixels of a respective frame and associated with said first portion of said pixels of a respective frame and associated with said first portion of said pixels;

(c) second auxiliary data descriptive of the content of a second portion of said pixels of said respective frame and associated with said second portion of said pixels, where said first portion is different than said second portion; and

(d) a sending device that sends said first portion of said pixels, said first auxiliary data, said second auxiliary data, and said second portion of said pixels to a receiving device.

8. The video system of claim 7 wherein said video includes audio information.

9. The video system of claim 7 wherein said first portion and said second portion are blocks of 16 by 16 pixels of said respective frame.

10. The video system of claim 7 wherein said first auxiliary data is a set of descriptors and parameter values resulting form said first portion.

11. The video system of claim 7 wherein said first auxiliary data is not directly representative of an image.

12. The video system of claim 7 wherein said first auxiliary data is viewable on a display device by selecting said first portion.

13. A video system comprising:

(a) a plurality of frames of video each of which is defined by a plurality of scene elements, where said plurality of scene elements for a respective frame together define an image of said respective frame, wherein each of said scene elements is a separately encoded image with respect to other scene elements for said respective frame, wherein said image is created by composing said separately encoded scene elements within a respective frame based upon location information defining the position of said scene elements within said respective frame by said video system;

(b) each of said scene elements defined by a scene description format including a scene element data object format that defines the number of elementary streams and elementary stream descriptors;

(c) each of said elementary stream descriptors including a decoder type identifier;

(d) each of said decoder type identifiers represents at least one of an audio stream, a video stream, object descriptor, and auxiliary data, with at least one of said decoder type identifiers represents said auxiliary data; and

(e) said auxiliary data descriptive of the content of a respective one of said scene elements.

18. The system of claim 13 wherein said plurality of frames including said scene elements are represented in a bitstream.

14. A system for encoding a moving picture comprised of a sequence of time separated frames depicting scenes, wherein each of said frames is defined by a plurality of scene elements, wherein each of the scene elements is a separately encoded image with respect to other scene elements for the respective frame, wherein said image is created by decoding a bitstream and composing said separately encoded scene elements within a respective frame based upon location information defining the position of said scene elements within said respective frame, said system comprising a scene description format, referencing at least one of said scene elements, including a scene element data object format that includes an elementary stream descriptor format including, in turn, a decoder type identifier format wherein any one of a set of decoder types may be represented by one out of a set of predetermined bit values and one of said predetermined bit values represents an auxiliary information decoder type, wherein said auxiliary information is descriptive of the content of at least one of said scene elements.

15. The system of claim 14 wherein an audio stream is also encoded.

16. A system for encoding a moving picture comprised of a sequence of time separated frames, wherein each of said frames includes a plurality of scene elements that are a separately encoded image with respect to other scene elements for a respective frame, said system adapted to divide said frames into scene elements, wherein said frame is created by composing said scene elements within said respective frame based upon location information defining the position of said scene elements within said respective frame by said system, and including a receiver display subsystem adapted to receive and display said frames and a pipe assignment subsystem adapted assign pipes to the scene elements and wherein said subsystem assigns at least one of a logical and physical transmission channel designator to each pipe and transmits a quantity representative of said pipe designator to said receiver display subsystem, wherein said system includes a plurality of auxiliary data descriptive of the content of a respective plurality of scene elements.

17. The system of claim 16 wherein an audio stream is also encoded.