Method and apparatus that provides a scalable media delivery system

• US 6,490,627 B1
• Filed: 06/28/1999
• Issued: 12/03/2002
• Est. Priority Date: 12/17/1996
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle Corporation
• Inventors: Guruswamy, Kowsik, Kalra, Devendra, Krishnamohan, Karnamadakala, Ramamoorthy, Venkatasubbarao, Burr, Timothy J., Balakrishnan, Jeyendran
• Primary Examiner(s): Dinh, Dung C.
• Application Number:
  US09/340,624
• Time in Patent Office: 1,254 Days
• Field of Search: 348/384.1, 348/387.1, 348/390.1, 348/396.1, 348/410.1, 348/420.1, 348/421.1, 382/232, 382/239, 382/240, 709/230, 709/231, 709/247
• US Class Current: 709/231
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04N 19/39 involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

The present invention provides an apparatus and method for encoding, storing, transmitting and decoding multimedia information in the form of scalable, streamed digital data. A base stream containing basic informational content and subsequent streams containing additive informational content are initially created from standard digital multimedia data by a transcoder. Client computers, each of which may have different configurations and capabilities are capable of accessing a stream server that contains the scalable streamed digital data. Each different client computer, therefore, may access different stream combinations according to a profile associated with each different client computer. Thus, the streams accessed from the server are tailored to match the profile of each client computer so that the best combination of streams can be provided to maximize the resolution of the 3D, audio and video components.

Claims

1. A method of communicating multimedia digital data from a server computer to a client computer comprising the steps of:

determining multimedia characteristics of said client computer;

determining a selected set of adaptive streams containing said multimedia digital data relating to sight and intended to be transmitted to said client computer based upon said determined multimedia characteristics said adaptive streams being stored on a memory of said server computer and formed of a base stream and a plurality of additive streams, said additive streams containing additive data which corresponds to base data associated with said base stream, such that additional additive data from each additive stream provides increasingly greater resolution and wherein said selected set of adaptive streams is derived from a sequence of digital video pictures, each of said digital video pictures containing a plurality of slices, and each of said slices containing a plurality of blocks, each block representing one of luminance and chrominance information, said luminance and chrominance information in each of said blocks being segmented into a base stream and a plurality of additive streams, each said additive stream containing additive data that corresponds to base data associated with said base stream, such that additional additive data from each additive stream provides increasingly greater resolution of said one luminance and chrominance information for each block; and

transmitting said selected set of adaptive streams from said server computer to said client computer.

2. A method according to claim 1 wherein said base stream comprises a single DCT coefficient, and each additive stream comprises an increasingly greater number of DCT coefficients.

3. A method according to claim 2 wherein each block is an 8 by 8 array of data and there are 7 additive streams.

4. A method according to claim 3 wherein said single DCT coefficient of said base stream comprises a DCT 0 coefficient and each additive stream comprises DCT coefficients that are immediately adjacent to DCT coefficients that have been previously selected for inclusion in the previously defined stream, such that the DCT coefficients used for each additive stream within the DCT matrix forms a backwards “

L”

.

5. A method according to claim 1 wherein said step of determining said selected set of adaptive streams further includes the steps of:

receiving at said server computer, from said client computer, a user specified preference for quality of vision as compared to quality of audio; and

using said user specified preference to determine said selected set of said adaptive streams for sight.

6. A method according to claim 1 wherein said step of determining said selected set of adaptive streams further includes the steps of determining available bandwidth and determining said selected set of adaptive streams to obtain an updated selected set of adaptive streams;

and

during said step of transmitting, transmitting said updated selected set of adaptive streams to said client computer.

7. A method according to claim 1 wherein said multimedia digital data relates to sight in the form of a rendered graphical image such that at least one object within said rendered image contains a texture in the form of a digital video picture, said digital video picture containing a plurality of slices, and each of said slices containing a plurality of blocks, each block representing one of luminance and chrominance information, said luminance and chrominance information in each of said blocks being segmented into a base stream and a plurality of additive streams, each said additive stream containing additive data that corresponds to base data associated with said base stream, such that additional additive data from each additive stream provides increasingly greater resolution of said one luminance and chrominance information for each block.

8. A method according to claim 1 wherein said selected set of adaptive streams and other graphics data are usable to produce a sequence of graphical image frames from a scene containing a plurality of objects;

wherein said selected set of adaptive streams includes attribute adaptive stream data that includes base attribute adaptive stream data and additive attribute adaptive stream data;

wherein said other graphics data includes scene definition data including global scene data and spatial partitioning data; and

wherein said step of determining said selected set of adaptive streams further includes the steps of;

obtaining a graphic priority table that identifies relative priorities for said attribute adaptive stream data on an object by object basis,

using said relative priorities to determine a priority order of which of said attribute adaptive stream data to transmit more frequently after said transmission step is initiated; and

wherein said transmitting step transmits, from said server computer to said client computer, said global scene data, said spatial partitioning data, and said base and additive attribute adaptive stream data based upon said determined priority order.

9. A method according to claim 1 wherein said attribute adaptive stream data includes geometry attribute adaptive stream data.

10. A method according to claim 1 wherein said selected set of adaptive streams and other graphics data are usable to produce a sequence of graphical image frames from a scene containing a plurality of objects, wherein said adaptive streams include, for each object, attribute adaptive stream data that includes base attribute adaptive stream data and additive attribute adaptive stream data, wherein said other graphics data includes scene definition data including global scene data and spatial partitioning data;

and said method further comprises the steps of;

retrieving at said client computer said global scene data including a visual portion of the scene data and said spatial positioning data and determining boundaries of said scene and locations of objects in said scene;

drawing a first frame relating to a first visual portion of the scene at said client computer using transmitted base and attribute adaptive stream data; and

determining at said client computer whether to send a message to said server computer indicating that one of updated base attribute adaptive stream data and updated additive attribute adaptive stream data is required due to a change in one of level of detail and said visual portion.

11. A method according to claim 1 wherein said selected set of adaptive streams and other graphics data are usable to produce a sequence of graphical image frames from a scene containing a plurality of objects, wherein said adaptive streams include, for each object, attribute adaptive stream data that includes base attribute adaptive stream data and additive attribute adaptive stream data, wherein said other graphics data includes scene definition data including global scene data and spatial partitioning data;

and said method further comprises the steps of;

retrieving at said client computer said global scene data including a visual portion of the scene data and said spatial positioning data and determining boundaries of said scene and locations of objects in said scene;

drawing a first frame relating to a first visual portion of the scene at said client computer using transmitted base and attribute adaptive stream data, said first visual portion of the scene containing a first object that is further from a camera position than a second object, and said transmitted attribute adaptive stream data associated with said first object having a lower level of detail than said transmitted attribute adaptive stream data associated with said second object.

12. A computer-readable medium carrying instructions for communicating multimedia digital data from a server computer to a client computer, the instructions comprising instructions for performing the steps of:

determining multimedia characteristics of said client computer;

determining a selected set of adaptive streams containing said multimedia digital data relating to sight and intended to be transmitted to said client computer based upon said determined multimedia characteristics,

said adaptive streams being stored on a memory of said server computer and formed of a base stream and a plurality of additive streams, said additive streams containing additive data which corresponds to base data associated with said base stream, such that additional additive data from each additive stream provides increasingly greater resolution and wherein said selected set of adaptive steams is derived from a sequence of digital video pictures,

each of said digital video pictures containing a plurality of slices, and

each of said slices containing a plurality of blocks,

each block representing one of luminance and chrominance information, said luminance and chrominance information in each of said blocks being segmented into a base stream and a plurality of additive streams,

each said additive steam containing additive data that corresponds to base data associated with said base stream, such that additional additive data from each additive stream provides increasingly greater resolution of said one luminance and chrominance information for each block; and

transmitting said selected set of adaptive streams from said server computer to said client computer.

13. A computer-readable medium according to claim 12 wherein said base stream comprises a single DCT coefficient, and each additive stream comprises an increasingly greater number of DCT coefficients.

14. A computer-readable medium according to claim 12 wherein each block is an 28 by 28 array of data and there are 27 additive streams.

15. A computer-readable medium according to claim 14 wherein said single DCT coefficient of said base stream comprises a DCT 0 coefficient and each additive stream comprises DCT coefficients that are immediately adjacent to DCT coefficients that have been previously selected for inclusion in the previously defined stream, such that the DCT coefficients used for each additive stream within the DCT matrix forms a backwards “

L”

.

16. A computer-readable medium according to claim 12 wherein said step of determining said selected set of adaptive streams further includes the steps of:

receiving at said server computer, from said client computer, a user specified preference for quality of vision as compared to quality of audio; and

using said user specified preference to determine said selected set of said adaptive streams for sight.

17. A computer-readable medium according to claim 12 wherein said step of determining said selected set of adaptive streams further includes the steps of

determining available bandwidth and determining said selected set of adaptive streams to obtain an updated selected set of adaptive streams;

and

during said step of transmitting, transmitting said updated selected set of adaptive streams to said client computer.

18. A computer-readable medium according to claim 12 wherein

said multimedia digital data relates to sight in the form of a rendered graphical image such that at least one object within said rendered image contains a texture in the form of a digital video picture,

said digital video picture containing a plurality of slices, and

each of said slices containing a plurality of blocks,

each block representing one of luminance and chrominance information, said luminance and chrominance information in each of said blocks being segmented into a base stream and a plurality of additive streams,

each said additive stream containing additive data that corresponds to base data associated with said base stream, such that additional additive data from each additive stream provides increasingly greater resolution of said one luminance and chrominance information for each block.

19. A computer-readable medium according to claim 12 wherein said selected set of adaptive streams and other graphics data are usable to produce a sequence of graphical image frames from a scene containing a plurality of objects;

wherein said selected set of adaptive streams includes attribute adaptive stream data that includes base attribute adaptive stream data and additive attribute adaptive stream data;

wherein said other graphics data includes scene definition data including global scene data and spatial partitioning data; and

wherein said step of determining said selected set of adaptive streams further includes the steps of;

obtaining a graphic priority table that identifies relative priorities for said attribute adaptive stream data on an object by object basis,

using said relative priorities to determine a priority order of which of said attribute adaptive stream data to transmit more frequently after said transmission step is initiated; and

wherein said transmitting step transmits, from said server computer to said client computer, said global scene data, said spatial partitioning data, and said base and additive attribute adaptive stream data based upon said determined priority order.

20. A computer-readable medium according to claim 12 wherein said attribute adaptive stream data includes geometry attribute adaptive stream data.

21. A computer-readable medium according to claim 12 wherein said selected set of adaptive streams and other graphics data are usable to produce a sequence of graphical image frames from a scene containing a plurality of objects, wherein said adaptive streams include, for each object, attribute adaptive stream data that includes base attribute adaptive stream data and additive attribute adaptive stream data, wherein said other graphics data includes scene definition data including global scene data and spatial partitioning data;

and said instructions further comprising instructions for performing the steps of;

retrieving at said client computer said global scene data including a visual portion of the scene data and said spatial positioning data and determining boundaries of said scene and locations of objects in said scene;

drawing a first frame relating to a first visual portion of the scene at said client computer using transmitted base and attribute adaptive stream data; and

determining at said client computer whether to send a message to said server computer indicating that one of updated base attribute adaptive stream data and updated additive attribute adaptive stream data is required due to a change in one of level of detail and said visual portion.

22. A computer-readable medium according to claim 12 wherein said selected set of adaptive streams and other graphics data are usable to produce a sequence of graphical image frames from a scene containing a plurality of objects, wherein said adaptive streams include, for each object, attribute adaptive stream data that includes base attribute adaptive stream data and additive attribute adaptive stream data, wherein said other graphics data includes scene definition data including global scene data and spatial partitioning data;

and said instructions further comprising instructions for performing the steps of;

retrieving at said client computer said global scene data including a visual portion of the scene data and said spatial positioning data and determining boundaries of said scene and locations of objects in said scene;

drawing a first frame relating to a first visual portion of the scene at said client computer using transmitted base and attribute adaptive stream data, said first visual portion of the scene containing a first object that is further from a camera position than a second object, and said transmitted attribute adaptive stream data associated with said first object having a lower level of detail than said transmitted attribute adaptive stream data associated with said second object.