FINE-GRAINED CLIENT-SIDE CONTROL OF SCALABLE MEDIA DELIVERY

• US 20100080290A1
• Filed: 09/30/2008
• Published: 04/01/2010
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mehrotra, Sanjeev
• Granted Patent: US 8,265,140 B2
• Time in Patent Office: Days
• US Class Current: 375/240.70
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Techniques and tools for adjusting quality and bit rate of multiple chunks of media delivered over a network are described. For example, each of the multiple chunks is encoded as multiple layers (e.g., a base layer and multiple embedded residual layers) for fine-grained scalability at different rate/quality points. A server stores the encoded data for the layers of chunks as well as curve information that parameterizes rate-distortion curves for the chunks. The server sends the curve information to a client. For the multiple chunks, the client uses the curve information to determine rate-distortion preferences for the respective chunks, then sends feedback indicating the rate-distortion preferences to the server. For each of the multiple chunks, the server, based at least in part upon the feedback, selects one or more scalable layers of the chunk to deliver to the client.

Claims

1. A method of delivering media data organized as plural chunks, each of the plural chunks being encoded as plural scalable layers for different quality levels with fine-grained scalability, the method comprising:

sending curve information for the plural chunks; and

for each of the plural chunks;

receiving feedback that indicates a rate-distortion preference;

based at least in part upon the rate-distortion preference, selecting one or more scalable layers from among the plural scalable layers of the chunk; and

sending the selected one or more scalable layers of the chunk.

2. The method of claim 1 wherein each of the plural chunks is encoded without dependencies on any other chunk of the plural chunks, and wherein the method further comprises sending a stream header for the media data organized as plural chunks.

3. The method of claim 1 wherein, for each of the plural chunks:

the plural scalable layers include a base layer and plural residual layers;

the base layer is always selected and sent for the chunk; and

zero or more of the plural residual layers are selected and sent for the chunk, depending on the rate-distortion preference.

4. The method of claim 1 wherein, for each of the plural chunks, the plural scalable layers include one or more residual embedded layers resulting from bit plane encoding of the chunk, each of the one or more residual embedded layers representing a bit plane of transform coefficients of the chunk.

5. The method of claim 1 wherein, for each of the plural chunks, the curve information includes rate-distortion points that parameterize a rate-distortion curve for the chunk.

6. The method of claim 5 wherein, for each of the plural scalable layers of the chunk, the curve information includes rate information that indicates a bit count for the scalable layer.

7. The method of claim 1 wherein, for each of the plural chunks, a set of parameters in the curve information for the chunk indicates overall shape of a rate-distortion curve for the chunk.

8. The method of claim 1 further comprising, for each of the plural chunks:

with a scalable media encoder, encoding the chunk as the plural scalable layers;

determining the curve information for the chunk; and

storing the plural scalable layers of the chunk and the curve information for the chunk in storage for delivery to plural media clients.

9. One or more storage media having stored thereon computer software instructions for a method of processing media data organized as plural chunks, each of the plural chunks being encoded as plural scalable layers for different quality levels with fine-grained scalability, the method comprising:

receiving curve information for the plural chunks; and

for each of the plural chunks;

determining a rate-distortion preference for the chunk using at least some of the received curve information;

sending the rate-distortion preference for the chunk; and

receiving one or more scalable layers from among the plural scalable layers of the chunk.

10. The one or more storage media of claim 9 wherein:

for each of the plural chunks, for the plural scalable layers of the chunk, the curve information at least in part indicates distortion as a function of download time; and

the determining the rate-distortion preference includes, for a segment of one or more chunks of the plural chunks;

determining a playback time constraint for the segment; and

for each of the one or more chunks in the segment, selecting a download time so as to reduce overall distortion among the one or more chunks in the segment while satisfying the playback time constraint for the segment.

11. The one or more storage media of claim 10 wherein the playback time constraint for the segment indicates a cumulative time duration, and wherein the selecting a download time for each of the one or more chunks in the segment comprises:

setting a weight parameter such that cumulative download time for the segment matches the cumulative time duration, wherein for each of the one or more chunks in the segment the weight parameter indicates the download time for the chunk according to the curve information for the chunk.

12. The one or more storage media of claim 10 wherein the selecting a download time for each of the one or more chunks in the segment comprises:

splitting the segment into plural sub-segments; and

for each of the plural sub-segments, recursively;

determining a playback time constraint for the sub-segment; and

for each of one or more chunks in the sub-segment, selecting a download time so as to reduce overall distortion among the one or more chunks in the sub-segment while satisfying the playback time constraint for the sub-segment

13. The one or more storage media of claim 9 wherein for each of the plural chunks:

the plural scalable layers include a base layer and one or more residual layers;

the base layer is always received for the chunk; and

zero or more of the one or more residual layers are received for the chunk, depending on the rate-distortion preference.

14. The one or more storage media of claim 9 wherein each of the plural chunks is encoded without dependencies on any other chunk of the plural chunks, and wherein the method further comprises receiving a stream header for the media data organized as the plural chunks, the stream header including program description information for the media data organized as the plural chunks.

15. The one or more storage media of claim 9 wherein the determining the rate-distortion preference for the chunk includes determining a count of bits or range of bits for the chunk, and wherein the sending the rate-distortion preference for the chunk includes sending a request for the count of bits or range of bits for the chunk.

16. The one or more storage media of claim 9 wherein the method further comprises, for each of the plural chunks:

with a scalable media decoder, decoding the received one or more scalable layers for the chunk; and

reconstructing a version of the chunk from the decoded one or more scalable layers.

17. A system comprising:

a processor;

a controller adapted to;

receive a stream header for media data organized as plural chunks, each of the plural chunks being encoded as plural scalable layers for different quality levels with fine-grained scalability;

receive curve information for the plural chunks, wherein, for each of the plural chunks, the curve information for the chunk indicates a different rate-distortion point associated with each of the plural scalable layers for the chunk;

determine rate-distortion preferences for the plural chunks, respectively, using the received curve information;

send the rate-distortion preferences for the plural chunks, respectively; and

for each of the plural chunks, receive one or more scalable layers from among the plural scalable layers for the chunk; and

a scalable media decoder adapted to decode the received scalable layers for the plural chunks.

18. The system of claim 17 wherein:

for each of the plural chunks, according to the different rate-distortion points, the curve information at least in part indicates distortion as a function of download time; and

the controller is adapted to determine the rate-distortion preferences by, for each of plural segments of one or more chunks of the plural chunks;

determining a playback time constraint for the segment;

for each of the one or more chunks in the segment, selecting a download time so as to reduce overall distortion among the one or more chunks in the segment while satisfying the playback time constraint for the segment; and

for each of the one or more chunks in the segment, using the selected download time for the chunk to determine a count of bits or range of bits to request for the chunk.

19. The system of claim 18 wherein the playback time constraint for the segment indicates a cumulative time duration, and wherein the selecting a download time for each of the one or more chunks in the segment comprises:

setting a weight parameter such that cumulative download time for the segment matches the cumulative time duration, wherein for each of the one or more chunks in the segment the weight parameter indicates the download time for the chunk according to the curve information for the chunk.

20. The system of claim 17 wherein for each of the plural chunks, the plural scalable layers include a base layer and one or more residual layers.