OPTIMIZED CLIENT SIDE RATE CONTROL AND INDEXED FILE LAYOUT FOR STREAMING MEDIA

• US 20090282162A1
• Filed: 05/12/2008
• Published: 11/12/2009
• Est. Priority Date: 05/12/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Siravara, Bharath, Mehrotra, Sanjeev, Smirnov, Serge, Holcomb, Thomas W., Kotteri, Kishore, Gao, Hui
• Granted Patent: US 8,379,851 B2
• Time in Patent Office: Days
• US Class Current: 709/233
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18

Abstract

An indexed file layout, comprising index information, is defined for segmented streaming of multimedia content. The index information can comprise program description information and streaming segment index information. In addition, the layout can comprise files containing streaming segments of the program, where the streaming segments are each encoded at one or more bitrates independently of other streaming segments of the program. The layout supports client switching between different bitrates at segment boundaries. Optimized client-side rate control of streaming content can be provided by defining a plurality of states, selecting available paths based on constraint conditions, and selecting a best path through the states (e.g., based on a distortion measure). In one client-side rate control solution states correspond to a specific bitrate of a specific streaming segment, and in another client-side rate control solution states correspond to a measure of client buffer fullness.

Claims

1. An indexed file layout for segmented streaming of multimedia content, the layout comprising:

one or more files containing index information for a program, the index information comprising;

program description information describing the program, wherein the program description information comprises a duration of the program and track information for each audio and video track in the program; and

streaming segment index information describing each streaming segment of the program; and

one or more files containing streaming segments of the program, wherein each streaming segment of the program is encoded at one or more bitrates, corresponding to one or more available streaming segment encodings, independently of other streaming segments of the program, and wherein the layout supports client switching between different bitrates when transitioning between different streaming segments.

2. The layout of claim 1 wherein the program comprises one or more audio tracks, one or more video tracks, or one or more audio tracks and one or more video tracks, and wherein at least one streaming segment of the program is encoded at a different number of bitrates than at least one other streaming segment of the program.

3. The layout of claim 1 wherein the streaming segment index information comprises, for each streaming segment:

a start position of the streaming segment; and

an indication of bitrates for each available streaming segment encoding of the streaming segment.

4. The layout of claim 3 wherein the streaming segment index information further comprises, for each available streaming segment encoding for each streaming segment:

a size measure of the available streaming segment encoding.

5. The layout of claim 3 wherein the streaming segment index information further comprises, for each available streaming segment encoding for each streaming segment:

a quality measure of the available streaming segment encoding; and

a decoding complexity measure of the available streaming segment encoding.

6. The layout of claim 1 wherein the index information further comprises:

file-level index information, wherein the file-level index information describes file segments of the program;

wherein the streaming segments of the program are organized by file segment, and wherein the layout comprises a separate streaming segment index information file for each file segment.

7. The layout of claim 1 wherein a client selects a streaming segment encoding of a streaming segment from available streaming segment encodings of the streaming segment using a client side rate control algorithm that uses, at least in part, the index information, wherein the client side rate control algorithm calculates a best path through states, wherein each state corresponds to a measure of client buffer fullness for a specific streaming segment encoding of a specific streaming segment.

8. The layout of claim 1 wherein a client selects a streaming segment encoding of a streaming segment from available streaming segment encodings of the streaming segment based only on the index information, available network bandwidth, desired startup delay, and client buffer information.

9. The layout of claim 1 wherein the program is segmented into the one or more streaming segments, wherein each of the one or more streaming segments covers a contiguous time range of corresponding tracks of the program, and wherein all of the one or more streaming segments of the program combine to exactly cover the whole program.

10. A method for optimizing client-side rate control of streaming content, the method comprising:

for each of a plurality of states;

selecting one or more available paths that end in the state, where the one or more available paths are those paths ending in the state that satisfy a constraint condition;

selecting a best path to the state from the one or more available paths, wherein the best path has the lowest distortion amongst the one or more available paths; and

eliminating, as available paths, all of the one or more available paths other than the best path; and

downloading, by a client from a server, one or more streaming segments encoded at bitrates corresponding to the best path to an ending state through the plurality of states, wherein each of the plurality of states corresponds to a specific bitrate encoding of a specific streaming segment.

11. The method of claim 10 wherein the constraint condition is satisfied when a sum of time it takes to download all streaming segments up to a current position is less than or equal to initial startup delay plus cumulative playback time up to the current position.

12. The method of claim 10 wherein the constraint condition is defined by the following equation:

$T_0+\underset{i=0}{\overset{n-1}{\sum <br><br>}}<br><br>t<br><br>\left[i\right]\ge <br><br>\underset{i=0}{\overset{n}{\sum <br><br>}}<br><br>s<br><br>\left[u<br><br>\left[i\right],i\right],$

where T₀ is the initial delay, “

t[i]”

is the playback duration of the i'"'"'th position, “

u[i]”

is the index of the chosen bitrate for the i'"'"'th position, “

s[u[i],i]”

is the download time for downloading the u[i] bitrate for the i'"'"'th position, and “

n”

is the current position.

13. The method of claim 10 further comprising:

receiving, by a client from a server, index information for a program, wherein the index information is used by the client to identify the plurality of states, select the one or more available paths, and select the best path.

14. The method of claim 10 further comprising:

displaying, by the client, the decoded streaming segments corresponding to the best path.

15. A method for optimizing client-side rate control of streaming content, the method comprising:

for each of a plurality of states;

selecting one or more available paths that end in the state, where the one or more available paths are those paths ending in the state that satisfy a constraint condition; and

retaining the one or more available paths as possible paths to be included in a best end-to-end path;

performing an end-to-end analysis of the possible paths to determine the best end-to-end path, wherein the best end-to-end path has the lowest overall distortion; and

downloading, by a client from a server, one or more streaming segments encoded at bitrates corresponding to the best end-to-end path through the plurality of states, wherein each of the plurality of states corresponds to a measure of client buffer fullness for a specific bitrate encoding of a specific streaming segment.

16. The method of claim 15 wherein the constraint condition is satisfied when a sum of time it takes to download all streaming segments up to a current position is less than or equal to initial startup delay plus cumulative playback time up to the current position.

17. The method of claim 15 wherein the constraint condition is defined by the following equation:

$T_0+\underset{i=0}{\overset{n-1}{\sum <br><br>}}<br><br>t<br><br>\left[i\right]\ge <br><br>\underset{i=0}{\overset{n}{\sum <br><br>}}<br><br>s<br><br>\left[u<br><br>\left[i\right],i\right],$

where T0 is the initial delay, “

t[i]”

is the playback duration of the i'"'"'th position, “

u[i]”

is the index of the chosen bitrate for the i'"'"'th position, “

s[u[i],i]”

is the download time for downloading the u[i] bitrate for the i'"'"'th position, and “

n”

is the current position.

18. The method of claim 15 further comprising:

receiving, by a client from a server, index information for a program, wherein the index information is used by the client to identify the plurality of states and select the one or more available paths.

19. The method of claim 15 further comprising:

displaying, by the client, the decoded streaming segments corresponding to the best end-to-end path.

20. The method of claim 15 wherein the plurality of states are organized into a trellis.