Predictive adaptive media streaming

• US 8,396,983 B1
• Filed: 03/13/2012
• Issued: 03/12/2013
• Est. Priority Date: 03/13/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Inskip, Thomas, MacLean, Duncan
• Primary Examiner(s): BAYARD, DJENANE M
• Application Number:
  US13/419,041
• Time in Patent Office: 364 Days
• Field of Search: 709/200, 709/203, 709/233
• US Class Current: 709/233
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

A method and corresponding apparatus for controlling streaming of media content from a server to a client. A computing device predicts the bit rate of an upcoming portion of the media content for each of a plurality of variable bit-rate versions of the media content, and uses the predicted bit rates, as well as a measure of network bandwidth, as a basis to select one of the variable bit-rate versions to be streamed to the client. By repeating this prediction and selection function on a sliding window basis over the course of the media content, the selection of variable bit-rate version can dynamically change, to help accommodate variations in bit rate throughout the media content as well as variations in network bandwidth.

Claims

1. A method for controlling streaming of media content from a server to a client, wherein the server has access to a plurality of variable bit rate (VBR) versions of the media content, the method comprising carrying out the following functions for each of a plurality of sequentially occurring portions of the media content while the media content is being streamed from the server to the client:

identifying, in advance of the portion of the media content being received by the client, one or more of the VBR versions of the media content, the identifying being based at least in part on determining that the portion of media content in each of the one or more VBR versions has a bit rate lower than a bandwidth threshold;

selecting, from the determined one or more VBR versions of the media content, a highest bit-rate VBR version of the media content; and

causing the selected VBR version of the media content to be streamed from the server to the client,

the method further comprising storing an index of the media content, the index containing data representing bit rates of the VBR versions over time, wherein selecting, from the determined one or more VBR versions of the media content, a highest bit-rate VBR version of the media content comprises using the stored index as a basis for determining a bit rate of each VBR version of the media content over the next sequential portion of the media content.

2. The method of claim 1, wherein the identifying is further based at least in part on level of fullness of a playback buffer at the client.

3. The method of claim 1, wherein identifying, in advance of the portion of the media content being received by the client, one or more of the VBR versions of the media content comprises:

for each VBR version of the plurality, predicting a measure of bit rate of the portion of the media content;

determining for each VBR version of the plurality whether the predicted measure of bit rate of the portion of the media content is lower than the bandwidth threshold.

4. The method of claim 3, wherein the measure comprises an average or a maximum.

5. The method of claim 3, further comprising, for each of the plurality of sequentially occurring portions of the media content while the media content is being streamed from the server to the client:

determining the bandwidth threshold based on evaluation of network bandwidth between the server and the client.

6. The method of claim 5, wherein the sequentially occurring portions of the media content overlap with each other to cooperatively define a sliding window over the media content.

7. The method of claim 5, wherein the sequentially occurring portions of the media content are mutually exclusive portions of the media content.

8. The method of claim 1, wherein causing the selected VBR version of the media content to be streamed from the server to the client comprises causing the server to transition from streaming another one of the VBR versions of the media content to streaming the selected VBR version of the media content.

9. The method of claim 1, carried out by the server,

wherein the method further comprises receiving from the client a measure of network bandwidth, the bandwidth threshold being based on the received measure.

10. A non-transitory computer readable medium having stored thereon instructions executable to cause a device to carry out the following functions for each of a plurality of sequentially occurring portions of media content during streaming of the media content from a server to the device:

determining network bandwidth for communication from the server to the device;

identifying, in advance of the portion of the media content being received by the device, one or more of a plurality of variable bit rate (VBR) versions of the media content, the identifying being based at least in part on the portion of the media content in each of the one or more VBR versions having a bit rate less than or equal to the determined network bandwidth;

selecting, from the identified one or more VBR versions of the media content, a highest bit-rate VBR version of the media content; and

transmitting to the server a control signal requesting the server to stream the selected VBR version of the media content to the device,

wherein the instructions are further executable to cause the device to receive and store an index of the media content, the index including data representing bit rates of the plurality of VBR versions over time, wherein the data in the index specifies, for each VBR version of the plurality, sizes and durations of sequential chunks of the VBR version,

wherein identifying one or more of the VBR versions of the media content comprises using the index as a basis for the identifying, and

wherein using the index as a basis for the identifying comprises (i) determining from the index a measure of chunk-size per time for the portion of the media content, (ii) using the determined measure as a representation of bit rate of the portion of the media content, and (iii) determining that the VBR version is one in which the portion of the media content will have a bit rate less than or equal to the determined network bandwidth if the representation of bit rate of the portion of the media content is less than or equal to the determined network bandwidth.

11. The non-transitory computer readable medium of claim 10, wherein the identifying is further based at least in part on a level of fullness of a playback buffer at the device.

12. The non-transitory computer readable medium of claim 10, wherein the device is a handheld wireless communication device, and wherein the non-transitory computer readable medium comprises data storage of the device.

13. The non-transitory computer readable medium of claim 10, wherein the measure of chunk size per time for the portion of the media content comprises an average or a maximum.

14. The non-transitory computer readable medium of claim 10, wherein determining network bandwidth for communication from the server to the device comprises measuring a rate of arrival of data at the device from the server, the network bandwidth being based at least in part on the measured rate of arrival.

15. The non-transitory computer readable medium of claim 10, wherein the sequentially occurring portions of the media content overlap with each other to cooperatively define a sliding window over the media content.

16. The non-transitory computer readable medium of claim 15, wherein the sliding window is size-bound or time-bound.

17. The non-transitory computer readable medium of claim 10, wherein the sequentially occurring portions of the media content are mutually exclusive.

18. The non-transitory computer readable medium of claim 10, wherein the control signal requests the server to transition from streaming another one of the VBR versions of the media content to the device to streaming the selected VBR version of the media content to the device.

19. A non-transitory computer readable medium having stored thereon instructions executable to cause a device to carry out the following functions for each of a plurality of sequentially occurring portions of media content during streaming of the media content from a server to the device:

determining network bandwidth for communication from the server to the device;

identifying, in advance of the portion of the media content being received by the device, one or more of a plurality of variable bit rate (VBR) versions of the media content, the identifying being based at least in part on the portion of the media content in each of the one or more VBR versions having a bit rate less than or equal to the determined network bandwidth;

selecting, from the identified one or more VBR versions of the media content, a highest bit-rate VBR version of the media content; and

transmitting to the server a control signal requesting the server to stream the selected VBR version of the media content to the device,

wherein the instructions are further executable to cause the device to receive and store an index of the media content, the index including data representing bit rates of the plurality of VBR versions over time, wherein the data in the index specifies for each VBR version a mapping between media content time positions and media content data positions,

wherein identifying one or more of the VBR versions of the media content comprises using the index as a basis for the identifying, and

wherein using the index as a basis for the identifying comprises (i) determining from the index a measure of media content data-size per time-range for the portion of the media content, (ii) using the measure as a representation of bit rate of the portion of the media content, and (iii) determining that the VBR version is one in which the portion of the media content will have a bit rate less than or equal to the determined network bandwidth if the representation of bit rate of the portion of the media content is less than or equal to the determined network bandwidth.

20. The non-transitory computer readable medium of claim 19, wherein the measure comprises an average or a maximum.

21. The non-transitory computer readable medium of claim 19, wherein determining network bandwidth for communication from the server to the device comprises measuring a rate of arrival of data at the device from the server, the network bandwidth being based at least in part on the measured rate of arrival.

22. The non-transitory computer readable medium of claim 19, wherein the sequentially occurring portions of the media content overlap with each other to cooperatively define a sliding window over the media content.

23. The non-transitory computer readable medium of claim 22, wherein the sliding window is size-bound or time-bound.

24. The non-transitory computer readable medium of claim 19, wherein the sequentially occurring portions of the media content are mutually exclusive.

25. The non-transitory computer readable medium of claim 19, wherein the control signal requests the server to transition from streaming another one of the VBR versions of the media content to the device to streaming the selected VBR version of the media content to the device.

26. The non-transitory computer readable medium of claim 19, wherein the identifying is further based at least in part on a level of fullness of a playback buffer at the device.

27. The non-transitory computer readable medium of claim 19, wherein the device is a handheld wireless communication device, and wherein the non-transitory computer readable medium comprises data storage of the device.

28. A computing device comprising:

a network communication interface;

a user interface;

a processor;

data storage; and

program instructions stored in the data storage and executable by the processor to carry out at least the following functions;

(a) receiving, by the network communication interface, media content being streamed to the computing device from a server,

(b) playing out, at the user interface, the media content being received; and

(c) while receiving the media content being streamed to the computing device from the server, periodically (i) determining a measure of network bandwidth for communication from the server to the computing device, (ii) evaluating a plurality of variable-bit-rate (VBR) versions of the media content and selecting, from the plurality of VBR versions, a highest bit-rate VBR version having a bit rate over an upcoming sliding window of the media content that will be less than or equal to the determined network bandwidth, and (iii) transmitting by the network communication interface to the server a request for the server to stream the selected VBR version to the computing device,

wherein the data storage holds an index of the media content, the index containing data representing bit rates of the VBR versions over time, wherein evaluating the plurality of VBR versions of the media content and selecting, from the plurality of VBR versions, a highest bit-rate VBR version having a bit rate over an upcoming sliding window of the media content that will be less than or equal to the determined network bandwidth comprises using the index as a basis for determining a bit rate of each VBR version of the media content over the upcoming sliding window of the media content.

29. The computing device of claim 28, wherein:

during the playing out of the media content, the data storage defines a playback buffer for the media content, and

the functions further comprise selecting the highest bit-rate VBR version based on a level of fullness of the playback buffer.

30. The computing device of claim 28, wherein the request is for the server to transition from streaming another one of the VBR versions to streaming the selected VBR version.