Adaptive playback rate with look-ahead

• US 8,375,140 B2
• Filed: 12/03/2009
• Issued: 02/12/2013
• Est. Priority Date: 12/04/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Tippin, Chad A., Wormley, Nicholas A.
• Primary Examiner(s): Backer, Firmin
• Assistant Examiner(s): EDWARDS, JAMES A
• Application Number:
  US12/630,582
• Publication Number:
  US 20100146145A1
• Time in Patent Office: 1,167 Days
• Field of Search: 709/236
• US Class Current: 709/236
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

The disclosure provides for a download agent executing on a computing device to dynamically select between media files when a portion of the media file that is currently being downloaded has a portion average playback rate that is greater than a throughput rate at which the computing device is receiving media files. During the portion where the portion average playback rate is greater than the throughput rate, the download agent may dynamically transition to a different media file where the portion average playback rate for the portion is less than or equal to the throughput rate. The download agent may then transition back to the original media file after downloading the portion form the different media file.

Claims

1. A method, comprising:

employing a processor executing computer executable instructions embodied on at least one non-transitory computer readable medium to perform operations comprising;

downloading a first media file from a media server at a throughput rate and concurrently playing the first media file, wherein the first media file is a first version of a media asset, the first media file is encoded at a variable bit rate and has a first overall average playback rate less than or equal to the throughput rate, the first media file has a plurality of portions with respective portion average playback rates, wherein at least two portions have different portion average playback rates;

forecasting an interruption will occur in playing of the first media file by identifying an undownloaded portion of the first media file that has a portion average playback rate that is greater than the throughput rate;

in response to forecasting the interruption;

selecting a second media file where a portion average playback rate of the second media file for a portion that is within the second media file and that corresponds to the undownloaded portion is less than or equal to the throughput rate, wherein the second media file is a second version of the media asset, the second media file is encoded at a variable bit rate and has a second overall average playback rate that is lower than the first overall average playback rate, the second media file has a plurality of portions with respective portion average playback rates, wherein at least two portions have different portion average playback rates;

identifying a first playback point in the first and second media files correlated to the undownloaded portion;

downloading one or more portions of the second media file that include at least the first playback point and the portion within the second media file that corresponds to the undownloaded portion; and

transitioning playing of the first media file to playing the second media file at the identified first playback point.

2. The method of claim 1, wherein the first playback point is a key frame in the first media file that corresponds to a key frame in the second media file.

3. The method of claim 2, wherein selecting the second media file comprises:

determining the portion average playback rate for the undownloaded portion of the first media file;

scanning a portion average playback rate for each of a plurality of media files corresponding to versions of the media asset for portions that correspond to the undownloaded portion; and

selecting the second media file based on the scan.

4. The method of claim 3, wherein the selected second media file has a portion corresponding to the undownloaded portion with a highest portion average playback rate of the portions that correspond to the undownloaded portion of the plurality of media files.

5. The method of claim 2, wherein the key frame in the first media file and the key frame in the second media file are intra frames.

6. The method of claim 2, wherein the key frame in the first media file and the key frame in the second media file are frames that provide an entire encoded picture.

7. The method of claim 2, wherein the key frame in the first media file and the key frame in the second media file are frames that are encoded without reference to any other frame.

8. The method of claim 2, further comprising:

determining an amount of byte offset between the key frame of the second media file and the key frame of the first media file; and

correlating a timestamp of the key frame in the first media file to a timestamp of the key frame in the second media file based on the amount of byte offset.

9. The method of claim 1, wherein the first playback point is a temporal location in the first media file that corresponds to a temporal location in the second media file.

10. The method of claim 1, further comprising transitioning playing of the second media file to playing of the first media file after playing of the portion within the second media file that corresponds to the undownloaded portion, wherein playing of the first media file occurs at a second playback point after the undownloaded portion.

11. The method of claim 10, wherein the second playback point after the undownloaded portion is a key frame in the second media file correlated to an end of the portion within the second media file that corresponds to the undownloaded portion.

12. The method of claim 1, wherein forecasting the interruption further comprises:

determining an amount of bits buffered in a buffer;

determining a duration of the undownloaded portion;

multiplying the duration of the undownloaded portion and the portion average playback rate of the undownloaded portion to determine a number of bits that are played in the duration;

subtracting the amount of bits buffered in the buffer from the number of bits that are played in the duration;

determining whether the result of the subtraction meets a threshold; and

in response to a determination that the result of the subtraction meets the threshold, forecasting that the interruption will occur.

13. The method of claim 1, wherein selecting the second media file further comprises:

determining the portion average playback rate for the undownloaded portion of the first media file;

dividing the portion average playback rate for the undownloaded portion by an overall average playback rate of the first media file to generate a ratio;

dividing the throughput rate by the ratio to generate a desired overall average playback rate; and

selecting the second media file from a plurality of media files corresponding to versions of the media asset, wherein an overall average playback rate for the second media file is closest to the desired overall average playback rate.

14. The method of claim 1, wherein the media asset is a variable bit rate video.

15. A non-transitory computer-readable storage medium comprising instructions that cause one or more processors to perform operations comprising:

downloading a first media file from a media server at a throughput rate and concurrently play the first media file, wherein the first media file is a first version of a media asset, the first media file is encoded at a variable bit rate and has a first overall average playback rate less than or equal to the throughput rate, the first media file has a plurality of portions with respective portion average playback rates, wherein at least two portions have different portion average playback rates;

forecasting an interruption will occur in playing of the first media file by identifying an undownloaded portion of the first media file that has a portion average playback rate that is greater than the throughput rate;

in response to forecasting the interruption;

selecting a second media file where a portion average playback rate of the second media file for a portion that is within the second media file and that corresponds to the undownloaded portion is less than or equal to the throughput rate, wherein the second media file is a second version of the media asset, the second media file is encoded at a variable bit rate and has a second overall average playback rate that is lower than the first overall average playback rate, the second media file has a plurality of portions with respective portion average playback rates, wherein at least two portions have different portion average playback rates;

identifying a playback point in the first and second media files correlated to the undownloaded portion;

downloading one or more portions of the second media file that include at least the playback point and the portion within the second media file that corresponds to the undownloaded portion; and

transitioning playing of the first media file to playing the second media file at the identified playback point.

16. A system, comprising:

a processor;

a memory communicatively coupled to processor, the memory having stored therein computer-executable instructions, comprising;

a playback controller that;

downloads a first media file from a media server at a throughput rate and concurrently plays the first media file, wherein the first media file is a first version of a media asset, the first media file is encoded at a variable bit rate and has a first overall average playback rate less than or equal to the throughput rate, the first media file has a plurality of portions with respective portion average playback rates, wherein at least two portions have different portion average playback rates;

forecasts an interruption will occur in playing of the first media file by identifying an undownloaded portion of the first media file that has a portion average playback that is greater than the throughput rate; and

in response to forecasting the interruption;

selects a second media file where a portion average playback rate of the second media file for a portion that is within the second media file and that corresponds to the undownloaded portion is less than or equal to the throughput rate, wherein the second media file is a second version of the media asset, the second media file is encoded at a variable bit rate and has a second overall average playback rate that is lower than the first overall average playback rate, the second media file has a plurality of portions with respective portion average playback rates, wherein at least two portions have different portion average playback rates;

identifies a first playback point in the first and second media files correlated to the undownloaded portion; and

downloads one or more portions of the second media file that include at least the first playback point and the portion within the second media file that corresponds to the undownloaded portion; and

a source manager that transitions playing of the first media file to playing the second media file at the identified first playback point.

17. The system of claim 16, wherein the first playback point is a key frame in the first media file that corresponds to a key frame in the second media file.

18. The system of claim 17, wherein the key frame in the first media file and the key frame in the second media file are intra frames.

19. The system of claim 17, wherein the key frame in the first media file and the key frame in the second media file are frames that provide an entire encoded picture.

20. The system of claim 17, wherein the key frame in the first media file and the key frame in the second media file are frames that are encoded without reference to any other frame.

21. The system of claim 17, further comprising:

a temporal metadata that indicates an amount of byte offset between the identified key frame of the second media file and the identified key frame of the first media file, and correlates a timestamp of the key frame in the first media file to a timestamp of the key frame in the second media file based on the amount of byte offset.

22. The system of claim 16, wherein the first playback point is a temporal location in the first media file that corresponds to a temporal location in the second media file.

23. The system of claim 16, wherein the playback controller causes the source manager to transition playing of the second media file to playing of the first media file after playing of the portion within the second media file that corresponds to the undownloaded portion, wherein playing of the first media file occurs at a second playback point after the undownloaded portion.

24. The system of claim 23, wherein the second playback point after the undownloaded portion is a key frame in the second media file correlated to an end of the portion within the second media file that corresponds to the undownloaded portion.

25. The system of claim 16,

wherein forecasts the interruption further comprises:

determines an amount of bits buffered in the media player;

determines a duration of the undownloaded portion;

multiplies the duration of the undownloaded portion and the portion average playback rate of the undownloaded portion to determine a number of bits that are played in the duration;

subtracts the amount of bits buffered in a buffer from the number of bits that are played in the duration;

determines whether the result of the subtraction meets a threshold; and

in response to a determination that the result of the subtraction meets the threshold, forecasts that the interruption will occur.

26. The system of claim 16, wherein selects the second media file further comprises:

determines the portion average playback rate for the undownloaded portion of the first media file;

divides the portion average playback rate for the undownloaded portion by an overall average playback rate of the first media file to generate a ratio;

divides the throughput rate by the ratio to generate a desired overall average playback rate; and

selects the second media file from a plurality of media files corresponding to versions of the media asset, wherein an overall average playback rate for the second media file is closest to the desired overall average playback rate.

27. The system of claim 16, wherein selects the second media file further comprises:

determines the portion average playback rate for the undownloaded portion of the first media file;

scans a portion average playback rate for each of a plurality of media files corresponding to versions of the media asset for portions that correspond to the undownloaded portion; and

selects the second media file based on the scan.

28. The system of claim 27, wherein the selected second media file has a portion corresponding to the undownloaded portion with a highest portion average playback rate of the portions that correspond to the undownloaded portion of the plurality of media files.