ADAPTIVE PLAYBACK RATE WITH LOOK-AHEAD

• US 20100146145A1
• Filed: 12/03/2009
• Published: 06/10/2010
• Est. Priority Date: 12/04/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Swarmcast, Inc.
• Inventors: Wormley, Nicholas A., Tippin, Chad A.
• Granted Patent: US 8,375,140 B2
• Time in Patent Office: Days
• US Class Current: 709/236
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

The disclosure provides for a download agent executing on a computing device to dynamically select between media files when a portion of the media file that is currently being downloaded has a portion average playback rate that is greater than a throughput rate at which the computing device is receiving media files. During the portion where the portion average playback rate is greater than the throughput rate, the download agent may dynamically transition to a different media file where the portion average playback rate for the portion is less than or equal to the throughput rate. The download agent may then transition back to the original media file after downloading the portion form the different media file.

Claims

1. A method for a client device to dynamically transition between different representations of media from a media server while presenting the media to a user, the method comprising:

determining, with the client device, a portion within a first media file being played on the client device where a portion average playback rate of the first media file for the portion is greater than a throughput rate at which the client device is downloading from the media server;

selecting, with the client device, a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate;

identifying, with the client device, a key frame in the first media file that is correlated to the portion;

identifying, with the client device, a key frame in the second media file having a timestamp that is the same as or temporally proximate to the key frame in the first media file with respect to real-time playback of the media;

transitioning, with the client device, from the first media file to the second media file;

generating, with the client device, a bit stream on the client device by splicing data from the first media file up to the key frame in the first media file with data in the second media file following the key frame in the second media file to form the bit stream; and

transmitting, with the client device, the bit stream to the media player for presentment to the user.

2. The method of claim 1, wherein splicing data comprises:

buffering data that is previous to the key frame in the first media file in a buffer; and

adding data from the second media file to the buffer that is subsequent to the key frame in the second media file.

3. The method of claim 1, wherein splicing data comprises:

transmitting data to the media player from the first media file until the key frame in the first media file;

stopping transmission of the first media file when the key frame in the first media file is played; and

transmitting data from the second media file subsequent to the key frame in the second media file.

4. The method of claim 1, further comprising transitioning back from the second media file to the first media file after downloading the portion from the second media file.

5. The method of claim 4, wherein transitioning back from the second media file to the first media file comprises transitioning back from the second media file to the first media file at a key frame correlated to an end of the portion.

6. The method of claim 1, wherein transitioning from the first media file to the second media file comprises;

determining an amount of bits buffered in a buffer;

determining a duration of the portion;

multiplying the duration of the portion and the portion average playback rate to determine a number of bits that are played in the duration;

subtracting the amount of bits buffered in the buffer from the number of bits that are played in the duration;

comparing the subtraction to a threshold; and

transitioning from the first media file to the second media file based on the comparison.

7. The method of claim 6, wherein generating a bit stream on the client device by splicing data from the first media file up to the key frame in the first media file with data in the second media file following the key frame in the second media file to form the bit stream comprises generating a bit stream on the client device by splicing data from the first media file up to the key frame in the first media file with data in the second media file following the key frame in the second media file to form the bit stream based on the comparison.

8. The method of claim 1, wherein selecting a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate comprises:

determining the portion average playback rate for the first media file;

dividing the portion average playback rate for the first media file by an overall average playback rate of the first media file to generate ratio;

dividing the throughput rate by the ratio to generate a desired overall average playback rate; and

selecting the second media file, wherein an overall average playback rate for the second media file is substantially close to the desired overall average playback rate.

9. The method of claim 1, wherein selecting a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate comprises:

determining the portion average playback rate for the first media file;

scanning a portion average playback rate for each representation of media; and

selecting the second media file based on the scan.

10. The method of claim 1, wherein the media file is a variable bit rate (VBR) video.

11. The method of claim 1, wherein the key frame in the first media file and the key frame in the second media file are intra frames.

12. The method of claim 1, wherein the key frame in the first media file and the key frame in the second media file are frames that provide an entire encoded picture.

13. The method of claim 1, wherein the key frame in the first media file and the key frame in the second media file are frames that are encoded without reference to any other frame.

14. The method of claim 1, further comprising:

determining an amount of byte offset between the identified key frame of the second media file and the identified key frame of the first media file; and

correlating a timestamp of the key frame in the first media file to a timestamp of the key frame in the second media file based on the amount of byte offset.

18. The client device of claim 14, wherein splicing data comprises:

the stream agent transmitting data to the media player from the first media file until the key frame in the first media file, stopping transmission of the first media file when the key frame in the first media file is played, and transmitting data from the second media file subsequent to the key frame in the second media file.

15. A computer-readable storage medium comprising instructions that cause one or more processors to:

determine a portion within a first media file being played on a client device where a portion average playback rate of the first media file for the portion is greater than a throughput rate at which the client device is downloading from the media server;

select a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate;

identify a key frame in the first media file that is correlated to the portion;

identify a key frame in the second media file having a timestamp that is the same as or temporally proximate to the key frame in the first media file with respect to real-time playback of the media;

transition from the first media file to the second media file;

generate a bit stream on the client device by splicing data from the first media file up to the key frame in the first media file with data in the second media file following the key frame in the second media file to form the bit stream; and

transmit the bit stream to a media player for presentment to a user.

16. A client device to dynamically transition between different representations of media from a media server while presenting the media to a user, the client device comprising:

a processor; and

a download agent executing on the processor, the download agent comprising;

a playback controller that;

determines a portion within a first media file being played on the client device where a portion average playback rate of the first media file for the portion is greater than a throughput rate at which the client device is downloading from the media server; and

selects a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate;

a source manager that transitions from the first media file to the second media file;

a stream agent that identifies a key frame in the first media file that is subsequent to a current frame begin played by the media player, identifies a key frame in the second media file having a timestamp that is the same as or temporally proximate to the key frame in the first media file with respect to real-time playback of the media, generates a bit stream, and transmits the bit stream to the media player for presentment to the user, wherein the stream agent generates the bit stream by splicing data from the first media file up to the key frame in the first media file with data in the second media file following the key frame in the second media file.

17. The client device of claim 16, wherein splicing data comprises:

the stream agent buffering data that is previous to the key frame in the first media file in a buffer, and adding data from the second media file to the buffer that is subsequent to the key frame in the second media file.

19. The client device of claim 16, wherein the playback controller causes the source manager to transition back from the second media file to the first media file after downloading the portion from the second media file.

20. The client device of claim 19, wherein the playback controller causes the source manager to transition back from the second media file to the first media file at a key frame correlated to an end of the portion.

21. The client device of claim 16,

wherein the playback controller determines an amount of bits buffered in the media player, determines a duration of the portion, multiplies the duration of the portion and the portion average playback rate to determine a number of bits that are played in the duration, subtracts the amount of bits buffered in a buffer from the number of bits that are played in the duration, compares the subtraction to a threshold, and

wherein the source manager transitions from the first media file to the second media file based on the comparison.

22. The client device of claim 16, wherein the playback controller selects a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate by:

determining the portion average playback rate for the first media file;

dividing the portion average playback rate for the first media file by an overall average playback rate of the first media file to generate ratio;

dividing the throughput rate by the ratio to generate a desired overall average playback rate; and

selecting the second media file, wherein an overall average playback rate for the second media file is substantially close to the desired overall average playback rate.

23. The client device of claim 16, wherein the playback controller selects a second media file where a portion average playback rate of the second media file for the portion is less than or equal to the throughput rate by:

determining the portion average playback rate for the first media file;

scanning a portion average playback rate for each representation of media; and

selecting the second media file based on the scan.

24. The client device of claim 16, wherein the media file is a variable bit rate (VBR) video.

25. The client device of claim 16, wherein the key frame in the first media file and the key frame in the second media file are intra frames.

26. The client device of claim 16, wherein the key frame in the first media file and the key frame in the second media file are frames that provide an entire encoded picture.

27. The client device of claim 16, wherein the key frame in the first media file and the key frame in the second media file are frames that are encoded without reference to any other frame.

28. The client device of claim 16, further comprising:

a temporal metadata that determines an amount of byte offset between the identified key frame of the second media file and the identified key frame of the first media file, and correlates a timestamp of the key frame in the first media file to a timestamp of the key frame in the second media file based on the amount of byte offset.