Adaptive trick play streaming

• US 8,407,747 B1
• Filed: 03/13/2012
• Issued: 03/26/2013
• Est. Priority Date: 03/13/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Inskip, Thomas, MacLean, Duncan
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): MENGESHA, MULUGETA A
• Application Number:
  US13/419,048
• Time in Patent Office: 378 Days
• Field of Search: 725/88, 725/90, 725/94, 725/99, 386/68
• US Class Current: 725/90
• CPC Class Codes: H04N 21/234381 by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312

Abstract

Methods and devices for controlling trick-play video streaming. Upon receipt of a request to play video at a fast-play (e.g., fast-forward or fast-reverse) speed that is some multiple of a normal playback speed, one of multiple possible frame-rate versions of a trick-play track may be selected based on the requested fast-play speed and an available network bandwidth, and a server may stream the selected frame-rate version of the trick-play track to a client for playout. The various frame-rate versions may differ from each other in terms of a rate at which they skip I-frames of the video.

Claims

1. A non-transitory computer readable medium having stored thereon instructions executable to cause a processing unit to carry out functions comprising:

detecting a request to play video at a fast-play speed defining a multiple of a normal playback speed of the video; and

responsive to the request, (i) selecting from a plurality of versions of a trick-play track of the video a version that has the highest I-frame rate that, when played out at the requested fast-play speed, will have a bit-rate that does not exceed a network bandwidth threshold, wherein each version has a normal speed bit-rate, and wherein selecting from the plurality of versions of the trick-play track of the video a version that has the highest I-frame rate that, when played out at the requested fast-play speed, will have a bit-rate that does not exceed the network bandwidth threshold comprises a function selected from the group consisting of (a) determining which of the versions would have a highest increased bit-rate less than or equal to the network bandwidth threshold when played out at the requested fast-play speed and (b) determining which of the versions has a highest normal speed bit-rate less than or equal to the network bandwidth threshold divided by the requested fast play speed, and (ii) causing the selected version of the trick-play track to be streamed over a network from a server to a client for playout.

2. The non-transitory computer readable medium of claim 1, wherein the functions further comprise determining the network bandwidth threshold.

3. The non-transitory computer readable medium of claim 1, wherein determining the network bandwidth threshold comprises determining the network bandwidth threshold based on a rate of arrival of data at the client from the server.

4. The non-transitory computer readable medium of claim 1, wherein detecting the request comprises receiving a signal representing the request having been input at a user interface of the client.

5. The non-transitory computer readable medium of claim 4, wherein the processing unit is part of the client.

6. The non-transitory computer readable medium of claim 4,

wherein the processing unit is part of the server,

wherein receiving the signal representing the request comprises receiving the signal from the client, and

wherein causing the selected version of the trick-play track to be streamed over the network from the server to the client for playout comprises causing a network communication interface of the server to stream the selected version of the trick-play track in real-time to the client.

7. The non-transitory computer readable medium of claim 6, wherein the signal transmitted from the client to the server comprises a signal specifying the requested fast-play speed and the network bandwidth threshold.

8. The non-transitory computer readable medium of claim 7, wherein the functions further comprise:

generating the selected version on the fly for streaming to the client.

9. The non-transitory computer readable medium of claim 8, wherein generating the selected version on the fly comprises stripping a plurality of I-frames from a base version of the trick-play track to produce the selected version of the trick-play track.

10. The non-transitory computer readable medium of claim 1, wherein selecting the version from the plurality of versions comprises referring to an index of the video that specifies the plurality of versions and selecting the version from the specified plurality.

11. The non-transitory computer readable medium of claim 10,

wherein the index specifies respectively for each version of the plurality the normal-speed bit-rate of the version, and

wherein selecting from the plurality of versions a version that has the highest I-frame rate that when played out at the requested fast-play speed will have a bit-rate that does not exceed the network bandwidth threshold comprises determining, if the normal-speed bit-rate of each version of the plurality were to be multiplied by the fast-play speed to establish the increased bit-rate of the version, which version of the plurality would have the highest increased bit-rate that does not exceed the network bandwidth threshold.

12. The non-transitory computer readable medium of claim 1,

wherein selecting from the plurality of versions a version that has the highest I-frame rate that when played out at the requested fast-play speed will have a bit-rate that does not exceed the network bandwidth threshold comprises determining, if the normal-speed bit-rate of each version of the plurality were to be multiplied by the fast-play speed to establish an increased bit-rate, which version of the plurality would have the highest increased bit-rate that does not exceed the network bandwidth threshold.

13. The non-transitory computer readable medium of claim 1,

wherein the normal speed bit-rate of each version of the plurality of versions is a trick-play bit-rate of the version, and

wherein selecting from the plurality of versions a version that has the highest I-frame rate that when played out at the requested fast-play speed will have a bit-rate that does not exceed the network bandwidth threshold comprises (i) dividing the network bandwidth threshold by the requested fast-play speed to determine a maximum trick-play bit-rate, and (ii) selecting from the plurality of versions the version that has the highest trick-play bit-rate that does not exceed the determined maximum trick-play bit-rate.

14. The non-transitory computer readable medium of claim 1, wherein causing the selected version of the trick-play track to be streamed from the server to the client for playout comprises transmitting from the client to the server a control signal that directs the server to stream the selected version of the trick-play track to the client for playout.

15. The non-transitory computer readable medium of claim 1, wherein the functions further comprise:

receiving the selected version of the trick-play track streamed from the server; and

while receiving the selected version of the trick-play track streamed from the server, playing out the selected version of the trick-play track at the requested fast-play speed.

16. A method comprising:

receiving a request to play video at a fast-play speed that defines a multiple of a normal playback speed of the video;

identifying a plurality of different frame-rate versions of a trick play track corresponding with the video, where each frame-rate version has a respective bit-rate that is different than the bit-rate of each other frame-rate version due at least in part to each frame-rate version skipping a different number of I-frames of the video than each other frame-rate version;

selecting from the identified plurality of frame-rate versions a best frame-rate version by (i) determining, if the bit-rate of each frame-rate version were multiplied by the requested fast-play speed to produce an increased bit-rate per frame-rate version, which frame-rate version would have the highest increased bit-rate that is less than a network bandwidth threshold and (ii) selecting the determined frame-rate version as the best frame-rate version; and

causing a server to stream the selected frame-rate version over a network to a client.

17. The method of claim 16, further comprising:

determining the network bandwidth threshold based on a rate of arrival of data at the client from the server.

18. The method of claim 16, further comprising:

obtaining an index of the video, the index specifying the plurality of different frame-rate versions of the trick-play track,

wherein identifying the plurality of different frame-rate versions comprises referring to the index that specifies the plurality of different frame-rate versions of the trick-play track.

19. The method of claim 16, carried out by the client.

20. The method of claim 16,

wherein selecting the best frame-rate version comprises (a) dividing the network bandwidth threshold by the requested fast-play speed to determine a maximum trick-play bit rate, and (b) selecting as the best frame-rate version the frame-rate version that has the highest bit-rate not exceeding the determined maximum trick-play bit rate.

21. The method of claim 16, wherein causing the server to stream the selected frame-rate version over the network to the client comprises transmitting from the client to the server a control signal that directs the server to stream the selected frame-rate version to the client.

22. The method of claim 16, further comprising:

receiving the selected frame-rate version of the trick-play track being streamed from the server; and

while receiving the selected frame-rate version of the trick-play track streamed from the server, playing out the selected frame-rate version of the trick-play track at the requested fast-play speed.

23. A computing device comprising:

a network communication interface;

a user interface;

a processor;

data storage; and

program logic stored in the data storage and executable by the processor to carry out functions comprising;

receiving from the user interface a request to play video at a fast-play speed that defines a multiple of a normal playback speed of the video, and

responsive to the request, (a) selecting a version of a trick-play track of the video based at least in part on the requested fast-play speed and a network bandwidth threshold, wherein selecting the version of the trick-play track based at least in part on the requested fast-play speed and the network bandwidth threshold comprises selecting as the version of the trick-play track a version of the trick-play track that has a rate of skipping I-frames of the video sufficient that the version of the trick-play track, when played at the requested fast-play speed, would have bit rate that does not exceed the network bandwidth threshold, and (b) causing a server to stream the selected version of the trick-play track over a network to the computing device for playout at the user interface,

wherein the data storage holds an index of the video, the index specifying a plurality of versions of the trick-play track, wherein the plurality of versions of the trick-play track differ from each other at least in terms of their respective rate of skipping I-frames of the video and thus their respective bit rate, and

wherein selecting the version of the trick-play track comprises (i) referring to the index to identify one or more of the specified plurality of versions of the trick-play track that, when played at the requested fast-play speed, would have an increased bit rate that would not exceed the network bandwidth threshold, and (ii) selecting a highest bit-rate one of the identified one or more identified versions of the trick-play track.

24. The computing device of claim 23, wherein the functions further comprise determining the network bandwidth threshold.

25. The computing device of claim 23, wherein determining the network bandwidth threshold comprises determining the network bandwidth threshold based on a rate of arrival of data at the computing device from the server.

26. The computing device of claim 23, wherein causing the server to stream the selected version of the trick-play track to the computing device comprises transmitting to the server via the network communication interface a control signal that directs the server to stream to the computing device the selected version of the trick-play track.

27. The computing device of claim 23,

wherein the data storage holds an index of the video, the index specifying a plurality of versions of the trick-play track, wherein the plurality of versions of the trick-play track differ from each other at least in terms of their respective rate of skipping I-frames of the video and thus their respective bit rate, and

wherein selecting the version of the trick-play track comprises (a) dividing the network bandwidth threshold by the requested fast-play speed to determine a maximum trick-play bit rate, and (b) selecting from the plurality of versions of the trick-play track the version that has the highest bit-rate not exceeding the determined maximum trick-play bit rate.

28. A method for controlling fast-play playout of video at a requested fast-play speed that defines a multiple of a normal playback speed of the video, the method comprising:

determining a lowest rate at which a trick-play track of the video should skip I-frames of the video in order for the trick-play track, played at the requested fast-play speed, to have a bit rate less than or equal to a measure of network bandwidth;

given an option for a server to stream to a client any of a plurality of different frame-rate versions of a trick-play track of the video, wherein each frame-rate version has a bit-rate that is different than the bit-rate of each other frame-rate version due at least in part to each frame-rate version skipping I-frames at a different rate than each other frame-rate version, selecting the frame-rate version that skips I-frames at the determined lowest rate; and

causing the server to stream to the client the selected frame-rate version of the trick-play track.

29. The method of claim 28, wherein the selected frame-rate version of the trick-play track does not include any P-frames or B-frames of the video.