System and method for adaptive segment prefetching of streaming media

• US 9,392,228 B2
• Filed: 08/24/2011
• Issued: 07/12/2016
• Est. Priority Date: 05/23/2008
• Status: Active Grant

• Current Assignee: Stripe, Inc.
• Original Assignee: Porto Technology, LLC (Concert Technology Corporation)
• Inventors: Kandekar, Kunal, Walsh, Richard J., Issa, Alfredo C., Katpelly, Ravi Reddy
• Primary Examiner(s): SHAW, PELING ANDREW
• Application Number:
  US13/216,517
• Publication Number:
  US 20120072611A1
• Time in Patent Office: 1,784 Days
• Field of Search: 709231-235
• US Class Current: 1/1
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Systems and methods are provided for obtaining look-ahead segments for a streaming media item during playback such that the viewer can easily skip to the look-ahead segments without experiencing any substantial interruption or delay in playback. In general, a streaming media source begins streaming a media item to a streaming media client for playback. In one embodiment, the streaming media client selects one or more segments of the media item as one or more look-ahead segments to prefetch from the streaming media source. Then, while the media item is being streamed to the streaming media client for playback, the streaming media client prefetches the one or more look-ahead segments of the media item from the streaming media source. The look-ahead segments are thereafter utilized to enable the viewer to skip ahead in playback without experiencing a substantial interruption or delay in playback.

Claims

1. A method of operation of a user device comprising:

receiving a stream of a media item from a streaming media source;

providing playback of the stream of the media item;

obtaining one or more look-ahead segments of the media item, wherein the one or more look-ahead segments are one or more segments of the media item that have not yet been streamed to the user device selected based on a comparison of information that describes content of the one or more look-ahead segments and user preferences of a user of the user device; and

storing the one or more look-ahead segments in local storage of the user device.

2. The method of claim 1 wherein at least one look-ahead segment of the one or more look-ahead segments is a segment of the media item beyond that which is being streamed to the user device at a time the at least one look-ahead segment is obtained.

3. The method of claim 1 wherein at least one look-ahead segment of the one or more look-ahead segments is a segment of the media item prior to a current point in playback of the media item that has not yet been streamed to the user device at a time at which the at least one look-ahead segment is obtained.

4. The method of claim 1 further comprising:

receiving user input from a user of the user device instructing the user device to skip to a look-ahead segment of the one or more look-ahead segments during playback of the stream of the media item;

providing playback of the look-ahead segment from the local storage;

requesting the stream of the media item starting at a point corresponding to an end of the look-ahead segment stored in the local storage;

receiving the stream of the media item starting at the point corresponding to the end of the look-ahead segment stored in the local storage from the streaming media source while providing playback of the look-ahead segment from the local storage such that the stream of the media item starting at the point corresponding to the end of the look-ahead segment stored in the local storage is buffered while providing playback of the look-ahead segment from the local storage; and

transitioning playback from the look-ahead segment to the stream of the media item starting at the point corresponding to the end of the look-ahead segment stored in the local storage once playback of the look-ahead segment from the local storage is complete.

5. The method of claim 1 wherein obtaining the one or more look-ahead segments of the media item comprises:

selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments; and

prefetching the one or more look-ahead segments from the streaming media source.

6. The method of claim 5 wherein obtaining the one or more look-ahead segments of the media item further comprises adaptively updating the one or more segments selected as the one or more look-ahead segments.

7. The method of claim 5 wherein obtaining the one or more look-ahead segments of the media item further comprises adaptively updating the one or more segments selected as the one or more look-ahead segments during playback of the stream of the media item based on at least one user action taken by a user of the user device during playback of the stream of the media item.

8. The method of claim 7 wherein the at least one user action is one of a group consisting of:

skipping over a segment of the stream of the media item during playback, skipping to a segment of the stream of the media item during playback, repeating playback of a segment of the stream of the media item during playback, and fast-forwarding through a segment of the stream of the media item during playback.

9. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments during playback of the stream of the media item based on the information that describes the content of the one or more look-ahead segments and at least one user action taken by a user of the user device during playback of the stream of the media item.

10. The method of claim 5 wherein selecting the one or more look-ahead segments during playback of the stream of the media item comprises selecting the one or more look-ahead segments from a plurality of segments of the media item that have not yet been streamed to the user device based on information describing content of the plurality of segments of the media item and information describing at least one of a group consisting of:

a segment of the stream of the media item skipped over during playback, a segment of the stream of the media item skipped to during playback, a segment of the stream of the media item for which playback was repeated during playback, and a segment of the stream of the media item that was fast-forwarded through during playback.

11. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and at least one playback history of a user of the user device for at least one previous playback of the media item.

12. The method of claim 11 wherein the at least one playback history is indicative of at least one segment of the media item that is of interest to the user of the user device.

13. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and a playback history for a previous playback of the media item for at least one user other than a user of the user device.

14. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and a playback history for a previous playback of the media item for at least one other user within a social network of a user of the user device.

15. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and a playback history for a user of the user device for a previous playback of a media item other than the media item.

16. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and at least one of a group consisting of:

bandwidth availability for prefetching the one or more look-ahead segments and bandwidth requirements for prefetching segments of the media item.

17. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and heuristics.

18. The method of claim 5 wherein selecting the one or more look-ahead segments comprises selecting the one or more look-ahead segments based on the information that describes the content of the one or more look-ahead segments and a profile of a user of the user device obtained from a social networking service.

19. The method of claim 5 further comprising prioritizing the one or more look-ahead segments to assign a priority to each of the one or more look-ahead segments.

20. The method of claim 19 wherein prefetching the one or more look-ahead segments comprises, for each look-ahead segment:

allocating an amount of an available bandwidth to the look-ahead segment based on the priority assigned to the look-ahead segment; and

prefetching the look-ahead segment from the streaming media source using the amount of the available bandwidth allocated to the look-ahead segment.

21. The method of claim 5 wherein prefetching the one or more look-ahead segments comprises, for each look-ahead segment from the one or more look-ahead segments, prefetching a complete version of the look-ahead segment from the streaming media source.

22. The method of claim 5 wherein prefetching the one or more look-ahead segments comprises, for each look-ahead segment from the one or more look-ahead segments, prefetching a portion of the look-ahead segment from the streaming media source.

23. The method of claim 5 wherein prefetching the one or more look-ahead segments comprises, for each look-ahead segment from the one or more look-ahead segments, prefetching a version of the look-ahead segment from the streaming media source having a quality essentially equal to that of the stream of the media item received by the user device.

24. The method of claim 5 wherein prefetching the one or more look-ahead segments comprises, for each look-ahead segment from the one or more look-ahead segments, prefetching a low-quality version of the look-ahead segment from the streaming media source.

25. The method of claim 5 wherein prefetching the one or more look-ahead segments comprises, for each look-ahead segment from the one or more look-ahead segments:

selecting one of a plurality of versions of the look-ahead segment based on a priority assigned to the look-ahead segment, each of the plurality of versions of the look-ahead segment being a different quality version of the look-ahead segment; and

prefetching the one of the plurality of versions of the look-ahead segment from the streaming media source.

26. The method of claim 25 wherein selecting the one of the plurality of versions of the look-ahead segment comprises selecting the one of the plurality of versions of the look-ahead segment based on the priority assigned to the look-ahead segment and a bandwidth requirement for delivering each of the plurality of versions of the look-ahead segment.

27. The method of claim 1 wherein obtaining the one or more look-ahead segments of the media item comprises obtaining the one or more look-ahead segments from a proxy.

28. The method of claim 27 wherein the proxy selects the one or more look-ahead segments from a plurality of segments of the media item that have not yet been streamed to the user device based on information that describes content of the plurality of segments of the media item, prefetches the one or more look-ahead segments from the streaming media source, and provides the one or more look-ahead segments to the user device.

29. The method of claim 1 wherein obtaining the one or more look-ahead segments of the media item comprises receiving the one or more look-ahead segments from the streaming media source in response to the one or more look-ahead segments being pushed by the streaming media source to the user device.

30. The method of claim 29 wherein the one or more look-ahead segments are selected from a plurality of segments of the media item by the streaming media source.

31. The method of claim 29 wherein the one or more look-ahead segments are selected from a plurality of segments of the media item by a proxy.

32. The method of claim 1 wherein obtaining the one or more look-ahead segments comprises:

receiving information identifying the one or more look-ahead segments; and

prefetching the one or more look-ahead segments from the streaming media source.

33. The method of claim 32 wherein receiving the information identifying the one or more look-ahead segments comprises receiving the information identifying the one or more look-ahead segments from the streaming media source.

34. The method of claim 32 wherein receiving the information identifying the one or more look-ahead segments comprises receiving the information identifying the one or more look-ahead segments from a proxy.

35. The method of claim 1 wherein the information that describes the content of the one or more look-ahead segments is not indicative of a location of the one or more look-ahead segments in the media item.

36. A user device comprising:

a communication interface communicatively coupling the user device to a streaming media source via a network; and

a control system associated with the communication interface and adapted to;

receive a stream of a media item from the streaming media source;

provide playback of the stream of the media item;

obtain one or more look-ahead segments of the media item, wherein the one or more look-ahead segments are one or more segments of the media item that have not yet been streamed to the user device selected based on a comparison of information that describes content of the one or more look-ahead segments and user preferences of a user of the user device; and

store the one or more look-ahead segments in local storage of the user device.

37. A method of operation of a streaming media source comprising:

receiving a request for a media item from a user device via a network;

streaming the media item to the user device via the network;

selecting one or more segments of the media item as one or more look-ahead segments to push to the user device, wherein the one or more look-ahead segments are one or more segments of the media item that have not yet been streamed to the user device selected based on a comparison of information that describes content of the one or more look-ahead segments and user preferences of a user of the user device; and

pushing the one or more look-ahead segments to the user device via the network.

38. The method of claim 37 wherein selecting the one or more segments of the media item as the one or more look-ahead segments comprises adaptively updating the one or more segments selected as the one or more look-ahead segments during playback of a stream of the media item based on user actions taken by a user of the user device during playback of the stream of the media item.

39. A streaming media source comprising:

a communication interface communicatively coupling the streaming media source to a user device via a network; and

a control system associated with the communication interface and adapted to;

receive a request for a media item from the user device;

stream the media item to the user device;

select one or more segments of the media item as one or more look-ahead segments to push to the user device, wherein the one or more look-ahead segments are one or more segments of the media item that have not yet been streamed to the user device selected based on a comparison of information that describes content of the one or more look-ahead segments and user preferences of a user of the user device; and

push the one or more look-ahead segments to the user device.

40. The streaming media source of claim 39 wherein the control system is further adapted to adaptively update the one or more segments selected as the one or more look-ahead segments during playback of a stream of the media item based on user actions taken by a user of the user device during playback of the stream of the media item.