Client side control of adaptive streaming

• US 9,819,715 B2
• Filed: 06/21/2012
• Issued: 11/14/2017
• Est. Priority Date: 06/21/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Scurtu, Tudor, Vergoti, Silviu, Cucu, Andrian, Alexandru, Mihai
• Primary Examiner(s): Tokuta, Shean
• Assistant Examiner(s): DUONG, THAO DUC
• Application Number:
  US13/529,050
• Publication Number:
  US 20130346590A1
• Time in Patent Office: 1,972 Days
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

One embodiment includes implementing adaptive streaming by a switch manager application executed at a client device. The switch manager application can determine a respective weight for a plurality of switching rules. Each switching rule can include one or more criteria for generating a recommended quality level. Each weight can include data describing a reliability of the rule. The switch manager application can select, based at least partially on the respective weight of the at least one switching rule, at least one switching rule of the plurality of switching rules to generate a recommendation. Another embodiment includes the switch manager application generating a recommendation including a quality level and a confidence level from each of the plurality of switching rules. The switch manager application can determine a selected quality level based on the weights of the switching rules and the quality levels and confidence levels of the recommendations.

Claims

1. A method comprising:

determining, by a switch manager application executed by a processing device of a client that retrieves streaming content from a server, a first weight for a first switching rule from a plurality of switching rules and a second weight for a second switching rule from the plurality of switching rules, wherein applying each switching rule generates a recommended quality level for electronic content, each weight describing a reliability of the respective switching rule, wherein the second weight indicates that the second switching rule is unreliable and wherein determining the second weight comprises determining that a history of recommended quality levels generated using the second switching rule includes a number of emergency down-switches exceeding a pre-determined threshold, wherein each emergency down-switch comprises selecting, responsive to an event indicating unsatisfactory playback of the electronic content, a rendition having a lower quality level than a current rendition;

determining, by the switch manager application, (i) a first confidence level for a first recommendation generated by the first switching rule and (ii) a second confidence level for a second recommendation generated by the second switching rule, wherein the first confidence level describes a first quantity of first quality-of-service data, wherein the first quality-of-service data is used by the first switching rule to generate recommendations, wherein the second confidence level describes a second quantity of second quality-of-service data, wherein the second quality-of-service data is used by the second switching rule to generate recommendations, wherein the first confidence level differs from the second confidence level based on the first quantity being different from the second quantity;

selecting, by the switch manager application, the first switching rule rather than the second switching rule to generate a recommendation, wherein the first switching rule is selected based at least partially on the first weight of the first switching rule and on the first confidence level of the first switching rule;

applying, by the processing device, the first switching rule and thereby generating a particular recommended quality level for the electronic content; and

downloading and playing, by the processing device, a particular rendition of the electronic content corresponding to the particular recommended quality level for the electronic content.

2. The method of claim 1, wherein the first confidence level indicates that the first quality-of-service data is sufficient for the first switching rule to be reliable and the second confidence level indicates that the second quality-of-service data is insufficient for the second switching rule to be reliable.

3. The method of claim 2, further comprising:

determining respective weights for the plurality of switching rules, each weight indicating a respective reliability of the respective switching rule;

for each of the plurality of switching rules, generating, by the switch manager application, a respective recommendation comprising a respective recommended quality level and a respective confidence level; and

determining, by the switch manager application, the particular recommended quality level based on the weights of the plurality of switching rules and the recommended quality levels and respective confidence levels of the recommendations.

4. The method of claim 3, further comprising, wherein the particular rendition has a quality level less than or equal to the particular recommended quality level, wherein a plurality of renditions comprising the particular rendition comprises a plurality of copies of an asset, each copy of the asset being encoded at a different quality level.

5. The method of claim 4, wherein a respective quality level for a respective one of the plurality of renditions corresponds to the respective rendition being encoded using a respective bit rate.

6. The method of claim 1, further comprising,

accessing, by the switch manager application, a subset of the plurality of switching rules, the subset comprising emergency switching rules, wherein each emergency switching rule is limited to recommending an emergency quality level that is less than or equal to the current quality level;

and

generating, by the switch manager application and for the emergency switching rules, respective emergency recommendations comprising respective emergency quality levels and respective emergency confidence levels;

determining, by the switch manager application, whether an emergency confidence level exceeds a threshold confidence level; and

performing one of;

determining respective weights of plurality of switching rules and selecting the first switching rule, wherein the respective weights are determined and the first switching rule is selected based on none of the emergency confidence levels exceeding the threshold confidence level, wherein the first switching rule allows the switch manager application to recommend a different quality level that is greater than a current quality level;

or

determining a selected quality level corresponding to one of the emergency recommendations associated with a lowest bit rate, wherein the selected quality level is determined based on the one of the emergency recommendations having a corresponding emergency confidence level exceeding the threshold confidence level.

7. The method of claim 1, wherein the respective weight for each of the plurality of switching rules is determined based on one or more of an actual bit rate metric, a bandwidth metric, a buffer fragments metric, and a dropped-frames-per-second metric.

8. The method of claim 1, wherein the plurality of switching rules comprises one or more of:

a bandwidth rule describing a bandwidth of a network connection between a client device including the processing device and a content provider providing the electronic content;

a bandwidth buffer rule describing the bandwidth of the network connection and a minimum amount of electronic content stored in a buffer that is accessible by the client device;

a hardware capabilities rule describing one or more of a battery life, memory capacity, or processing power of the client device;

a content utilization rule describing how the electronic content is being rendered for display or use at the client device;

or

a buffering rule describing whether one or more renditions have historically caused depletion of the buffer.

9. A non-transitory computer-readable medium embodying program code executable by a processing device of a client that retrieves streaming content from a server, the program code comprising:

program code for determining a first weight for a first switching rule from a plurality of switching rules and a second weight for a second switching rule from the plurality of switching rules, wherein applying each switching rule generates a recommended quality level for electronic content, each weight describing a reliability of the respective switching rule, wherein the second weight indicates that the second switching rule is unreliable and wherein determining the second weight comprises determining that a history of recommended quality levels generated using the second switching rule includes a number of emergency down-switches exceeding a pre-determined threshold, wherein each emergency down-switch comprises selecting, responsive to an event indicating unsatisfactory playback of the electronic content, a rendition having a lower quality level than a current rendition;

program code for determining (i) a first confidence level for a first recommendation generated by the first switching rule and (ii) a second confidence level for a second recommendation generated by the second switching rule, wherein the first confidence level describes a first quantity of first quality-of-service data, wherein the first quality-of-service data is used by the first switching rule to generate recommendations, wherein the second confidence level describes a second quantity of second quality-of-service data, wherein the second quality-of-service data is used by the second switching rule to generate recommendations, wherein the first confidence level differs from the second confidence level based on the first quantity being different from the second quantity;

program code for selecting the first switching rule rather than the second switching rule to generate a recommendation, wherein the first switching rule is selected based at least partially on the first weight of the first switching rule and on the first confidence level of the first switching rule;

program code for applying the first switching rule and thereby generating a particular recommended quality level for the electronic content; and

program code for downloading and playing a particular rendition of the electronic content corresponding to the particular recommended quality level for the electronic content.

10. The non-transitory computer-readable medium of claim 9, further comprising:

program code for generating a selected quality level having a bit rate less than or equal to a recommended bit rate corresponding to the particular recommended quality level.

11. The non-transitory computer-readable medium of claim 10, wherein the first switching rule comprises a dropped-frames-per-second rule and wherein the program code for generating the recommendation comprises program code for determining that a rendition of electronic being rendered is associated with a number of dropped frames exceeding a threshold number of dropped frames.

12. The non-transitory computer-readable medium of claim 10, wherein the first switching rule comprises a buffering rule and wherein the program code for generating the recommendation comprises program code for determining whether one or more renditions of the electronic content rendered for display at a client device have historically caused depletion of a buffer that is accessible by the client device.

13. The non-transitory computer-readable medium of claim 10, wherein the first switching rule comprises a bandwidth buffer rule and wherein the program code for generating the recommendation comprises program code for determining a bandwidth of a network connection between a client device and the server providing the electronic content and a minimum amount of electronic content stored in a buffer that is accessible by the client device.

14. A client computing device comprising:

a non-transitory computer-readable medium storing code providing a switch manager application; and

a processor communicatively coupled to the non-transitory computer-readable medium and configured to execute the code to perform operations comprising;

determining a first weight for a first switching rule from a plurality of switching rules and a second weight for a second switching rule from the plurality of switching rules, wherein applying each switching rule generates a recommended quality level for electronic content, each weight describing a reliability of the respective switching rule, wherein the second weight indicates that the second switching rule is unreliable and wherein determining the second weight comprises determining that a history of recommended quality levels generated using the second switching rule includes a number of emergency down-switches exceeding a pre-determined threshold, wherein each emergency down-switch comprises selecting, responsive to an event indicating unsatisfactory playback of the electronic content, a rendition having a lower quality level than a current rendition,

determining (i) a first confidence level for a first recommendation generated by the first switching rule and (ii) a second confidence level for a second recommendation generated by the second switching rule, wherein the first confidence level describes a first quantity of first quality-of-service data, wherein the first quality-of-service data is used by the first switching rule to generate recommendations, wherein the second confidence level describes a second quantity of second quality-of-service data, wherein the second quality-of-service data is used by the second switching rule to generate recommendations, wherein the first confidence level differs from the second confidence level based on the first quantity being different from the second quantity,

selecting the first switching rule rather than the second switching rule to generate a recommendation, wherein the first switching rule is selected based at least partially on the first weight of the first switching rule and on the first confidence level of the first switching rule,

applying the first switching rule and thereby generating a particular recommended quality level for the electronic content, and

downloading and playing a particular rendition of the electronic content corresponding to the particular recommended quality level for the electronic content.

15. The client computing device of claim 14,

wherein the first confidence level indicates that the first quality-of-service data is sufficient for the first switching rule to be reliable and the second confidence level indicates that the second quality-of-service data is insufficient for the second switching rule to be reliable.

16. The client computing device of claim 15, wherein the non-transitory computer-readable medium stores additional code providing a viewer application, wherein the processor is configured to execute the additional code to perform additional operations comprising:

rendering for display a rendition of electronic content having a respective quality level;

analyzing data associated with rendering the rendition to generate the first quality-of-service data and the second quality-of-service data.

17. The client computing device of claim 14, further comprising a display device configurable by the processor to display the electronic content and wherein the first switching rule comprises a content utilization rule describing whether the electronic content is being rendered for full-screen display at the display device.

18. The client computing device of claim 14, further comprising a display device configurable by the processor to display the electronic content and wherein the first switching rule comprises a content utilization rule describing one or more characteristics of the display device.