MULTI-TENANT THROTTLING APPROACHES

• US 20160344651A1
• Filed: 08/08/2016
• Published: 11/24/2016
• Est. Priority Date: 09/26/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Kusters, Norbert P., Lee, Kerry Q., Pisolkar, Raghav Vijay, Certain, Tate Andrew
• Granted Patent: US 10,700,994 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04L 12/1403 Architecture for metering, charging or billing
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 47/762 triggered by the network
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 67/146 Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
• CPC Class Codes: H04L 69/321 Interlayer communication protocols or service data unit [SDU] definitions; Interfaces between layers
• CPC Class Codes: H04L 69/325 in the network layer [OSI layer 3], e.g. X.25 H04L69/16 takes precedence
• CPC Class Codes: H04M 15/765 Linked or grouped accounts, e.g. of users or devices
• CPC Class Codes: H04M 15/7652 shared by users
• CPC Class Codes: H04M 15/7655 shared by technologies
• CPC Class Codes: H04M 15/83 Notification aspects
• CPC Class Codes: H04M 15/85 characterised by the type of condition triggering a notification
• CPC Class Codes: H04M 15/853 Calculate maximum communication time or volume
• CPC Class Codes: H04M 15/854 Available credit

Abstract

An opportunistic throttling approach can be used for customers of shared resources in a multi-tenant environment. Each customer can have a respective token bucket with a guaranteed fill rate. When a request is received for an amount of work to be performed by a resource, the corresponding number of tokens are obtained from, or charged against, a global token bucket. If the global bucket has enough tokens, and if the customer has not exceeded a maximum work rate or other such metric, the customer can charge less than the full number of tokens against the customer'"'"'s token bucket, in order to reduce the number of tokens that need to be taken from the customer bucket. Such an approach can enable the customer to do more work and enable the customer'"'"'s bucket to fill more quickly as fewer tokens are charged against the customer bucket for the same amount of work.

Claims

1. (canceled)

2. A computer-implemented method, comprising:

receiving a request for usage of at least one resource by a computing system in a multi-tenant computing environment that includes a plurality of computing resources in communication via a network;

when the requested usage exceeds a threshold usage of the at least one resource, delaying processing of the request;

when the requested usage is below the threshold, determining an amount of usage needed to process the request;

charging the amount of usage against a global available usage amount associated with the at least one resource;

determining a portion of the amount of usage to be charged against a requestor available usage amount for the request based at least in part on a fill level of the global available usage amount; and

processing the request when at least the portion of the amount of usage is available in the requestor available usage amount.

3. The computer-implemented method of claim 2, wherein when the fill level of the global available usage amount is greater than 50%, charging up to the amount of usage against the global available usage amount.

4. The computer-implemented method of claim 3, wherein when the fill level of the global available usage amount is less than 50%, charging up to the amount of usage to the requestor available usage amount.

5. The computer-implemented method of claim 4, further comprising:

determining the requestor available usage amount is less than or equal to the amount of usage; and

throttling subsequent requests until the requestor available usage amount is refilled.

6. The computer-implemented method of claim 2, wherein the requested usage is provided at up to a maximum rate for the requestor when the global available usage amount at least meets a global threshold.

7. The computer-implemented method of claim 2, wherein the requested usage is provided at up to an intermediate rate for the requestor when the global available usage amount is less than the global threshold but above a global minimum threshold.

8. The computer-implemented method of claim 7, wherein the intermediate rate is a function of the global fill rate and a current global available amount of usage based on the global fill threshold and the global minimum threshold.

9. An apparatus comprising:

at least one processor; and

at least one memory device including instructions that, when executed by the at least one processor, cause the computing system to;

receive a request for usage of at least one resource by a computing system in a multi-tenant computing environment that includes a plurality of computing resources in communication via a network;

when the requested usage exceeds a threshold usage of the at least one resource, delay processing of the request;

when the requested usage is below the threshold, determine an amount of usage needed to process the request;

charge the amount of usage against a global available usage amount associated with the at least one resource;

determine a portion of the amount of usage to be charged against a requestor available usage amount for the request based at least in part on a fill level of the global available usage amount; and

process the request when at least the portion of the amount of usage is available in the requestor available usage amount.

10. The apparatus of claim 9, wherein when the fill level of the global available usage amount is greater than 50%, charging up to the amount of usage against the global available usage amount.

11. The apparatus of claim 10, wherein when the fill level of the global available usage amount is less than 50%, charging up to the amount of usage to the requestor available usage amount.

12. The apparatus of claim 11, wherein the at least one memory device includes instructions that, when executed by the at least one processor, further cause the computing system to:

determine the requestor available usage amount is less than or equal to the amount of usage; and

throttle subsequent requests until the requestor available usage amount is refilled.

13. The apparatus of claim 9, wherein the requested usage is provided at up to a maximum rate for the requestor when the global available usage amount at least meets a global threshold.

14. The apparatus of claim 9, wherein the requested usage is provided at up to an intermediate rate for the requestor when the global available usage amount is less than the global threshold but above a global minimum threshold.

15. The apparatus of claim 14, wherein the intermediate rate is a function of the global fill rate and a current global available amount of usage based on the global fill threshold and the global minimum threshold.

16. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing system, cause the computing system to:

receive a request for usage of at least one resource by a computing system in a multi-tenant computing environment that includes a plurality of computing resources in communication via a network;

when the requested usage exceeds a threshold usage of the at least one resource, delay processing of the request;

when the requested usage is below the threshold, determine an amount of usage needed to process the request;

charge the amount of usage against a global available usage amount associated with the at least one resource;

determine a portion of the amount of usage to be charged against a requestor available usage amount for the request based at least in part on a fill level of the global available usage amount; and

process the request when at least the portion of the amount of usage is available in the requestor available usage amount.

17. The non-transitory computer-readable storage medium of claim 16, wherein when the fill level of the global available usage amount is greater than 50%, charging up to the amount of usage against the global available usage amount.

18. The non-transitory computer-readable storage medium of claim 17, wherein when the fill level of the global available usage amount is less than 50%, charging up to the amount of usage to the requestor available usage amount.

19. The non-transitory computer-readable storage medium of claim 18, storing instructions that, when executed by the at least one processor, further cause the computing system to:

determine the requestor available usage amount is less than or equal to the amount of usage; and

throttle subsequent requests until the requestor available usage amount is refilled.

20. The non-transitory computer-readable storage medium of claim 16, wherein the requested usage is provided at up to a maximum rate for the requestor when the global available usage amount at least meets a global threshold.

21. The non-transitory computer-readable storage medium of claim 16, wherein the requested usage is provided at up to an intermediate rate for the requestor when the global available usage amount is less than the global threshold but above a global minimum threshold, and wherein the intermediate rate is a function of the global fill rate and a current global available amount of usage based on the global fill threshold and the global minimum threshold.