Weighted service requests throttling

• US 9,501,321 B1
• Filed: 01/24/2014
• Issued: 11/22/2016
• Est. Priority Date: 01/24/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sharifi Mehr, Nima
• Primary Examiner(s): Patel, Hiren
• Application Number:
  US14/163,023
• Time in Patent Office: 1,033 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/483 Multiproc
• CPC Class Codes: G06F 9/4812 by interrupt, e.g. masked
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals

Abstract

Computing platforms may enable clients to use services to execute data processing tasks. Each of these services consumes resources during execution. Described herein are systems, devices and methods for generating resource consumption data for service calls. The resource consumption data may indicate amounts of resources consumed to process computational tasks associated with service calls. The system may determine when to process computational tasks based on the resource consumption data.

Claims

1. A system comprising:

a service fleet comprising a plurality of host computing devices, the plurality of host computing devices including;

a memory storing computer-executable instructions; and

a processor, coupled to the memory, to execute the computer-executable instructions to provide a load balancer, a sampler host, and a service host, wherein;

the load balancer is configured to;

receive service calls including a first service call comprising a first request for a first performance of a first computational task, a second service call comprising a second request for a second performance of a second computational task, and a third service call comprising a third request for a third performance of a third computational task;

provide the first service call and the second service call to the sampler host; and

provide the third service call to the service host;

the sampler host is configured to;

process a single service call at a time to individually determine resource consumed by each of the service calls;

for the first service call, generate first resource consumption data indicating a first amount of resources consumed to perform the first computational task;

for the second service call, generate second resource consumption data indicating a second amount of resources consumed to perform the second computational task;

access configurable formula data indicative of a first weight factor associated with the first resource consumption data and indicative of a second weight factor associated with the second resource consumption data;

generate a first weighted value by multiplying the first weight factor by the first resource consumption data;

generate a second weighted value by multiplying the second weight factor by the second resource consumption data;

generate weight data comprising a value generated by combining the first weighted value and the second weighted value; and

provide the weight data to the service host in response to a determination that a threshold amount of resource consumption data has been generated;

the service host is configured to process the third service call based on the weight data.

2. The system of claim 1, wherein:

the first resource consumption data and the second resource consumption data indicative of the amount of resources consumed to perform the first computational task and the second computational task includes at least one of;

information indicative of an average amount of resources consumed to perform the first computational task and the second computational task;

information indicative of a maximum amount of resources consumed to perform the first computational task and the second computational task; and

information indicative of a cost to perform the first computational task and the second computational task.

3. The system of claim 1, wherein:

the first service call, the second service call, and the third service call are a first type of service call, the first service type of service call comprising service calls sharing characteristics different from service calls being of a second type of service call;

the information indicative of the amount of resources includes a first average amount of resources consumed to perform the first computational task and the second computational task;

the weight data includes the first weighted value associated with the first type of service call; and

the second weighted value associated with the second type of service call.

4. A method comprising:

receiving, by a load balancer, a first service call comprising a request for performance of a first computational task;

providing the first service call to a first host, the first host comprising one or more computing devices configured to process a single service call at a time;

generating first resource consumption data indicating a first amount of resources consumed to perform the first computational task;

receiving, by a load balancer, a second service call comprising a request for performance of a second computational task;

providing the second service call to the first host;

generating second resource consumption data indicating a second amount of resources consumed to perform the second computational task;

accessing configurable formula data indicative of a first weight factor associated with the first resource consumption data and indicative of a second weight factor associated with the second resource consumption data;

generating a first weighted value by multiplying the first weight factor by the first resource consumption data;

generating a second weighted value by multiplying the second weight factor by the second resource consumption data;

generating weight data comprising a value generated by combining the first weighted value and the second weighted value; and

providing the weight data to one or more second hosts in response to a determination that a threshold amount of resource consumption data has been generated, the one or more second hosts being configured to contemporaneously process a third service call and a fourth service call based on the weight data.

5. The method of claim 4, wherein the first host is configured to process a single service call in response to the first host receiving configuration data used to cause the first host to be configured to have a single connection with the load balancer.

6. The method of claim 4, further comprising:

determining that less than a threshold amount of resource consumption data has been generated; and

providing the first service call to the first host in response to the determination that less than the threshold amount of resource consumption data has been generated, wherein the threshold amount is a predetermined amount determined based on a level of confidence in accuracy of the value of the weight data.

7. The method of claim 4, wherein the first amount of resources consumed to perform the first computational task comprise at least one of processor resources consumed within a first period of time, memory resources consumed within a second period of time, network resources consumed within a third period of time;

or disk resources consumed within a fourth period of time.

8. The method of claim 4, wherein the first service call is provided to the first host based on a predetermined time interval, the predetermined time interval being generated based on inputs received from an administrative user associated with the first host.

9. The method of claim 4, wherein the first amount of resources consumed to perform the first computational task comprises an accumulated amount of resources consumed to perform the first computational task, the accumulated amount including an amount of resources consumed by one or more predecessor service fleets comprising one or more computing devices used to perform other computational tasks to support the performance of the first computational task, wherein the other computational tasks are performed before the first computational task is performed.

10. The method of claim 4, wherein:

the weight data comprises the first weighted value associated with the first type of service call, the first type of service call comprising service calls sharing characteristics different from service calls being of a second type of service call; and

the first, second, third and fourth service calls are of the first type of service call.

11. The method of claim 10, further comprising generating the second weighted value associated with the second type of service call based on a second average amount of resources consumed to perform the second type of service call, wherein the second average amount of resources is generated based on at least a fifth service call and a sixth service call which are of the second type of service call.

12. The method of claim 4, further comprising:

determining that a threshold amount of resource consumption data has been generated; and

in response to the determination that the threshold amount of resource consumption data has been generated, determining to provide the weight data to the one or more second hosts, wherein the threshold amount is a predetermined amount determined based on a level of confidence in accuracy of the value of the weight data.

13. The method of claim 4, wherein:

the first amount of resources consumed to perform the first computational task is determined based on a first average amount of a first resource consumed and the second average amount of a second resource consumed; and

the value of the weight data is generated based on a weighted sum of the first average and the second average.

14. The method of claim 13, wherein the formula data is configurable by enabling a user to set one or more factors used to adjust at least one weight factor applied to the first average or the second average.

15. The method of claim 4, further comprising:

receiving a fifth service call comprising a request for performance of a fifth computational task;

providing the fifth service call to a third host, the third host comprising one or more computing devices configured to process a single service call at a time;

generating third resource consumption data indicating a third amount of resources consumed to perform the fifth computational task; and

wherein the value of the weight data is further generated based on the amount of resources consumed to perform the first computational task, the second computational task, and the fifth computation task.

16. The method of claim 4, wherein the one or more service hosts are configured to determine when to process the third service call and the fourth service call based on the weight data.

17. A method comprising:

receiving service calls for processing a single service call at a time to determine resources consumed for each individual service call;

for a first plurality of service calls having a first type, generating first weighted value by multiplying a first weight factor by first resource consumption data indicating a first amount of resources consumed to perform first computational task associated with the first plurality of service calls;

for a second plurality of service calls having a second type, generating second weighted value by multiplying a second weight factor by second resource consumption data indicating a second amount of resources consumed to perform second computational task associated with the second plurality of service calls;

generating weight data comprising a value generated by combining the first weighted value and the second weighted value; and

providing the weight data to one or more service hosts in response to a determination that a threshold amount of resource consumption data has been generated, the one or more service hosts being configured to determine when to process other service calls based on the weight data.

18. The method of claim 17, wherein the first plurality of service calls and the second plurality of service calls are provided by a client computing device.

19. The method of claim 17, wherein:

the first plurality of service calls are provided by a first client device, the first client device being a first type of device; and

the first weighted value is further generated based on the type of the device.

20. The method of claim 17, further comprising:

determining that the weight data is equal to or more than a threshold value;

generating alert data indicating the occurrence of the weight data being equal to or more than the threshold value; and

providing the alert data to a computing device executing a monitoring service associated with an administrative user of a service fleet.