DYNAMIC VIRTUAL RESOURCE REQUEST RATE CONTROL FOR UTILIZING PHYSICAL RESOURCES

• US 20160080484A1
• Filed: 09/11/2014
• Published: 03/17/2016
• Est. Priority Date: 09/11/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: EARL, WILLIAM JOHN
• Granted Patent: US 9,635,103 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/5013 Request control
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests

Abstract

A virtualization host may implement dynamic virtual resource request rate controls for physical resources. Individual virtual resource request queues may be maintained for different virtual compute instances implemented at a virtualization host for a particular physical computer resource. After placing a work request from one of the individual virtual resource request queues into a physical resource request queue to be performed at the physical computer resource, a delay may be dynamically determined based, at least in part, on the workload of the physical resource request queue. After imposing the delay, a next work request from the individual virtual resource request queue may be placed into the physical resource request queue. In at least some embodiments, the dynamically determined delay may include a randomly added delay.

Claims

1. A system, comprising:

at least one processor;

a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to implement a virtualization host for a plurality of virtual compute instances;

wherein the virtualization host is configured to;

maintain a plurality of individual virtual resource request queues for respective virtual computer resources of the plurality of virtual compute instances that utilize a physical computer resource;

implement a dynamic rate control for individual ones of the plurality of individual virtual resource request queues;

the dynamic rate control, configured to;

place a work request for a virtual computer resource of a virtual compute instance from an individual virtual resource request queue into a physical resource request queue to perform the work request at the physical computer resource;

in response to the placement of the work request;

dynamically determine a delay based, at least in part, on a workload of the physical resource request queue; and

after imposition of the delay, place a next work request from the individual virtual resource request queue into the physical resource request queue;

wherein a work request from at least one other individual virtual resource request queue of the plurality of individual virtual resource request queues is placed in the physical resource request queue during the delay.

2. The system of claim 1, wherein, to dynamically determine the delay, the dynamic rate control is configured to:

identify an initial delay for the individual virtual resource request queue based, at least in part, on a determined utilization of the physical computer resource for the virtual compute instance;

calculate a probability for adding a random delay based, at least in part, on the workload of the physical resource request queue; and

add the random delay to the initial delay according to the calculated probability.

3. The system of claim 2, wherein the determined utilization of the physical computer resource for the virtual compute instance is based, at least in part, on a resource credit balance for the physical resource maintained for the virtual compute instance.

4. The system of claim 1, wherein the virtualization host is implemented as part of a provider network that offers a network-based virtual computing service, wherein the virtualization host is multi-tenant such that at least one of the plurality of virtual compute instances implemented at the virtualization host is maintained for a client of the provider network that is different than another client of the provider network maintaining another one of the plurality of virtual compute instances at the virtualization host.

5. A method, comprising:

performing, by one or more computing devices that together implement a virtualization host for a plurality of virtual compute instances;

placing a work request for a virtual computer resource from an individual virtual resource request queue maintained for a virtual compute instance into a physical resource request queue to perform the work request at a physical computer resource, wherein the virtual compute instance is one of the plurality virtual compute instances, wherein the individual virtual resource request queue is one of a plurality of respective individual virtual resource request queues for respective virtual computer resources including the virtual computer resource of the plurality of virtual compute instances;

in response to placing the work request;

dynamically determining a delay based, at least in part, on a workload of the physical resource request queue; and

after imposing the delay, placing a next work request from the individual virtual resource request queue into the physical resource request queue;

wherein a work request from at least one other individual virtual resource request queue of the plurality of individual virtual resource request queues is placed in the physical resource request queue during the delay.

6. The method of claim 5, wherein dynamically determining the delay comprises:

identifying an initial delay for the individual virtual resource request queue based, at least in part, on a determined utilization of the physical computer resource for the virtual compute instance;

calculating a probability for adding a random delay based, at least in part, on the workload of the physical resource request queue; and

adding the random delay to the initial delay according to the calculated probability.

7. The method of claim 6, wherein the determined utilization of the physical computer resource for the virtual compute instance is based, at least in part, on a resource credit balance for the physical resource maintained for the virtual compute instance.

8. The method of claim 6, wherein determining the workload of the physical resource request queue comprises smoothing one or more workload metrics indicating workload of the physical resource request queue at different points in time.

9. The method of claim 6, further comprising:

in response to placing the next work request into the physical resource request queue;

performing dynamically determining the delay, wherein according to the calculated probability the random delay is not added to the initial delay; and

imposing the initial delay prior to placing another work request from the individual virtual resource request queue into the physical resource request queue.

10. The method of claim 5, wherein placing the work request, dynamically determining the delay, and placing the next work request are performed for other individual virtual resource request queues for different virtual computer resources of the plurality of virtual compute instances that correspond to different physical computer resources.

11. The method of claim 5, wherein the different physical computer resources comprise:

processing resources;

networking resources;

or

storage resources.

12. The method of claim 5, wherein a determined utilization of the physical computer resource for the virtual compute instance is higher than a determined utilization of the physical computer resource for a virtual compute instance for which the work request from the at least one other individual virtual resource request queue is placed in the physical resource request queue during the delay.

13. The method of claim 5, wherein the virtualization host is implemented as part of a provider network that offers a network-based virtual computing service, wherein the virtualization host is multi-tenant such that at least one of the plurality of virtual compute instances implemented at the virtualization host is maintained for a client of the provider network that is different than another client of the provider network maintaining another one of the plurality of virtual compute instances at the virtualization host.

14. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement a virtualization host for a plurality of compute instances, wherein the virtualization host implements:

placing a work request for a virtual computer resource from an individual virtual resource request queue maintained for a virtual compute instance into a physical resource request queue to perform the work request at a physical computer resource, wherein the virtual compute instance is one of the plurality virtual compute instances, wherein the individual virtual resource request queue is one of a plurality of respective individual virtual resource request queues for respective virtual computer resources including the virtual computer resource of the plurality of virtual compute instances;

in response to placing the work request;

dynamically determining a delay based, at least in part, on a workload of the physical resource request queue; and

after imposing the delay, placing a next work request from the individual virtual resource request queue into the physical resource request queue;

wherein a work request from at least one other individual virtual resource request queue of the plurality of individual virtual resource request queues is placed in the physical resource request queue during the delay.

15. The non-transitory, computer-readable storage medium of claim 14, wherein, in dynamically determining the delay, the program instructions cause the virtualization host to further implement:

identifying an initial delay for the individual virtual resource request queue based, at least in part, on a determined utilization of the physical computer resource for the virtual compute instance;

calculating a probability for adding a random delay based, at least in part, on the workload of the physical resource request queue; and

adding the random delay to the initial delay according to the calculated probability.

16. The non-transitory, computer-readable storage medium of claim 15, wherein the determined utilization of the physical computer resource for the virtual compute instance is based, at least in part, on a resource credit balance for the physical resource maintained for the virtual compute instance.

17. The non-transitory, computer-readable storage medium of claim 15, wherein determining the workload of the physical resource request queue comprises smoothing one or more workload metrics indicating workload of the physical resource request queue at different points in time.

18. The non-transitory, computer-readable storage medium of claim 14, wherein the program instructions cause the virtualization host to further implement:

in response to placing the next work request into the physical resource request queue;

performing dynamically determining the delay, wherein according to the calculated probability the random delay is not added to the initial delay; and

imposing the initial delay prior to placing another work request from the individual virtual resource request queue into the physical resource request queue.

19. The non-transitory, computer-readable storage medium of claim 14, wherein a determined utilization of the physical computer resource for the virtual compute instance is lower than a determined utilization of the physical computer resource for a virtual compute instance for which the work request from the at least one other individual virtual resource request queue is placed in the physical resource request queue during the delay.

20. The non-transitory, computer-readable storage medium of claim 14, wherein the virtualization host is implemented as part of a provider network that offers a network-based virtual computing service, wherein the virtualization host is multi-tenant such that at least one of the plurality of virtual compute instances implemented at the virtualization host is maintained for a client of the provider network that is different than another client of the provider network maintaining another one of the plurality of virtual compute instances at the virtualization host.