Rolling resource credits for scheduling of virtual computer resources

• US 10,649,796 B2
• Filed: 07/15/2014
• Issued: 05/12/2020
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Phillips, John Merrill, Earl, William John, Singh, Deepak
• Primary Examiner(s): Bullock, Jr., Lewis A
• Assistant Examiner(s): Kepnang, Gilles R
• Application Number:
  US14/331,745
• Publication Number:
  US 20150378753A1
• Time in Patent Office: 2,128 Days
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06Q 20/22 Payment schemes or models

Abstract

A network-based virtual computing resource provider may offer virtual compute instances that implement rolling resource credits for scheduling virtual computing resources. Work requests for a virtual compute instance may be received at a virtualization manager. A resource credit balance may be determined for the virtual compute instance. The resource credit balance may accumulate resource credits in rolling fashion, carrying over unused credits from previous time periods. Resource credits may then be applied when generating scheduling instructions to provide to a physical resource to perform the work requests, such as a physical CPU in order to increase the utilization of the resource according to the number of credits applied. Applied resource credits may then be deducted from the credit balance.

Claims

1. A system, comprising:

a compute node, comprising at least one respective processor and a memory, that implements a virtualization host;

the virtualization host comprising executable instructions, configured to;

receive a work request for one or more virtual central processing units (vCPUs) submitted by a virtual compute instance as part of performing an application for a client that reserved the virtual compute instance;

automatically add a same number of resource credits, at a resource credit accumulation rate set for the vCPUs, to a current resource credit balance for respective consecutive time periods to calculate the current resource credit balance for the virtual compute instance respective to the vCPUs, wherein;

the current resource credit balance comprises a set of current resource credits that, when used, increase processing time allocated for a time period in excess of a baseline amount of processing time provided by a baseline utilization guarantee; and

at least one current resource credit of the current resource credit balance was automatically added to the current resource credit balance from a particular time period prior to the current time period, wherein during the particular time period, the virtualization host had performed one or more prior work requests, submitted by the virtual compute instance as part of performing the application for the client, that utilized a prior amount of processing time not in excess of the baseline amount of processing time provided by the baseline utilization guarantee;

determine that a current amount of processing time to perform the work request, for the current time period, submitted by the virtual compute instance as part of performing the application for the client, is in excess of the baseline amount of processing time provided by the baseline utilization guarantee for the current time period, and responsive to the determination;

use the at least one current resource credit, automatically added for the particular time period prior to the current time period, from the current resource credit balance to increase allocation of processing time of the respective at least one processor for the current time period in excess of the baseline amount of processing time provided by the baseline utilization guarantee for the virtual compute instance;

generate one or more scheduling instructions that schedule the work request for performance utilizing the respective at least one processor of the compute node according to the use of the at least one resource credit;

perform the work request utilizing the respective at least one processor of the compute node according to the one or more scheduling instructions; and

update the current resource credit balance to deduct the at least one resource credit applied to perform the work request.

2. The system of claim 1, wherein to generate the one or more scheduling instructions, the virtualization host is further configured to:

subsequent to the application of the one or more resource credits to the work request;

in response to a determination that there is no remaining resource credit in the current resource credit balance to apply to the work request, configure at least some of the one or more scheduling instructions such that utilization of the respective at least one processor is incrementally lowered to the baseline utilization for the virtual compute instance.

3. The system of claim 1,

wherein the virtualization host comprises a monitoring agent, configured to track the current resource credit balance for the virtual compute instance over time and applied resource credits for work requests for the virtual compute instance over time;

wherein the compute node is implemented as part of a plurality of compute nodes that together implement a network-based virtual computing service, wherein the network-based virtual computing service comprises a network-based interface configured to provide to the client of the network-based virtual computing service the current resource credit balance for the virtual compute instance over time or applied resource credits for work requests for the virtual compute instance over time.

4. The system of claim 3, wherein the virtual compute instance is one of a plurality of different, types of virtual compute instance offered via the network-based virtual computing service, and wherein each of the different types of virtual compute instance corresponds to a different respective resource credit accumulation rate.

5. A method, comprising:

performing, by one or more computing systems;

receiving, at a virtualization manager for a virtualization host, a work request utilizing one or more virtual computer resources submitted by a virtual compute instance hosted as part of performing an application for a client that reserved the virtual compute instance at the virtualization host;

automatically adding a same number of resource credits, at a resource credit accumulation rate set for the one or more virtual computer resources, to a current resource credit balance for the virtual compute instance respective to the one or more virtual computer resources, wherein;

the current resource credit balance comprises a set of current resource credits that, when used, increase processing time of the one or more virtual computer resources for a time period in excess of a baseline amount of processing time provided by a baseline utilization guarantee; and

at least one current resource credit of the current resource credit balance was automatically added to the current resource credit balance from a particular time period prior to the current time period, wherein during the particular time period, the virtualization host had performed one or more prior work requests, submitted by the virtual compute instance as part of performing the application for the client, that utilized a prior amount of processing time not in excess of the baseline amount of processing time provided by the baseline utilization guarantee;

determining that a current amount of processing time to perform the work request for the current time period, submitted by the virtual compute instance as part of performing the application for the client, is in excess of the baseline amount of processing time provided by the baseline utilization guarantee for the current time period, and responsive to the determination;

using the at least one current resource credit, automatically added for the particular time period prior to the current time period, from the current resource credit balance to increase processing time of the respective one or more virtual computer resources for the current time period in excess of the baseline amount of processing time provided by the baseline utilization guarantee for the virtual compute instance;

generating one or more scheduling instructions that schedule the work request for performance at one or more physical computer resources implemented as part of the virtualization host according to the use of the at least one resource credit;

performing the work request utilizing the respective at least one processor of the compute node according to the one or more scheduling instructions; and

updating the current resource credit balance for the one virtual compute instance to deduct the at least one resource credit applied to perform the work request.

6. The method of claim 5, further comprising:

wherein said receiving, said determining, said applying, said generating, said performing, and said updating are performed for a plurality of different work requests;

recording, as credit balance metrics, the determined current resource credit balance for each of the plurality of different work requests; and

recording, as credit usage metrics, the applied one or more resource credits for each of the plurality of different work requests.

7. The method of claim 6, further comprising:

receiving a request via a network-based interface for at least some of the credit balance metrics for the virtual compute instance; and

in response to receiving the request, providing the at least some credit balance metrics for the virtual compute instance via the network-based interface.

8. The method of claim 6, further comprising:

receiving a request via a network-based interface for at least some of the credit usage metrics for the virtual compute instance; and

in response to receiving the request, providing the at least some credit usage metrics for the virtual compute instance via the network-based interface.

9. The method of claim 6, further comprising:

monitoring the credit balance metrics or the credit usage metrics for the virtual compute instance;

based, at least in part, on said monitoring, detecting an alarm event for the virtual compute instance; and

in response to detecting the alarm event, providing a notification of the alarm event for the virtual compute instance.

10. The method of claim 5, wherein said generating the one or more scheduling instructions to send to the scheduler comprises:

subsequent to applying the one or more resource credits to the work request;

determining that there is no remaining resource credit in the current resource credit balance to apply to the work request;

in response to determining that there is no remaining resource credit, configuring at least one of the one or more generated scheduling instructions such that utilization of the one or more physical computing resources is incrementally lowered to the baseline utilization for the virtual compute instance.

11. The method of claim 5, wherein the one or more virtual computer resources are one or more virtual central processing units (vCPUs), wherein the one or more physical computer resources are one or more central processing units (CPUs), and wherein the method further comprises sending the one or more scheduling instructions to a scheduler for the one or more physical CPUs.

12. The method of claim 5, wherein said determining the current resource credit balance for the one virtual compute instance comprises excluding, from the current resource credit balance, those unused resource credits accumulated prior to a resource credit accumulation time period.

13. The method of claim 5, wherein the one or more virtual computer resources are some of a plurality of different computer resources, wherein a different respective current resource credit balance is maintained for different ones of the plurality of different computer resources, wherein another work request is received for another one of the plurality of different computer resources, and wherein said determining, said applying, said generating, said performing, and said updating are performed for the other work request.

14. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:

receiving, at a virtualization manager for a virtualization host, a work request for one or more virtual computer resources submitted by a virtual compute instance as part of performing an application for a client that reserved the virtual compute instance for the client hosted at the virtualization host;

automatically adding a same number of resource credits, at a resource credit accumulation rate set for the one or more virtual computer resources, to a current resource credit balance for respective consecutive time periods to calculate the current resource credit balance for the virtual compute instance respective to the one or more virtual computer resources, wherein;

the current resource credit balance comprises a set of current resource credits that, when used, increase processing time of the one or more virtual computer resources for a time period in excess of a baseline amount of processing time provided by a baseline utilization guarantee; and

at least one current resource credit of the current resource credit balance was automatically added to the current resource credit balance from a particular time period prior to the current time period, wherein duns the particular time period, the virtualization host had performed one or more prior work requests, submitted by the virtual compute instance as part of performing the application for the client, that utilized a prior amount of processing time not in excess of the baseline amount of processing time provided by the baseline utilization guarantee;

determining that a current amount of processing time to perform the work request for the current time period, submitted by the virtual compute instance as part of performing the application for the client, is in excess of the baseline amount of processing time provided by the baseline utilization guarantee for the current time period, and responsive to the determination;

using the at least one current resource credit, automatically added for the particular time period prior to the current time period, from the current resource credit balance to increase the current amount of processing time of the respective one or more virtual computer resources for the current time period in excess of the amount of processing time provided by the baseline utilization guarantee for the virtual compute instance;

generating one or more scheduling instructions that schedule the work request for performance utilizing at one or more physical computer resources implemented as part of the virtualization host, according to the use of the at least one resource credits;

performing the work request utilizing the one or more physical computer resources according to the one or more scheduling instructions; and

updating the current resource credit balance for the one virtual compute instance to deduct the at least one resource credit applied to perform the work request.

15. The non-transitory, computer-readable storage medium of claim 14, wherein the program instructions cause the one or more computing devices to further implement:

wherein said receiving, said determining, said applying, said generating, said performing, and said updating are performed for a plurality of different work requests;

recording, as credit balance metrics, the determined current resource credit balance for each of the plurality of different work requests; and

recording, as credit usage metrics, the applied one or more resource credits for each of the plurality of different work requests.

16. The non-transitory, computer-readable storage medium of claim 15, wherein the program instructions cause the one or more computing devices to further implement:

monitoring the credit balance metrics or the credit usage metrics for the virtual compute instance;

based, at least in part, on said monitoring, detect an alarm event for the virtual compute instance; and

in response to detecting the alarm event, providing a notification of the alarm event for the virtual compute instance.

17. The non-transitory, computer-readable storage medium of claim 14, wherein the one or more virtual computer resources are one or more networking resources and wherein the one or more physical computer resources are one or more network communication devices.

18. The non-transitory, computer-readable storage medium of claim 14, wherein, in said generating the one or more scheduling instructions to send to the scheduler, the program instructions cause the one or more computing devices to implement:

subsequent to applying the one or more resource credits to the work request;

determining that there is no remaining resource credit in the current resource credit balance to apply to the work request;

in response to determining that there is no remaining resource credit, configuring at least one of the one or more generated scheduling instructions such that utilization of the one or more physical computer resources is incrementally lowered to the baseline utilization for the virtual compute instance.

19. The non-transitory, computer-readable storage medium of claim 14, wherein, in said determining the current resource credit balance for the virtual compute instance, the program instructions cause the one or more computing devices to implement excluding, from the current resource credit balance, those unused resource credits accumulated prior to a resource credit accumulation time period.

20. The non-transitory, computer-readable storage medium of claim 14, wherein the virtualization host is implemented as part of a network-based virtual computing service, wherein the one virtual compute instance is one of a plurality of different types of virtual compute instance offered via the network-based virtual computing service, and wherein each of the different types of virtual compute instance corresponds to a different respective resource credit accumulation rate.