Adjusting variable limit on concurrent code executions

• US 10,402,231 B2
• Filed: 10/15/2018
• Issued: 09/03/2019
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Marriner, Dylan Owen, Roman, Mauricio, Brooker, Marc John, Herwitz, Julian Embry, Reque, Sean
• Primary Examiner(s): Nguyen, Van H
• Application Number:
  US16/160,674
• Publication Number:
  US 20190050271A1
• Time in Patent Office: 323 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2209/504 Resource capping
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Systems and methods are described for adjusting a number of concurrent code executions allowed to be performed for a given user on an on-demand code execution environment or other distributed code execution environments. Such environments utilize pre-initialized virtual machine instances to enable execution of user-specified code in a rapid manner, without delays typically caused by initialization of the virtual machine instances. However, to improve utilization of computing resources, such environments may temporarily restrict the number of concurrent code executions performed on behalf of the given user to a number less than the maximum number of concurrent code executions allowed for the given user. Such environments may adjust the temporary restriction on the number of concurrent code executions based on the number of incoming code execution requests associated with the given user.

Claims

1. A system comprising:

one or more processors;

one or more memories, the one or more memories having stored thereon instructions, which, when executed by the one or more processors, cause the one or more processors to;

receive a request to execute a program code, the request including an identifier to be used to acquire compute capacity;

identify, using the identifier, a first virtual machine instance manager from a set of virtual machine instance managers configured to acquire compute capacity;

request compute capacity from the first virtual machine instance manager;

determine that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity;

cause a second virtual machine instance manager to be added to the set, the second virtual machine instance manager being different from the first virtual machine instance manager;

request compute capacity from the second virtual machine instance manager; and

cause the program code to be executed using the compute capacity acquired by the second virtual machine instance manager.

2. The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to determine, prior to causing the second virtual machine instance manager to be added to the set, that a condition for adding an additional virtual machine instance manager to the set is satisfied.

3. The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

calculate a hash value by applying a hash function to the identifier included in the request; and

select the first virtual machine instance manager corresponding to the calculated hash value.

4. The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

access a cache indicating the set of virtual machine instance managers associated with the identifier; and

select the first virtual machine instance manager from the set of virtual machine instance managers.

5. The system of claim 1, wherein the virtual machine instance managers in the set are arranged in a specific order, the instructions, when executed by the one or more processors, further cause the one or more processors to:

receive another request to execute the program code, said another request including the identifier; and

request compute capacity from a next virtual machine instance manager in the set that immediately follows the second virtual machine instance manager in the specific order.

6. The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to cause the second virtual machine instance manager to be removed from the set in response to determining that a condition for removing a virtual machine instance manager from the set is satisfied.

7. The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

in response to determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity, request load information associated with the identifier from another frontend module associated with another set of virtual machine instance managers; and

cause a number of virtual machine instance managers in said another set to be adjusted based on the load information.

8. A computer-implemented method comprising:

receiving a request to execute a program code, the request including an identifier to be used to acquire compute capacity;

identifying, using the identifier, a first virtual machine instance manager from a set of virtual machine instance managers configured to acquire compute capacity;

requesting compute capacity from the first virtual machine instance manager;

determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity;

adding a second virtual machine instance manager to the set, the second virtual machine instance manager being different from the first virtual machine instance manager;

requesting compute capacity from the second virtual machine instance manager; and

executing the program code using the compute capacity acquired by the second virtual machine instance manager.

9. The computer-implemented method of claim 8, further comprising determining, prior to adding the second virtual machine instance manager to the set, that a condition for adding an additional virtual machine instance manager to the set is satisfied.

10. The computer-implemented method of claim 8, wherein identifying the first virtual machine instance manager comprises:

calculating a hash value by applying a hash function to the identifier included in the request; and

selecting the first virtual machine instance manager corresponding to the calculated hash value.

11. The computer-implemented method of claim 8, wherein identifying the first virtual machine instance manager comprises:

accessing a cache indicating the set of virtual machine instance managers associated with the identifier; and

selecting the first virtual machine instance manager from the set of virtual machine instance managers.

12. The computer-implemented method of claim 8, wherein the virtual machine instance managers in the set are arranged in a specific order, the method further comprising:

receiving another request to execute the program code, said another request including the identifier; and

requesting compute capacity from a next virtual machine instance manager in the set that immediately follows the second virtual machine instance manager in the specific order.

13. The computer-implemented method of claim 8, further comprising, in response to determining that a condition for removing a virtual machine instance manager from the set is satisfied, removing the second virtual machine instance manager from the set.

14. The computer-implemented method of claim 8, further comprising:

in response to determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity, requesting load information associated with the identifier from another frontend module associated with another set of virtual machine instance managers; and

adjusting a number of virtual machine instance managers in said another set based on the load information.

15. Non-transitory physical computer storage including computer-executable instructions that, when executed by a computing system, cause the computing system to:

receive a request to execute a program code, the request including an identifier to be used to acquire compute capacity;

identify, using the identifier, a first virtual machine instance manager from a set of virtual machine instance managers configured to acquire compute capacity;

request compute capacity from the first virtual machine instance manager;

determine that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity;

cause a second virtual machine instance manager to be added to the set, the second virtual machine instance manager being different from the first virtual machine instance manager;

request compute capacity from the second virtual machine instance manager; and

cause the program code to be executed using the compute capacity acquired by the second virtual machine instance manager.

16. The non-transitory physical computer storage of claim 15, wherein the instructions, when executed by the computing system, further cause the computing system to:

calculate a hash value by applying a hash function to the identifier included in the request; and

select the first virtual machine instance manager corresponding to the calculated hash value.

17. The non-transitory physical computer storage of claim 15, wherein the instructions, when executed by the computing system, further cause the computing system to:

access a cache indicating the set of virtual machine instance managers associated with the identifier; and

select the first virtual machine instance manager from the set of virtual machine instance managers.

18. The non-transitory physical computer storage of claim 15, wherein the virtual machine instance managers in the set are arranged in a specific order, the instructions, when executed by the computing system, further cause the computing system to:

receive another request to execute the program code, said another request including the identifier; and

request compute capacity from a next virtual machine instance manager in the set that immediately follows the second virtual machine instance manager in the specific order.

19. The non-transitory physical computer storage of claim 15, wherein the instructions, when executed by the computing system, further cause the computing system to cause the second virtual machine instance manager to be removed from the set in response to determining that a condition for removing a virtual machine instance manager from the set is satisfied.

20. The non-transitory physical computer storage of claim 15, wherein the instructions, when executed by the computing system, further cause the computing system to:

in response to determining that the first virtual machine instance manager is unable to or has failed to acquire the requested compute capacity, request load information associated with the identifier from another frontend module associated with another set of virtual machine instance managers; and

cause a number of virtual machine instance managers in said another set to be adjusted based on the load information.