Cost optimization of cloud computing resources

• US 10,026,070 B2
• Filed: 04/28/2016
• Issued: 07/17/2018
• Est. Priority Date: 04/28/2015
• Status: Expired due to Fees

• Current Assignee: Solano Labs, Inc..
• Original Assignee: Solano Labs, Inc..
• Inventors: Thorpe, Christopher A., Josephson, William, Moorthi, Jay, Willis, Steven R.
• Primary Examiner(s): CAO, DIEM K
• Application Number:
  US15/140,727
• Publication Number:
  US 20160321115A1
• Time in Patent Office: 810 Days
• Field of Search: None
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06Q 10/0631 Resource planning, allocation, distributing or scheduling for enterprises or organisations
• CPC Class Codes: G06Q 20/145 Payments according to the detected use or quantity
• CPC Class Codes: H04L 47/823 Prediction of resource usage
• CPC Class Codes: H04L 47/83 based on usage prediction

Abstract

Systems and methods are provided for managing and/or executing cloud compute instances that may be pre-empted by their providers, known in at least one provider as “spot instances”. Various aspects manage early termination (i.e., before customer deallocation or compute task completion) of spot instances, enabling improved utilization, and increasing reliability of executing tasks on spot instances. In some embodiments, systems and/or methods are provided for prediction of when spot instance kills are likely to occur. These systems and/or methods can gracefully handle spot kills issued by Providers (e.g., trigger preservation operations), trigger transitions to spot instances at another Provider, trigger increased bidding to preserve current spot instances, and/or trigger a premium payment to allow for execution of preservation operations, among other options. Other embodiments enable customers to directly manage instance kill operations (e.g., a customer or a customer'"'"'s program selects instances to terminate via a UI or API, etc.).

Claims

1. A system for managing cloud compute resources, the system comprising:

at least one processor operatively connected to a memory, the at least one processor when executing configured to;

analyze historical time series of cloud compute resource (“

resource”

) characteristics, including at least cost and availability of the cloud compute resources, for one or more resource types from one or more cloud compute providers (“

providers”

);

model future resource characteristics for executing subsequent tasks responsive to analysis of the historical time series data;

predict one or more resource characteristics including at least resource availability over a future time duration at the one or more cloud compute providers required for executing a submitted compute task based on the model;

allocate terminable cloud compute resources to the submitted compute task and trigger execution of the submitted compute task;

at least one application program interface (API) configured to;

monitor resource characteristics of the terminable cloud compute resources and for the one or more resource types assigned to execute the submitted compute task from the one or more cloud compute providers;

update during the execution period predictions for one or more resource characteristics for remaining portions of the future time duration; and

update the historical time series of resource characteristics data with the monitored resource characteristics and update the model, including associated predictions of the one or more resource characteristics for determining re-allocation of execution resources of the compute task at one or more of the cloud compute providers; and

wherein the at least one processor is further configured to schedule or trigger execution of preservation operations in response to a likelihood of termination of the terminable resources exceeding a threshold value.

2. The system of claim 1, wherein the at least one processor is further configured to:

predict a likelihood of a spot kill which includes deallocation of a provisioned resource based at least in part on the predicted resource characteristics; and

update the prediction on the likelihood of the spot kill based on the updated predictions for the resource characteristics from the one or more providers.

3. The system of claim 1, wherein the terminable resources includes a spot instance.

4. The system of claim 1, wherein the at least one processor is further configured to:

monitor real-time characteristics of resources running compute tasks;

evaluate whether the characteristics are not optimal;

schedule execution of preservation operations; and

terminate resources with suboptimal characteristics once preservation operations have completed.

5. The system of claim 4, wherein the at least one processor is further configured to:

determine the characteristics are not optimal based on modeling execution characteristics across one or more cloud compute providers and identifying better performance characteristics at the current cloud compute provider or the one or more cloud compute providers.

6. The system of claim 1, wherein the at least one processor is further configured to:

schedule execution of recovery operations in response to a task termination being predicted.

7. The system of claim 6, wherein the at least one processor is further configured to:

receive and store one or more intermediate results of submitted compute tasks; and

provide the one or more intermediate results to a restarted task during execution of recovery operations for a task terminated.

8. The system of claim 1, wherein the at least one processor is further configured to:

predict availability characteristics for resources over the future time duration for executing the submitted compute task based on predicting availability characteristics for resources based on historical, projected or inferred cost characteristics associated with the resources.

9. The system of claim 1, wherein the at least one processor is further configured to:

predict availability characteristics for resources over the future time duration, based on predicted utilization of available resources from on historical, projected or inferred utilization of the one or more provider'"'"'s resources.

10. The system of claim 9, wherein the at least one processor is further configured to:

determine a utilization model based on the projected or inferred utilization to achieve at least one optimization goal from a group comprising;

maximize revenue, minimize costs, maximize resource utilization, minimize spot kills, minimize time to completion, and minimize power consumption.

11. A method for managing cloud compute resources, the method comprising:

analyzing historical time series of cloud compute resource (“

resource”

) characteristics, including at least cost and availability, for one or more resource types from one or more cloud compute providers by at least one application program interface (API);

model future resource characteristics for executing subsequent tasks responsive to analysis of the historical time series data;

predicting, by a computer system, one or more resource characteristics over a future time duration the one or more resource characteristics define parameters for executing a submitted compute task at the one or more cloud compute providers;

allocate terminable cloud compute resources to the submitted compute task and trigger execution of the submitted compute task;

monitoring, by at least one API, resource characteristics for the one or more resource types from the one or more cloud compute providers during execution of the submitted compute task;

updating, by the at least one API, during an execution period predictions for one or more resource characteristics for remaining portions of the future time duration responsive to analysis the monitored resource characteristics; and

updating, by the at least one API, the historical time series of resource characteristics data with the monitored resource characteristics and update the model including associated predictions of the one or more resource characteristics for determining re-allocation of execution resources of the compute task at one or more of the cloud compute providers; and

wherein the method further comprises scheduling or triggering execution of preservation operations in response to a likelihood of termination of the terminable resources exceeding a threshold value.

12. The method of claim 11, further comprising an act of predicting, by a computer system, a likelihood of a spot kill which includes deallocation of a provisioned resource based at least in part on the predicted resource characteristics;

and

updating, by the computer system, the prediction on the likelihood of the spot kill based on the updated predictions for the resource characteristics from the one or more providers.

13. The method of claim 11, further comprising the acts of:

monitoring, by the computer system, real-time characteristics of resources running compute tasks;

evaluating whether the characteristics are not optimal;

scheduling, by the computer system, execution of preservation operations; and

proactively terminating resources with suboptimal characteristics once preservation operations have completed.

14. The method of claim 13, wherein evaluating the characteristics are not optimal includes modeling execution characteristics across a plurality of other cloud compute providers and identifying better performance characteristics at the current cloud compute provider or the plurality of other cloud compute providers.

15. The method of claim 11, further comprising an act of scheduling, by the computer system, execution of recovery operations in response to a task termination being predicted.

16. The method of claim 15, further comprising:

receiving and storing, by the computer system, one or more intermediate results of submitted compute tasks; and

providing, by the computer system, the one or more intermediate results to a restarted task during execution of recovery operations for a task terminated.

17. The method of claim 11, wherein the act of predicting, by the computer system, availability characteristics for resources over the future time duration for executing the submitted compute task, includes an act of predicting availability characteristics for resources based on historical, projected or inferred cost characteristics associated with the resources.

18. The method of claim 11, wherein the act of predicting, by the computer system, availability characteristics for resources over the future time duration, includes an act of predicting utilization of available resources based on historical, projected or inferred utilization of the one or more provider'"'"'s resources.

19. The method of claim 18, further comprising an act of determining a utilization model based on the projected or inferred utilization to achieve at least one optimization goal from a group comprising:

maximize revenue, minimize costs, maximize resource utilization, minimize spot kills, minimize time to completion, and minimize power consumption.