Proactive cloud orchestration

• US 10,558,517 B2
• Filed: 07/11/2017
• Issued: 02/11/2020
• Est. Priority Date: 03/09/2015
• Status: Active Grant

• Current Assignee: Seagate Technology LLC (Seagate Technology Holdings Plc)
• Original Assignee: Seagate Technology LLC (Seagate Technology Holdings Plc)
• Inventors: Rajasekharan, Ajaykumar, Madsen, Christian Brunn, Khurshudov, Andrei
• Primary Examiner(s): Manoskey, Joseph D
• Application Number:
  US15/646,510
• Publication Number:
  US 20170308429A1
• Time in Patent Office: 945 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3616 using software metrics
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: H04L 41/0661 by reconfiguring faulty entities
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 43/0823 Errors, e.g. transmission errors
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

Methods, systems, and devices are described for providing proactive cloud orchestration services for a cloud hardware infrastructure. A health management system may monitor component(s) of the cloud hardware infrastructure. The health management system may determine a failure probability metric for the component(s) based on the monitoring of the component and in consideration of historical information associated with the component, or similar components. The health management system may determine an optimization strategy for the component and, when an optimization decision has been reached, initiate a reconfiguration procedure to implement the optimization strategy. The optimization strategy may provide for mitigating or eliminating the consequences of the component failure associated with data loss, downtime, and the like.

Claims

1. A method, comprising:

analyzing one or more components of a cloud hardware infrastructure, the cloud hardware infrastructure comprising one or more storage drives; and

determining a failure probability metric for at least one of the one or more components based at least in part on the analyzing, the failure probability metric comprising an indication of a time until an expected failure for the at least one component of the cloud hardware infrastructure, determining the failure probability metric comprising analyzing, for the at least one component, a cluster migration metric.

2. The method of claim 1, wherein determining the failure probability metric comprises:

determining, based at least in part on the failure probability metric, an optimization strategy for the cloud hardware infrastructure; and

initiating a reconfiguration procedure to implement the optimization strategy.

3. The method of claim 2, wherein determining the optimization strategy comprises:

determining a resource utilization metric associated with the cloud hardware infrastructure; and

determining the optimization strategy based at least in part on the resource utilization metric and the failure probability metric.

4. The method of claim 3, wherein the resource utilization metric is based at least in part on a workload distribution of the cloud hardware infrastructure, a data redundancy metric for one or more data storage components of the cloud hardware infrastructure, a servicing schedule for the cloud hardware infrastructure, a resource requirements metric for the cloud hardware infrastructure, or combinations thereof.

5. The method of claim 1, wherein determining the failure probability metric further comprises:

analyzing, for the at least one component, one or more historical models associated with the at least one component, a reliability model associated with the at least one component, a self-monitoring feedback metric from the at least one component, a self-testing metric, or combinations thereof.

6. The method of claim 5, further comprising:

analyzing the one or more historical models, the reliability model, the self-monitoring feedback metric, the self-testing metric, the cluster migration metric, or combinations thereof, a predetermined number of iterations to determine the failure probability metric.

7. The method of claim 1, wherein the reconfiguration procedure comprises:

transferring, based at least in part on a failure probability metric that indicates the at least one component will fail within a time period, one or more resources of the cloud hardware infrastructure to a different component.

8. The method of claim 7, wherein the one or more resources comprises a data resource, a computational resource, a storage resource, a communication resource, or combinations thereof.

9. The method of claim 1, wherein the reconfiguration procedure comprises:

reducing, based at least in part on a failure probability metric that indicates the at least one component will not fail within a time period, the resources associated with a redundancy function of the cloud hardware infrastructure.

10. The method of claim 9, wherein reducing the resources of the redundancy function comprises reducing a number of redundant storage locations of a storage system.

11. An apparatus, comprising:

a hardware processing system executing software instructions to perform one or more functions of at least one of a prognostic engine, an orchestration engine, and a procedure engine, the prognostic engine configured to analyze one or more components of a cloud hardware infrastructure and determine a failure probability metric associated with at least one or more components of the cloud hardware infrastructure, the failure probability metric comprising an indication of a time until an expected failure for the at least one component of the cloud hardware infrastructure, the cloud hardware infrastructure comprising one or more storage drives, determining the failure probability metric comprising analyzing, for the at least one component, a cluster migration metric.

12. The apparatus of claim 11, further comprising:

the orchestration engine configured to determine, based at least in part on the failure probability metric, an optimization strategy for the cloud hardware infrastructure; and

a reconfiguration engine configured to initiate a reconfiguration procedure to implement the optimization strategy.

13. The apparatus of claim 11, wherein the prognostic engine is further configured to analyze, for the at least one component, one or more historical models associated with the at least one component, a reliability model associated with the at least one component, a self-monitoring feedback metric from the at least one component, a self-testing metric, or combinations thereof, to determine the failure probability metric.

14. The apparatus of claim 11, wherein the orchestration engine is further configured to determine a resource utilization metric associated with the cloud hardware infrastructure, and to determine the optimization strategy based at least in part on the resource utilization metric and the failure probability metric.

15. The apparatus of claim 14, wherein the resource utilization metric is based at least in part on a workload distribution of the cloud hardware infrastructure, a data redundancy metric for one or more data storage components of the cloud hardware infrastructure, a servicing schedule for the cloud hardware infrastructure, a resource requirements metric for the cloud hardware infrastructure, or combinations thereof.

16. The apparatus of claim 11, wherein the reconfiguration engine is configured to transfer, based at least in part on a failure probability metric that indicates the at least one component will fail within a time period, one or more resources of the cloud hardware infrastructure to a different component.

17. The apparatus of claim 11, wherein the reconfiguration engine is configured to reduce, based at least in part on a failure probability metric that indicates the at least one component will not fail within a time period, the resources associated with a redundancy function of the cloud hardware infrastructure.

18. A cloud hardware infrastructure system, comprising:

one or more components of the cloud hardware infrastructure system;

a communication system providing communications between at least a portion of the one or more components of the cloud hardware infrastructure; and

a health management system configured to;

analyze one or more components of a cloud hardware infrastructure, the cloud hardware infrastructure comprising one or more storage drives, wherein analyzing the one or more components comprises analyzing a self-testing metric; and

determine a failure probability metric for at least one of the one or more components based at least in part on the analyzing, the failure probability metric comprising an indication of a time until an expected failure for the at least one component of the cloud hardware infrastructure, determining the failure probability metric comprising analyzing, for the at least one component, a cluster migration metric.

19. The system of claim 18, wherein the health management system is configured to:

determine, based at least in part on the failure probability metric, an optimization strategy for the cloud hardware infrastructure system; and

initiate a reconfiguration procedure to implement the optimization strategy.