Failure-model-driven repair and backup

• US 8,140,914 B2
• Filed: 06/15/2009
• Issued: 03/20/2012
• Est. Priority Date: 06/15/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Murphy, Elissa E. S., Nichols, David A., Mehr, John D.
• Primary Examiner(s): Wilson, Yolanda L
• Application Number:
  US12/484,549
• Publication Number:
  US 20100318837A1
• Time in Patent Office: 1,009 Days
• Field of Search: 714/47.1, 714/47.3
• US Class Current: 714/47.1
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/1461 Backup scheduling policy

Abstract

A predictive failure model is used to generate a failure prediction associated with a node. A repair or backup action may also be determined to perform on the node based on the failure prediction.

Claims

1. A computing device comprising:

a processor;

a failure model operated by the processor and configured to generate failure predictions associated with a node, the failure model having been built based on node data received from the node and based on comparisons of the failure predictions to other node data received from the node subsequent to those failure predictions to determine whether failures occurred;

a state machine operated by the processor and configured to select repair or backup actions to perform on the node based on the failure predictions, the repair or backup actions being associated with the failure predictions by one or more rules or definitions of the state machine; and

memory coupled to the processor and storing at least one of the failure model or the state machine.

2. The computing device of claim 1, wherein the state machine is further configured to prioritize the repair or backup actions.

3. The computing device of claim 1, wherein the failure model comprises one of a plurality of failure models, each of the failure models being configured to predict a different type of failure.

4. The computing device of claim 1, wherein the node has a monitoring agent to:

gather the node data from one or more sensors of the node,

provide the node data to the computing device, and

receive instructions for performing the repair or backup action.

5. A method comprising:

utilizing, by a computing device, a failure model to generate a failure prediction associated with a node, the failure model built based at least in part on comparisons of failure predictions to node data received from the node at times subsequent to the failure predictions to determine whether failures occurred; and

determining, by the computing device, a repair or backup action to perform on the node based on the failure prediction.

6. The method of claim 5, further comprising building the failure model based at least in part on the node data received from the node.

7. The method of claim 6, further comprising receiving the node data from sensors of the node.

8. The method of claim 5, wherein the failure model comprises one of a plurality of failure models, each of the failure models being configured to predict a different type of failure.

9. The method of claim 5, further comprising performing the repair or backup action or instructing the node to perform the repair or backup action.

10. The method of claim 5, wherein the determining of the repair or backup action is performed by a state machine configured to select a repair or backup action associated with the failure prediction by one or more rules or definitions of the state machine.

11. The method of claim 5, further comprising:

receiving, by the state machine, a plurality of failure predictions for the node and one or more other nodes;

selecting repair or backup actions based on the failure predictions; and

prioritizing the repair or backup actions.

12. The method of claim 5, wherein the node comprises a computing device in a client-server system, a computing device in a peer-to-peer system, or an element of a data center.

13. One or more computer storage devices encoded with instructions that, when executed by a processor of a device, configure the processor to perform acts comprising:

utilizing a failure model to generate a failure prediction associated with a node, the failure model built based at least in part on comparisons of failure predictions to node data received from the node at times subsequent to the failure predictions to determine whether failures occurred; and

determining a repair or backup action to perform on the node based on the failure prediction.

14. The computer storage devices of claim 13, wherein the failure prediction predicts a drive failure, a process failure, a controller failure, or an operating system failure and/or includes a predicted failure time.

15. The computer storage devices of claim 13, wherein the acts further comprise building the failure model based on node data received from the node.

16. The computer storage devices of claim 15, wherein the node data includes one or more of a load associated with the node, an indicator from a performance monitor, a temperature associated with the node, a context associated with the node, a log associated with software failures or software failure rates, and a proximity between the node and another node.

17. The computer storage devices of claim 13, wherein the acts further comprise performing the repair or backup action or instructing the node to perform the repair or backup action.

18. The computer storage devices of claim 13, wherein the repair or backup action is one of migrating data, replacing hardware, recovering from a backup copy, redirecting a load, or taking a node out of service.

19. The computer storage devices of claim 13, wherein the acts further comprise utilizing the failure model to generate a plurality of failure predictions and prioritizing repair or backup options based on the plurality of failure predictions.