Automated learning of failure recovery policies

• US 8,024,611 B1
• Filed: 02/26/2010
• Issued: 09/20/2011
• Est. Priority Date: 02/26/2010
• Status: Active Grant

• Current Assignee: ServiceNow Incorporated
• Original Assignee: Microsoft Corporation
• Inventors: Meek, Christopher A., Shani, Guy
• Primary Examiner(s): Beausoliel, Jr.; Robert
• Assistant Examiner(s): Mehrmanesh; Elmira
• Application Number:
  US12/713,195
• Publication Number:
  US 20110214006A1
• Time in Patent Office: 571 Days
• Field of Search: 714/2, 714/26, 714/33, 714/48
• US Class Current: 714/26
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36

Abstract

Described is automated learning of failure recovery policies based upon existing information regarding previous policies and actions. A learning mechanism automatically constructs a new policy for controlling a recovery process, based upon collected observable interactions of an existing policy with the process. In one aspect, the learning mechanism builds a partially observable Markov decision process (POMDP) model, and computes the new policy base upon the learned model. The new policy may perform automatic fault recovery, e.g., on a machine in a datacenter corresponding to the controlled process.

Claims

1. A system comprising:

a model learning component configured to access collected observable interactions of an existing repair policy with a process to build a model of the process, the model mapping states of the process to repair actions of the existing repair policy;

a policy computation component configured to compute a new policy based upon the model, the new policy identifying a number of times to retry a first one of the repair actions when the process is in a first one of the states of the process;

a controller configured to apply the new policy to the process and, in an instance when the first state is identified, retry the first repair action the number of times identified by the new policy before escalating the first repair action to a second one of the repair actions; and

one or more processing units configured to execute at least one of the model learning component, the policy computation component, or the controller.

2. The system of claim 1 further comprising a collection component configured to collect the observable interactions into a data structure for access by the model learning component.

3. The system of claim 1 wherein the new policy performs automatic fault recovery.

4. The system of claim 1, wherein the model comprises a partially observable Markov decision process.

5. The system of claim 4, wherein the model learning component uses an expectation maximization algorithm to learn the partially observable Markov decision process.

6. The system of claim 4, wherein the policy computation component computes the new policy using a point-based value iteration algorithm.

7. The system of claim 6, wherein the point-based value iteration algorithm is executed by a cost-based indefinite-horizon formalization.

8. The system of claim 1, wherein the process corresponds to a computing machine in a datacenter.

9. A method comprising:

accessing collected observable interactions of an existing repair policy with a process to build a model of the process, the model mapping states of the process to repair actions of the existing repair policy;

computing a new policy based upon the model, the new policy identifying a number of times to retry a first one of the repair actions when the process is in a first one of the states of the process; and

applying the new policy to the process and, in an instance when the first state is identified, retrying the first repair action the number of times identified by the new policy before escalating the first repair action to a second one of the repair actions.

10. The method according to claim 9, the first state reflecting an error message generated by the process.

11. The method according to claim 9, the observable interactions being accessed from a recovery log associated with the process.

12. The method according to claim 9, the model being computed based on a first action cost associated with the first repair action and a second action cost associated with the second repair action.

13. The method according to claim 12, the second action cost being higher than the first action cost.

14. One or more computer-readable storage devices comprising instructions which, when executed by one or more processing units, cause the one or more processing units to perform:

accessing collected observable interactions of an existing repair policy with a process to build a model of the process, the model mapping states of the process to repair actions of the existing repair policy;

computing a new policy based upon the model, the new policy identifying a number of times to retry a first one of the repair actions when the process is in a first one of the states of the process; and

applying the new policy to the process and, in an instance when the first state is identified, retrying the first repair action the number of times identified by the new policy before escalating the first repair action to a second one of the repair actions.

15. The one or more computer-readable storage devices according to claim 14, the first state reflecting an error message generated by the process.

16. The one or more computer-readable storage devices according to claim 14, the observable interactions being accessed from a recovery log associated with the process.

17. The one or more computer-readable storage devices according to claim 14, the model being computed based on a first action cost associated with the first repair action and a second action cost associated with the second repair action.

18. The one or more computer-readable storage devices according to claim 17, the second action cost being higher than the first action cost.