Method and apparatus for managing software errors in a computer system

• US 7,702,966 B2
• Filed: 09/07/2005
• Issued: 04/20/2010
• Est. Priority Date: 09/07/2005
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Mukherjee, Udayan, Hiremath, Chetan, Chandwani, Neelam, Dodeja, Rakesh
• Primary Examiner(s): Baderman; Scott T
• Assistant Examiner(s): KO, CHAE M
• Application Number:
  US11/220,796
• Publication Number:
  US 20070055914A1
• Time in Patent Office: 1,686 Days
• Field of Search: 714/38, 714/47
• US Class Current: 714/47.1
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0715 in a system implementing multitasking multitasking per se G06F9/46
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40

Abstract

A method for managing a system includes monitoring a plurality of applications running in the system for errors. A prediction is made as to whether errors detected would result in a failure. Fault recovery is initiated in response to a failure prediction. According to one aspect of the present invention, monitoring the plurality of applications includes reading error recorders associated with error occurrence. Other embodiments are described and claimed.

Claims

1. A method for managing a system, comprising:

monitoring a plurality of software applications running in the system for software errors;

predicting whether software errors detected would result in a failure of one of the plurality of software applications; and

initiating fault recovery in response to a failure prediction by performing audits to check communication links with other applications with which the one of the plurality of software applications is interfacing if a predicted failure is due to errors in interprocess communication mechanisms, and performing one of restarting the one of the plurality of software applications, and initiating failover of the one of the plurality of software applications prior to its failure to change a condition of the system.

2. The method of claim 1, wherein monitoring the plurality of software applications comprises reading error recorders associated with error occurrence.

3. The method of claim 1, wherein monitoring the plurality of software applications comprises reading log files associated with the applications for error occurrence.

4. The method of claim 1, wherein predicting whether the software errors detected would result in a failure comprises determining whether a number of software errors of a particular type reaches a threshold.

5. The method of claim 4, wherein the number of software errors is decremented periodically to age fault information over time.

6. The method of claim 1, wherein predicting whether the software errors detected would result in a failure comprises determining whether a number of software errors of a particular type reaches a threshold within a time window.

7. The method of claim 6, further comprising adjusting the time window to include a smaller period of time if error rate increases.

8. The method of claim 6, further comprising adjusting the time window to include a larger period of time if error rate decreases.

9. The method of claim 1, further comprising:

determining whether an application is alive after an expiration of a watch dog timer; and

performing recovery after an expiration of a recovery timer.

10. The method of claim 1, wherein monitoring the plurality of applications comprises providing an application programming interface (API) to allow applications to log errors detected.

11. The method of claim 1, further comprising learning appropriate values of heuristics parameters for an error.

23. The method of claim 1, wherein initiating fault recovery further comprises initiating a diagnostic of the one of the plurality of software applications prior to its failure.

24. The method of claim 1, wherein initiating fault recovery further comprises initiating a diagnostic of an operating system prior to its failure.

25. The method of claim 1, wherein initiating fault recovery further comprises saving data from the one of the plurality of software applications prior to its failure.

12. An article of manufacture comprising a machined accessible medium including sequences of instructions, the sequences of instructions including instructions which when executed cause the machine to perform:

monitoring a plurality of software applications running in the system for software errors;

predicting whether software errors detected would result in a failure of one of the plurality of software applications; and

initiating fault recovery in response to a failure prediction prior to a failure of the one of the software applications by performing audits to check communication links with other applications with which the one of the plurality of software applications is interfacing if a predicted failure is due to errors in interprocess communication mechanisms, and performing one of restarting the one of the plurality of software applications, and initiating failover of the one of the plurality of software applications.

13. The article of manufacture of claim 12, wherein monitoring the plurality of software applications comprises reading recorders associated with error occurrence.

14. The article of manufacture of claim 12, wherein predicting whether the software errors detected would result in a failure comprises determining whether a number of software errors of a particular type reaches a threshold.

15. The article of manufacture of claim 12, wherein the number of software errors is decremented periodically to age fault information over time.

16. The article of manufacture of claim 12, wherein predicting whether the software errors detected would result in a failure comprises determining whether a number of software errors of a particular type reaches a threshold within a time window.

17. A computer system, comprising:

a bus;

a memory;

a processor;

and a fault prediction module that includes a fault detection unit to monitor a plurality of software applications running in the system for software errors, a failure prediction unit to predict whether software errors detected will result in a failure in one of the plurality of software applications, and a fault recovery unit to initiate fault recovery in response to a failure prediction to change a condition of one of the computer system and the one of the plurality of software applications prior to a failure of the one of the plurality of software application by performing audits to check communication links with other applications with which the one of the plurality of software applications is interfacing if a predicted failure is due to errors in interprocess communication mechanisms, and performing one of restarting the one of the plurality of software applications, and initiating failover of the one of the plurality of software applications.

18. The computer system of claim 17, wherein the fault prediction module is implemented on an isolated partition on the processor.

19. The computer system of claim 17, wherein the fault prediction module is implemented as a set of processor native instructions designed and optimized for performing failure prediction.

20. The computer system of claim 17, wherein the processor executes a plurality of operating systems and applications.

21. The computer system of claim 17, wherein the fault prediction module is implemented on a dedicated microcontroller in the computer system.

22. The computer system of claim 17, wherein the fault prediction module comprises a plurality of error recorders.