Automated system problem diagnosing

• US 8,112,667 B2
• Filed: 01/25/2010
• Issued: 02/07/2012
• Est. Priority Date: 01/25/2010
• Status: Active Grant

• Current Assignee: Globalfoundries US Incorporated (GlobalFoundries, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Belluomini, Wendy Ann, Gill, Binny Sher, Yan, Xifeng, Zhou, Pin
• Primary Examiner(s): Baderman, Scott
• Assistant Examiner(s): TRUONG, LOAN
• Application Number:
  US12/693,373
• Publication Number:
  US 20110185233A1
• Time in Patent Office: 743 Days
• Field of Search: 714/27, 714/37, 707/603, 707/758
• US Class Current: 714/27
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence

Abstract

Embodiments of the invention relate to automated system problem diagnosing. An index is created with problem description information of previously diagnosed problems, a diagnosis for each problem, and a solution to each diagnosis. System states, traces and logs are extracted from a source system with a new problem. The problem diagnosis system generates problem description information of the new problem from the system states, traces and logs. Problem description information of the new problem is compared with problem description information in the problem description index. A search score is computed for each document in the problem description index. The search score is a measure of similarity between each document in the index and the description of the new problem. A matching score is assigned to each previously diagnosed problems based on the search score. The matching score is a measure of similarity between the new problem and each previously diagnosed problem. The system determines a diagnosis and solution of the new problem based on a diagnosis and solution of one of the previously diagnosed problems.

Claims

1. A method for automatically diagnosing a system problem, comprising:

creating a problem description, wherein a problem description index is created from a group consisting of;

a line-wise index comprising a document entry in the problem description index for each line in a problem description information of a plurality of previously diagnosed problems,

a description-wise index comprising a document entry in the problem description index for the problem description information of each of a plurality of previously diagnosed problems, and

a set-wise index comprising a document entry in the problem description index for each set of the problem description information of a plurality of previously diagnosed problems grouped together based on a problem cause;

receiving the problem description index and problem description information of a new problem, the problem description index comprising problem description information of previously diagnosed problems, wherein said problem description information of previously diagnosed problems and of said new problem comprises text content describing system events that have occurred;

comparing problem description information of the new problem with problem description information in the problem description index, wherein comparing the problem description information of the new problem with problem description information in the problem description index for a line-wise index comprises a line-wise search of the line-wise index, wherein said line-wise search comprises searching each line of text content in the problem description index for each line of text content from the problem description information of the new problem;

computing a search score for each document in the problem description index, wherein the search score is a measure of similarity between each document in the problem description index and the problem description information of the new problem;

assigning a matching score to each of the previously diagnosed problems based on the search score, wherein the matching score is a measure of similarity between the new problem and each of the previously diagnosed problems; and

determining a diagnosis of the new problem, wherein the diagnosis of the new problem is a diagnosis of at least one of the previously diagnosed problems.

2. The method of claim 1, further comprising:

extracting problem description information of the new problem when the new problem occurs, wherein problem description information of the new problem is extracted from the group consisting of;

at least one recorded log,

at least one recorded trace, and

at least one recorded system state.

3. The method of claim 1, wherein assigning a matching score comprises processing the search score based on a similarity definition for approximation.

4. The method of claim 1, wherein assigning a matching score comprises processing the search score based on a similarity definition for noise reduction.

5. The method of claim 1, further comprising:

determining a solution to the new problem from the diagnosis based on a solution of at least one of the previously diagnosed problems.

6. The method of claim 1, further comprising:

updating the problem description index, wherein updating comprises adding the problem description information of the new problem and the diagnosis of the new problem to the problem description index.

7. The method of claim 3, wherein processing for approximation comprises normalizing a term frequency-inverse document frequency score based on a frequency of the problem description information of the new problem being included in problem description information of previously diagnosed problem sets.

8. The method of claim 3, wherein processing for approximation comprises only considering a predetermined number of documents in the problem description index with highest search scores.

9. The method of claim 3, wherein processing for approximation comprises approximating a matching score of each problem set with a predetermined number of problems in the problem description index with highest matching scores.

10. The method of claim 4, wherein processing for noise reduction comprises omitting a timestamp, a sequence number identifier, and duplicate lines from the problem description information of the previously diagnosed problems in the problem description index and the problem description information of the new problem.

11. The method of claim 4, wherein processing for noise reduction comprises normalizing a matching score between a line in the problem description information of a new problem and problem description information of each previously diagnosed problem based on a number of similar lines in problem description information of the new problem.

12. The method of claim 4, wherein processing for noise reduction comprises omitting the matching score between a line in the problem description information of a new problem and problem description information of each previously diagnosed problem when the matching score is lower than a predetermined score.

13. The method of claim 1, wherein comparing the problem description information of the new problem with problem description information in the problem description index for a description-wise index comprises a description-wise search of the description-wise index, wherein said description-wise search comprises searching the problem description index for each term of text content from the problem description information of the new problem.

14. The method of claim 1, wherein comparing the problem description information of the new problem with problem description information in the problem description index for a set-wise index comprises a description-wise search of the set-wise index, wherein said set-wise search comprises searching each set of problem description information in the problem description index for each term of text content from the problem description information of the new problem.

15. A non-transitory computer-useable storage medium for automatically diagnosing a system problem, said medium having a computer-readable program, wherein a program upon being processed on a computer causes the computer to implement the steps of:

creating a problem description index, wherein the problem description index is created from a group consisting of;

a line-wise index comprising a document entry in the problem description index for each line in a problem description information of a plurality of previously diagnosed problems,

a description-wise index comprising a document entry in the problem description index for the problem description information of each of a plurality of previously diagnosed problems, and

a set-wise index comprising a document entry in the problem description index for each set of the problem description information of a plurality of previously diagnosed problems grouped together based on a problem cause;

receiving the problem description index and problem description information of a new problem, the problem description index comprising problem description information of previously diagnosed problems, wherein said problem description information of previously diagnosed problems and of said new problem comprises text content describing system events that have occurred;

comparing problem description information of the new problem with problem description information in the problem description index, wherein comparing the problem description information of the new problem with problem description information in the problem description index for a line-wise index comprises a line-wise search of the line-wise index, wherein said line-wise search comprises searching each line of text content in the problem description index for each line of text content from the problem description information of the new problem;

computing a search score for each document in the problem description index, wherein the search score is a measure of similarity between each document in the problem description index and the problem description information of the new problem;

assigning a matching score to each of the previously diagnosed problems based on the search score, wherein the matching score is a measure of similarity between the new problem and each of the previously diagnosed problems; and

determining a diagnosis of the new problem, wherein the diagnosis of the new problem is a diagnosis of at least one of the previously diagnosed problems.

16. The computer-useable storage medium of claim 15, wherein assigning a matching score comprises processing the search score based on a similarity definition for approximation.

17. The computer-useable storage medium of claim 15, wherein assigning a matching score comprises processing the search score based on a similarity definition for noise reduction.

18. The computer-useable storage medium of claim 15, wherein comparing the problem description information of the new problem with problem description information in the problem description index for a description-wise index comprises a description-wise search of the description-wise index, wherein said description-wise search comprises searching the problem description index for each term of text content from the problem description information of the new problem.

19. The computer-useable storage medium of claim 15, wherein comparing the problem description information of the new problem with problem description information in the problem description index for a set-wise index comprises a description-wise search of the set-wise index, wherein said set-wise search comprises searching each set of problem description information in the problem description index for each term of text content from the problem description information of the new problem.

20. The computer-useable storage medium of claim 15, further comprising:

extracting problem description information of the new problem when the new problem occurs, wherein problem description information of the new problem is extracted from the group consisting of;

at least one recorded log,

at least one recorded trace, and

at least one recorded system state.

21. The computer-useable storage medium of claim 15, wherein assigning a matching score comprises processing the search score based on a similarity definition for approximation.

22. The computer-useable storage medium of claim 15, wherein assigning a matching score comprises processing the search score based on a similarity definition for noise reduction.

23. The computer-useable storage medium of claim 15, further comprising:

determining a solution to the new problem from the diagnosis based on a solution of at least one of the previously diagnosed problems.

24. The computer-useable storage medium of claim 15, further comprising:

updating the problem description index, wherein updating comprises adding the problem description information of the new problem and the diagnosis of the new problem to the problem description index.

25. The computer-useable storage medium of claim 21, wherein processing for approximation comprises normalizing a term frequency-inverse document frequency score based on a frequency of the problem description information of the new problem being included in problem description information of previously diagnosed problem sets.

26. The computer-useable storage medium of claim 21, wherein processing for approximation comprises only considering a predetermined number of documents in the problem description index with highest search scores.

27. The computer-useable storage medium of claim 21, wherein processing for approximation comprises approximating a matching score of each problem set with a predetermined number of problems in the problem description index with highest matching scores.

28. The computer-useable storage medium of claim 22, wherein processing for noise reduction comprises omitting a timestamp, a sequence number identifier, and duplicate lines from the problem description information of the previously diagnosed problems in the problem description index and the problem description information of the new problem.

29. The computer-useable storage medium of claim 22, wherein processing for noise reduction comprises normalizing a matching score between a line in the problem description information of a new problem and problem description information of each previously diagnosed problem based on a number of similar lines in problem description information of the new problem.

30. The computer-useable storage medium of claim 22, wherein processing for noise reduction comprises omitting the matching score between a line in the problem description information of a new problem and problem description information of each previously diagnosed problem when the matching score is lower than a predetermined score.