Extraction of problem diagnostic knowledge from test cases

• US 9,348,739 B2
• Filed: 07/10/2014
• Issued: 05/24/2016
• Est. Priority Date: 07/10/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Krishna, Hariharan, Ramakrishnan, Arun, Shetty, Rohit
• Primary Examiner(s): WANG, RONGFA PHILIP
• Application Number:
  US14/327,983
• Publication Number:
  US 20160011960A1
• Time in Patent Office: 684 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3612 by runtime analysis performance monitoring G06F11/3466
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

Embodiments of the present invention enable users to extract knowledge from testing scenarios performed during application development, and later employ that knowledge to interpret application usage scenarios to enhance serviceability of applications by expediting identification and solving of problems. In an exemplary embodiment, log data generated during simulation of test cases is analyzed to create one or more rules based on patterns in which one or more log entries appear in the log data. Later, log data may be analyzed to look for a pattern of log entries that matches a pattern-based rule, thereby facilitating faster identification and resolution of the problem.

Claims

1. A method comprising:

receiving, by one or more computer processors, a first set of log data generated during execution of application code in a first scenario, wherein the first set of log data includes one or more log entries and a set of run-time data;

processing, by one or more computer processors, the first set of log data to identify the one or more log entries appearing in the first set of log data, wherein the one or more log entries are information pertaining to the application code in the first scenario;

creating, by one or more computer processors, a first rule based on a first pattern of the identified one or more log entries appearing in the first set of log data, wherein the first pattern includes a content of a plurality of message identifiers that appear in the first set of log data and a chronological order in which the plurality of message identifiers appear;

associating, by one or more computer processors, the first rule with information describing the first scenario; and

storing, by one or more computer processors, the first rule and the information describing the first scenario in a database.

2. The method of claim 1, further comprising:

receiving, by one or more computer processors, a second set of log data generated during execution of application code in a second scenario;

processing, by one or more computer processors, the second set of log data to identify one or more log entries appearing in the second set of log data;

determining, by one or more computer processors, whether the identified one or more log entries appearing in the second set of log data satisfy the first rule; and

responsive to determining that the identified one or more log entries appearing in the second set of log data satisfy the first rule, providing to a user, by one or more computer processors, the information describing the first scenario.

3. The method of claim 2, further comprising:

receiving, by one or more computer processors, information describing the second scenario;

associating, by one or more computer processors, the information describing the second scenario with the first rule and the information describing the first scenario; and

storing, by one or more computer processors, the information describing the second scenario in the database.

4. The method of claim 2, wherein the first scenario is an application development testing scenario, and the second scenario is an end-user application usage scenario.

5. The method of claim 2, wherein the step of processing, by one or more computer processors, the first set of log data to identify the one or more log entries comprises:

processing, by one or more computer processors, the first set of log data to identify a message identifier for each of the one or more log entries in the first set of log data.

6. The method of claim 5, wherein the step of determining, by one or more computer processors, whether the identified one or more log entries appearing in the second set of log data satisfy the first rule comprises:

determining, by one or more computer processors, whether message identifiers appearing in the second set of log data match content and a chronological order of the at least two message identifiers appearing in the first set of log data.

7. The method of claim 2, further comprising:

updating, by one or more computer processors, one or more lines of the application code such that upon execution of the application code, a third set of log data is generated;

receiving, by one or more computer processors, a third set of log data generated during execution of application code in a third scenario; and

creating, by one or more computer processors, a second rule based on a second pattern of one or more log entries appearing in the third set of log data.

8. A computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to receive a first set of log data generated during execution of application code in a first scenario, wherein the first set of log data includes one or more log entries and a set of run-time data;

program instructions to process the first set of log data to identify the one or more log entries appearing in the first set of log data, wherein the one or more log entries are information pertaining to the application code in the first scenario;

program instructions to create a first rule based on a first pattern of the identified one or more log entries appearing in the first set of log data, wherein the first pattern includes a content of a plurality of message identifiers that appear in the first set of log data and a chronological order in which the plurality of message identifiers appear;

program instructions to associate the first rule with information describing the first scenario; and

program instructions to store the first rule and the information describing the first scenario in a database.

9. The computer program product of claim 8, wherein the program instructions stored on the one or more computer readable storage media further comprise:

program instructions to receive a second set of log data generated during execution of application code in a second scenario;

program instructions to process the second set of log data to identify one or more log entries appearing in the second set of log data;

program instructions to determine whether the identified one or more log entries appearing in the second set of log data satisfy the first rule; and

program instructions to, responsive to determining that the identified one or more log entries appearing in the second set of log data satisfy the first rule, provide to a user the information describing the first scenario.

10. The computer program product of claim 9, wherein the program instructions stored on the one or more computer readable storage media further comprise:

program instructions to receive information describing the second scenario;

program instructions to associate the information describing the second scenario with the first rule and the information describing the first scenario; and

program instructions to store the information describing the second scenario in the database.

11. The computer program product of claim 9, wherein the first scenario is an application development testing scenario, and the second scenario is an end-user application usage scenario.

12. The computer program product of claim 9, wherein the program instructions to process the first set of log data to identify the one or more log entries comprise:

program instructions to process the first set of log data to identify a message identifier for each of the one or more log entries in the first set of log data.

13. The computer program product of claim 12, wherein the program instructions to determine whether the identified one or more log entries appearing in the second set of log data satisfy the first rule comprise:

program instructions to determine whether message identifiers appearing in the second set of log data match content and a chronological order of the at least two message identifiers appearing in the first set of log data.

14. The computer program product of claim 9, wherein the program instructions stored on the one or more computer readable storage media further comprise:

program instructions to update one or more lines of the application code such that upon execution of the application code, a third set of log data is generated;

program instructions to receive a third set of log data generated during execution of application code in a third scenario; and

program instructions to create a second rule based on a second pattern of one or more log entries appearing in the third set of log data.

15. A computer system comprising:

one or more computer processors;

one or more computer readable storage media;

program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to receive a first set of log data generated during execution of application code in a first scenario, wherein the first set of log data includes one or more log entries and a set of run-time data;

program instructions to process the first set of log data to identify the one or more log entries appearing in the first set of log data, wherein the one or more log entries are information pertaining to the application code in the first scenario;

program instructions to create a first rule based on a first pattern of the identified one or more log entries appearing in the first set of log data, wherein the first pattern includes a content of a plurality of message identifiers that appear in the first set of log data and a chronological order in which the plurality of message identifiers appear;

program instructions to associate the first rule with information describing the first scenario; and

program instructions to store the first rule and the information describing the first scenario in a database.

16. The computer system of claim 15, wherein the program instructions stored on the computer readable storage media for execution by at least one of the one or more processors further comprise:

program instructions to receive a second set of log data generated during execution of application code in a second scenario;

program instructions to process the second set of log data to identify one or more log entries appearing in the second set of log data;

program instructions to determine whether the identified one or more log entries appearing in the second set of log data satisfy the first rule; and

program instructions to, responsive to determining that the identified one or more log entries appearing in the second set of log data satisfy the first rule, provide to a user the information describing the first scenario.

17. The computer system of claim 16, wherein the program instructions to determine whether the identified one or more log entries appearing in the second set of log data satisfy the first rule comprise:

program instructions to determine whether message identifiers appearing in the second set of log data match content and a chronological order of at least two message identifiers appearing in the first set of log data.

18. The computer system of claim 16, wherein the program instructions stored on the computer readable storage media for execution by at least one of the one or more processors further comprise:

program instructions to update one or more lines of the application code such that upon execution of the application code, a third set of log data is generated;

program instructions to receive a third set of log data generated during execution of application code in a third scenario; and

program instructions to create a second rule based on a second pattern of one or more log entries appearing in the third set of log data.

19. The computer system of claim 16, wherein the program instructions stored on the computer readable storage media for execution by at least one of the one or more processors further comprise:

receiving, by one or more computer processors, information describing the second scenario;

associating, by one or more computer processors, the information describing the second scenario with the first rule and the information describing the first scenario; and

storing, by one or more computer processors, the information describing the second scenario in the database.

20. The computer system of claim 16, wherein the first scenario is an application development testing scenario, and the second scenario is an end-user application usage scenario.