Extraction of problem diagnostic knowledge from test cases

• US 10,248,541 B2
• Filed: 01/25/2017
• Issued: 04/02/2019
• Est. Priority Date: 07/10/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Krishna, Hariharan, Ramakrishnan, Arun, Shetty, Rohit
• Primary Examiner(s): Wang, Philip
• Application Number:
  US15/415,069
• Publication Number:
  US 20170132114A1
• Time in Patent Office: 797 Days
• Field of Search: None
• CPC Class Codes: G06F 11/3612 by runtime analysis performance monitoring G06F11/3466
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

Embodiments of the present invention enable users to extract knowledge from testing scenarios performed during application development, and later employ that knowledge to interpret application usage scenarios to enhance serviceability of applications by expediting identification and solving of problems. In an exemplary embodiment, log data generated during simulation of test cases is analyzed to create one or more rules based on patterns in which one or more log entries appear in the log data. Later, log data may be analyzed to look for a pattern of log entries that matches a pattern-based rule, thereby facilitating faster identification and resolution of the problem.

Claims

1. A method comprising:

identifying a first set of log entries generated during a first execution of an application code in a first scenario, wherein each log entry in the first set of log entries corresponds to a set of lines of code in the application code;

creating a first rule based, at least in part, on a first order of the first set of log entries, wherein the first order is based on a first chronological order;

identifying a second set of log entries generated during a second execution of the application code in a second scenario; and

determining the second set of log entries partially satisfies the first rule;

wherein;

at least determining the second set of log entries partially satisfies the first rule is performed by computer software running on computer hardware.

2. The method of claim 1, wherein determining the second set of log entries partially satisfies the first rule further includes:

determining, based on the first chronological order, a first log entry in the first set of log entries appears earlier in the first order than does a second log entry in the first set of log entries;

determining a third log entry in the second set of log entries matches the first log entry;

determining a fourth log entry in the second set of log entries matches the second log entry;

determining a second order of the second set of log entries based on a second chronological order; and

determining, based on the second chronological order, the third log entry appears earlier in the second order than does the fourth log entry.

3. The method of claim 2, wherein determining the second set of log entries partially satisfies the first rule further includes:

determining a third content of the third log entry matches a first content of the first log entry; and

determining a fourth content of the fourth log entry matches a second content of the second log entry.

4. The method of claim 1, further comprising:

responsive to determining the second set of log entries partially satisfies the first rule, providing, to a user, information describing the first scenario.

5. The method of claim 1, wherein:

the first scenario is an application development testing scenario; and

the second scenario is an end-user application usage scenario.

6. The method of claim 1, further comprising:

generating a first set of updated lines of code in the application code; and

identifying a third set of log entries generated during a third execution of the application code in a third scenario, wherein each log entry in the third set of log entries corresponds to the first set of updated lines of code.

7. The method of claim 6, further comprising:

creating a second rule based, at least in part, on a third order of the third set of log entries, wherein the third order is based on a third chronological order.

8. A computer program product comprising:

a computer readable storage medium having stored thereon;

first instructions executable by a device to cause the device to identify a first set of log entries generated during a first execution of an application code in a first scenario, wherein each log entry in the first set of log entries corresponds to a set of lines of code in the application code;

second instructions executable by a device to cause the device to create a first rule based, at least in part, on a first order of the first set of log entries, wherein the first order is based on a first chronological order;

third instructions executable by a device to cause the device to identify a second set of log entries generated during a second execution of the application code in a second scenario; and

fourth instructions executable by a device to cause the device to determine the second set of log entries partially satisfies the first rule.

9. The computer program product of claim 8, wherein fourth instructions to determine the second set of log entries partially satisfies the first rule further include:

fifth instructions executable by a device to cause the device to determine, based on the first chronological order, a first log entry in the first set of log entries appears earlier in the first order than does a second log entry in the first set of log entries;

sixth instructions executable by a device to cause the device to determine a third log entry in the second set of log entries matches the first log entry;

seventh instructions executable by a device to cause the device to determine a fourth log entry in the second set of log entries matches the second log entry;

eighth instructions executable by a device to cause the device to determine a second order of the second set of log entries based on a second chronological order; and

ninth instructions executable by a device to cause the device to determine, based on the second chronological order, the third log entry appears earlier in the second order than does the fourth log entry.

10. The computer program product of claim 9, wherein fourth instructions to determine the second set of log entries partially satisfies the first rule further include:

tenth instructions executable by a device to cause the device to determine a third content of the third log entry matches a first content of the first log entry; and

eleventh instructions executable by a device to cause the device to determine a fourth content of the fourth log entry matches a second content of the second log entry.

11. The computer program product of claim 8, further comprising:

responsive to fourth instructions to determine the second set of log entries partially satisfies the first rule, fifth instructions executable by a device to cause the device to provide, to a user, information describing the first scenario.

12. The computer program product of claim 8, wherein:

the first scenario is an application development testing scenario; and

the second scenario is an end-user application usage scenario.

13. The computer program product of claim 8, further comprising:

fifth instructions executable by a device to cause the device to generate a first set of updated lines of code in the application code; and

sixth instructions executable by a device to cause the device to identify a third set of log entries generated during a third execution of the application code in a third scenario, wherein each log entry in the third set of log entries corresponds to the first set of updated lines of code.

14. The computer program product of claim 13, further comprising:

seventh instructions executable by a device to cause the device to create a second rule based, at least in part, on a third order of the third set of log entries, wherein the third order is based on a third chronological order.

15. A computer system comprising:

a processor set; and

a computer readable storage medium;

wherein;

the processor set is structured, located, connected, and/or programmed to execute instructions stored on the computer readable storage medium; and

the instructions include;

first instructions executable by a device to cause the device to identify a first set of log entries generated during a first execution of an application code in a first scenario, wherein each log entry in the first set of log entries corresponds to a set of lines of code in the application code;

second instructions executable by a device to cause the device to create a first rule based, at least in part, on a first order of the first set of log entries, wherein the first order is based on a first chronological order;

third instructions executable by a device to cause the device to identify a second set of log entries generated during a second execution of the application code in a second scenario; and

fourth instructions executable by a device to cause the device to determine the second set of log entries partially satisfies the first rule.

16. The computer system of claim 15, wherein fourth instructions to determine the second set of log entries partially satisfies the first rule further include:

fifth instructions executable by a device to cause the device to determine, based on the first chronological order, a first log entry in the first set of log entries appears earlier in the first order than does a second log entry in the first set of log entries;

sixth instructions executable by a device to cause the device to determine a third log entry in the second set of log entries matches the first log entry;

seventh instructions executable by a device to cause the device to determine a fourth log entry in the second set of log entries matches the second log entry;

eighth instructions executable by a device to cause the device to determine a second order of the second set of log entries based on a second chronological order; and

ninth instructions executable by a device to cause the device to determine, based on the second chronological order, the third log entry appears earlier in the second order than does the fourth log entry.

17. The computer system of claim 16, wherein fourth instructions to determine the second set of log entries partially satisfies the first rule further include:

tenth instructions executable by a device to cause the device to determine a third content of the third log entry matches a first content of the first log entry; and

eleventh instructions executable by a device to cause the device to determine a fourth content of the fourth log entry matches a second content of the second log entry.

18. The computer system of claim 15, further comprising:

responsive to fourth instructions to determine the second set of log entries partially satisfies the first rule, fifth instructions executable by a device to cause the device to provide, to a user, information describing the first scenario.

19. The computer system of claim 15, wherein:

the first scenario is an application development testing scenario; and

the second scenario is an end-user application usage scenario.

20. The computer system of claim 15, further comprising:

fifth instructions executable by a device to cause the device to generate a first set of updated lines of code in the application code; and

sixth instructions executable by a device to cause the device to identify a third set of log entries generated during a third execution of the application code in a third scenario, wherein each log entry in the third set of log entries corresponds to the first set of updated lines of code.