ADVANCED AND AUTOMATIC ANALYSIS OF RECURRENT TEST FAILURES

• US 20140157036A1
• Filed: 11/30/2012
• Published: 06/05/2014
• Est. Priority Date: 11/30/2012
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Widjaja, Herman, Precious, Andrew, Campbell, Andrew, Walton, Thomas, Swaminathan, Anish, Shankle, Edwin Bruce III, Edmison, Sean, Beaudoin, Jacob
• Application Number:
  US13/689,754
• Publication Number:
  US 20140157036A1
• Time in Patent Office: Days
• US Class Current: 714/2
• CPC Class Codes: G06F 11/006 Identification G06F11/2289 takes precedence
• CPC Class Codes: G06F 11/0706 the processing taking place on a specific hardware platform or in a specific software environment
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

In one embodiment, a test case run analyzer may filter out failure events with known causes from a test report. The test case run analyzer may receive a test report of a test case run of an application process. The test case run analyzer may automatically identify a failure event in the test case run. The test case run analyzer may automatically compare the failure event to a failure pattern set. The test case run analyzer may filter the test report based on the failure pattern set.

Claims

1. A machine-implemented method, comprising:

receiving a test report of a test case run of an application process;

identifying automatically a failure event in the test case run;

comparing automatically the failure event to a failure pattern set; and

filtering the test report based on the failure pattern set.

2. The method of claim 1, further comprising:

identifying automatically a failure context surrounding the failure event.

3. The method of claim 1, further comprising:

converting the test report to a hierarchical format.

4. The method of claim 1, further comprising:

identifying a novel failure pattern based on the failure event.

5. The method of claim 1, further comprising:

alerting an administrator to a novel failure pattern.

6. The method of claim 1, further comprising:

identifying a matching failure pattern with the failure event.

7. The method of claim 1, further comprising:

determining an identified failure cause from a matching failure pattern.

8. The method of claim 1, further comprising:

selecting from an identified failure cause set associated with the failure event using a failure context.

9. The method of claim 1, further comprising:

executing a curative action associated with an identified failure cause.

10. The method of claim 1, further comprising:

removing the failure event from the test report when associated with a matching failure pattern.

11. The method of claim 1, further comprising:

compiling a filtered test report removing the failure event with a matching failure pattern.

12. A tangible machine-readable medium having a set of instructions detailing a method stored thereon that when executed by one or more processors cause the one or more processors to perform the method, the method comprising:

identifying automatically a predecessor failure event in a test case run of an application process;

comparing automatically the predecessor failure event to a failure pattern set; and

filtering a test report of the test case run based on the failure pattern set.

13. The tangible machine-readable medium of claim 12, wherein the method further comprises:

identifying automatically a failure context surrounding the predecessor failure event.

14. The tangible machine-readable medium of claim 12, wherein the method further comprises:

identifying automatically a successor matching failure pattern with a successor failure event of the test report.

15. The tangible machine-readable medium of claim 14, wherein the method further comprises:

connecting the successor failure event to the predecessor failure event if the successor matching failure pattern is connected to a predecessor matching failure pattern.

16. The tangible machine-readable medium of claim 12, wherein the method further comprises:

identifying automatically a predecessor matching failure pattern with the predecessor failure event.

17. The tangible machine-readable medium of claim 12, wherein the method further comprises:

determining an identified failure cause from a predecessor matching failure pattern.

18. The tangible machine-readable medium of claim 12, wherein the method further comprises:

executing a curative action associated with an identified failure cause.

19. A test case run analyzer, comprising:

an input/output device that receives a test report of a test case run of an application process having a failure event and a failure context;

a database interface that connects to a database storing a failure pattern set; and

a processor that automatically identifies the failure event and the failure context, automatically compares the failure event to the failure pattern set, and filters the test report based on the failure pattern set.

20. The test case run analyzer of claim 19, wherein the processor removes the failure event from the test report when associated with a matching failure pattern.