Test failure bucketing
First Claim
1. A computer-implemented method for bucketing failure messages generated as a result of a test performed on a target application, the computer-implemented method comprising computer-implemented operations for:
- receiving a failure message that is associated with a failure;
in response to receiving the failure message, searching a stack trace of a test application for a blame stack trace frame from which the failure message originated;
if the blame stack trace frame is found, finding a most similar match bucket from a plurality of buckets, each of the plurality of buckets being associated with the blame stack trace frame;
if the blame stack trace frame is not found, finding the most similar match bucket from the plurality of buckets, the plurality of buckets comprising a set of buckets each comprising failures and associated failure data that do not have an associated blame stack trace frame;
determining if the most similar match bucket at least meets a similarity threshold;
if it is determined that the most similar match bucket at least meets the similarity threshold, adding the failure message and associated failure data to the most similar match bucket; and
if it is determined that the most similar match bucket does not at least meet the similarity threshold, creating a new bucket and adding the failure message and the associated failure data to the new bucket.
2 Assignments
0 Petitions
Accused Products
Abstract
Failure messages generated as a result of tests performed on a target application are bucketed in an effort to correlate related failure messages with minimal or no human interaction. An exemplary method for bucketing failure messages includes receiving a failure message, searching a stack trace of a target application for a blame stack trace frame from which the failure message originated, finding a most similar match bucket from a plurality of buckets, determining if the most similar match bucket at least meets a similarity threshold, adding the failure message to the most similar match bucket, if it is determined that the most similar match bucket at least meets the similarity threshold, and creating a new bucket and adding the failure message to the new bucket, if it is determined that the most similar match bucket does not at least meet the similarity threshold.
38 Citations
17 Claims
-
1. A computer-implemented method for bucketing failure messages generated as a result of a test performed on a target application, the computer-implemented method comprising computer-implemented operations for:
-
receiving a failure message that is associated with a failure; in response to receiving the failure message, searching a stack trace of a test application for a blame stack trace frame from which the failure message originated; if the blame stack trace frame is found, finding a most similar match bucket from a plurality of buckets, each of the plurality of buckets being associated with the blame stack trace frame; if the blame stack trace frame is not found, finding the most similar match bucket from the plurality of buckets, the plurality of buckets comprising a set of buckets each comprising failures and associated failure data that do not have an associated blame stack trace frame; determining if the most similar match bucket at least meets a similarity threshold; if it is determined that the most similar match bucket at least meets the similarity threshold, adding the failure message and associated failure data to the most similar match bucket; and if it is determined that the most similar match bucket does not at least meet the similarity threshold, creating a new bucket and adding the failure message and the associated failure data to the new bucket. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A computer storage medium having computer readable instructions stored thereupon that, when executed by a computer, cause the computer to:
-
receive a failure message that is associated with a failure; in response to receiving the failure message, search a stack trace of a test application for a blame stack trace frame from which the failure message originated; if the blame stack trace frame is found, find a most similar match bucket from a plurality of buckets, each of the plurality of buckets being associated with the blame stack trace frame; if the blame stack trace frame is not found, find the most similar match bucket from the plurality of buckets, the plurality of buckets comprising a set of buckets each comprising failures and associated failure data that do not have an associated blame stack trace frame; determine if the most similar match bucket at least meets a similarity threshold; if it is determined that the most similar match bucket at least meets the similarity threshold, add the failure message and associated failure data to the most similar match bucket; and if it is determined that the most similar match bucket does not at least meet the similarity threshold, create a new bucket and adding the failure message and the associated failure data to the new bucket. - View Dependent Claims (8, 9, 10, 11, 12)
-
-
13. A computer-implemented method for managing failure messages, the computer-implemented method comprising computer-implemented operations for:
-
receiving a failure message comprising a text string; logging the failure message; pre-processing failure data associated with the failure message by finding a matching template for the failure message, determining if the failure message has stack trace data associated therewith, if the failure message has stack trace data associated therewith, inserting the trace data into a database and calculating a blame stack trace frame for a stack trace identified in the stack trace data, and if the failure message does not have the stack trace data associated therewith, inserting dummy stack trace data into the database, and using a dummy stack trace frame as the blame stack trace frame; bucketing the failure message and the failure data into a bucket; performing failure tagging for the failure message; determining if the failure message matches a failure tag identified via the failure tagging; if the failure message matches the failure tag, applying the failure tag to the failure message and instructing a reporting client computer to perform operations based upon the applied failure tag; and if the failure message does not match the failure tag, instructing the reporting client computer to perform operations based upon the bucket to which the failure message was applied. - View Dependent Claims (14, 15, 16, 17)
-
Specification