Automatically populating symptom databases for software applications

• US 8,327,191 B2
• Filed: 10/02/2008
• Issued: 12/04/2012
• Est. Priority Date: 10/19/2007
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Branca, Salvatore, Molinari, Angela, Turano, Edoardo
• Primary Examiner(s): Wilson, Yolanda L
• Application Number:
  US12/244,256
• Publication Number:
  US 20090106597A1
• Time in Patent Office: 1,524 Days
• Field of Search: 714/38, 714/26, 714/38.1, 714/41
• US Class Current: 714/38.1
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

A solution is proposed for populating a symptom collection—such as a symptom database—for a software application. The symptom collection includes an indication of a set of symptoms; each symptom consists of an undesired behavior of the software application, which symptom is due to each one of a set of corresponding problems of the software application (resulting from a misuse thereof). The symptom database also includes an indication of an explanation of each problem. A corresponding method involves performing a test of the software application for verifying a correctness of the software application. The symptom collection is then populated according to a result of the test.

Claims

1. A method, in a data processing system, for populating a symptom collection for a software application, the method comprises:

performing a test of the software application for verifying a correctness of the software application, wherein;

the test includes a plurality of test cases,

a subset of error test cases in the plurality of test cases verifies the correctness of the software application in non-allowable scenarios, wherein the non-allowable scenarios are scenarios indicative of end-user misuse,

execution of an error test case in the subset of error test cases purposely induces a problem in the software application, and

responsive to an expected outcome in response to executing the error test case, identifying a symptom indicative of an undesired behavior and a test action of the error test case that caused the problem; and

populating the symptom collection according to a result of the error test case with the symptom and the test action that caused the problem.

2. The method of claim 1, wherein the symptom collection further includes a correction of the problem.

3. The method of claim 1, wherein performing the test of the software application for verifying the correctness of the software application comprises:

defining the plurality of test cases each one including a set of test actions and the expected outcome,

executing the test actions of each test case,

detecting an actual outcome of the test actions of each test case, and

setting each test case as passed or failed according to a comparison between the corresponding actual outcome and the expected outcome,

and wherein populating the symptom collection comprises;

determining a new symptom according to the actual outcome of each one of a set of selected test cases.

4. The method of claim 3, wherein defining the plurality of test cases comprises:

flagging at least one test case as an error test case, each error test case being intended to verify the correctness of the software application with the problem, and wherein populating the symptom collection comprises;

selecting each passed error test case for the population of the symptom collection, and

determining a new explanation of the new symptom for each passed error test case according to the test actions of the passed error test case.

5. The method of claim 4, wherein defining the plurality of test cases comprises:

flagging at least one test action of each error test case inducing the corresponding problem as a problem test action,

and wherein determining the new explanation of the new symptom for each passed error test case comprises;

setting the new explanation according to the at least one problem test action of the passed error test case.

6. The method of claim 5, wherein defining the plurality of test cases comprises:

specifying at least one correction action for correcting the problem being induced by the at least one problem test action of each error test case,

and wherein determining the new explanation of the new symptom for each passed error test case comprises;

setting a new cause to an indication of the at least one problem test action of the passed error test case, and

setting a new correction to an indication of the at least one correction action of the passed error test case.

7. The method of claim 3, wherein performing the test of the software application for verifying the correctness of the software application comprises:

flagging each failed test case as wrong when the failure is due to a misuse of the software application, and

determining at least one wrong action causing a corresponding problem of each wrong failed test case,

and wherein populating the symptom collection comprises;

selecting each wrong failed test case for the population of the symptom collection, and

determining a new explanation of the new symptom for each wrong failed test case according to the at least one wrong action.

8. The method of claim 7, wherein performing the test of the software application for verifying the correctness of the software application comprises:

specifying at least one correction action for correcting the problem resulting from the at least one wrong action of each wrong failed test case,

and wherein determining the new explanation of the new symptom for each wrong failed test case comprises;

setting a new cause to an indication of the at least one wrong action of the wrong failed test case, and

setting a new correction to an indication of the at least one correction action of the wrong failed test case.

9. The method of claim 1, wherein the scenarios indicative of end-user misuse is at least one of a software application being installed in an improper way, the software application being configured wrongly, or he software application being invoked with unacceptable commands.

10. A computer program product for performing a method for populating a symptom collection for a software application comprising a non-transitory computer-usable medium embodying a computer readable program, wherein the computer readable program, when executed on a computing device, causes the computing device to:

perform a test of the software application for verifying a correctness of the software application, wherein;

the test includes a plurality of test cases,

a subset of error test cases in the plurality of test cases verifies the correctness of the software application in non-allowable scenarios, wherein the non-allowable scenarios are scenarios indicative of end-user misuse,

execution of an error test case in the subset of error test cases purposely induces a problem in the software application, and

responsive to an expected outcome in response to executing the error test case, identifying a symptom indicative of an undesired behavior and a test action of the error test case that caused the problem; and

populate the symptom collection according to a result of the error test case with the symptom and the test action that caused the problem.

11. The computer program product of claim 10, wherein the symptom collection further includes a correction of the problem.

12. The computer program product of claim 10, wherein the computer readable program to perform the test of the software application for verifying the correctness of the software application further causes the computing device to:

define the plurality of test cases each one including a set of test actions and the expected outcome,

execute the test actions of each test case,

detect an actual outcome of the test actions of each test case, and

set each test case as passed or failed according to a comparison between the corresponding actual outcome and the expected outcome,

and wherein the computer readable program to populate the symptom collection further causes the computing device to;

determine a new symptom according to the actual outcome of each one of a set of selected test cases.

13. The computer program product of claim 12, wherein the computer readable program to define the plurality of test cases further causes the computing device to:

flag at least one test case as an error test case, each error test case being intended to verify the correctness of the software application with the problem,

and wherein the computer readable program to populate the symptom collection further causes the computing device to;

select each passed error test case for the population of the symptom collection, and

determine a new explanation of the new symptom for each passed error test case according to the test actions of the passed error test case.

14. The computer program product of claim 13, wherein the computer readable program to define the plurality of test cases further causes the computing device to:

flag at least one test action of each error test case inducing the corresponding problem as a problem test action,

and wherein the computer readable program to determining the new explanation of the new symptom for each passed error test case further causes the computing device to;

set the new explanation according to the at least one problem test action of the passed error test case.

15. The computer program product of claim 14, wherein the computer readable program to define the plurality of test cases further causes the computing device to:

specify at least one correction action for correcting the problem being induced by the at least one problem test action of each error test case,

and wherein the computer readable program to determine the new explanation of the new symptom for each passed error test case further causes the computing device to;

set a new cause to an indication of the at least one problem test action of the passed error test case, and

set a new correction to an indication of the at least one correction action of the passed error test case.

16. The computer program product of claim 12, wherein the computer readable program to perform the test of the software application for verifying the correctness of the software application further causes the computing device to:

flag each failed test case as wrong when the failure is due to a misuse of the software application, and

determine at least one wrong action causing a corresponding problem of each wrong failed test case,

and wherein the computer readable program to populate the symptom collection further causes the computing device to;

select each wrong failed test case for the population of the symptom collection, and

determine a new explanation of the new symptom for each wrong failed test case according to the at least one wrong action.

17. The computer program product of claim 16, wherein the computer readable program to perform the test of the software application for verifying the correctness of the software application further causes the computing device to:

specify at least one correction action for correcting the problem resulting from the at least one wrong action of each wrong failed test case,

and wherein the computer readable program to determine the new explanation of the new symptom for each wrong failed test case further causes the computing device to;

set a new cause to an indication of the at least one wrong action of the wrong failed test case, and

set a new correction to an indication of the at least one correction action of the wrong failed test case.

18. The computer program product of claim 10, wherein the scenarios indicative of end-user misuse is at least one of a software application being installed in an improper way, the software application being configured wrongly, or he software application being invoked with unacceptable commands.

19. A system for populating a symptom collection for a software application, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises a computer program which configures the processor to;

perform a test of the software application for verifying a correctness of the software application, wherein;

the test includes a plurality of test cases,

a subset of error test cases in the plurality of test cases verifies the correctness of the software application in non-allowable scenarios, wherein the non-allowable scenarios are scenarios indicative of end-user misuse,

execution of an error test case in the subset of error test cases purposely induces a problem in the software application, and

responsive to an expected outcome in response to executing the error test case, identifying a symptom indicative of an undesired behavior and a test action of the error test case that caused the problem; and

populate the symptom collection according to a result of the error test case with the symptom and the test action that caused the problem.

20. The system of claim 19, wherein the symptom collection further includes a correction of the problem.

21. The system of claim 19, wherein the instructions to perform the test of the software application for verifying the correctness of the software application further causes the processor to:

define the plurality of test cases each one including a set of test actions and the expected outcome,

execute the test actions of each test case,

detect an actual outcome of the test actions of each test case, and

set each test case as passed or failed according to a comparison between the corresponding actual outcome and the expected outcome,

and wherein the instructions to populate the symptom collection further causes the processor to;

determine a new symptom according to the actual outcome of each one of a set of selected test cases.

22. The system of claim 19, wherein the scenarios indicative of end-user misuse is at least one of a software application being installed in an improper way, the software application being configured wrongly, or he software application being invoked with unacceptable commands.

23. A tool in a memory for populating a symptom collection for a software application, wherein the tool includes:

means for populating the symptom collection according to a result of a subset of error test cases with a symptom and a test action that caused a problem to occur in the software application being performed for a test for verifying a correctness of the software application, wherein;

the test includes a plurality of test cases,

the subset of error test cases in the plurality of test cases verifies the correctness of the software application in non-allowable scenarios, wherein the non-allowable scenarios are scenarios indicative of end-user misuse,

execution of an error test case in the subset of error test cases purposely induces the problem in the software application, and

responsive to an expected outcome in response to executing the error test case, identifying the symptom indicative of an undesired behavior and the test action of the test case that caused the problem.

24. The tool of claim 23, wherein the scenarios indicative of end-user misuse is at least one of a software application being installed in an improper way, the software application being configured wrongly, or he software application being invoked with unacceptable commands.