Computer-implemented systems and methods for software application testing

• US 8,087,001 B2
• Filed: 06/29/2007
• Issued: 12/27/2011
• Est. Priority Date: 06/29/2007
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Hoyek, Nabil Mounir, McCain, Christina Charisse, Coleman, Oita Kayonna Cheston
• Primary Examiner(s): Deng, Anna
• Assistant Examiner(s): Wu, Junchun
• Application Number:
  US11/771,271
• Publication Number:
  US 20090007078A1
• Time in Patent Office: 1,642 Days
• Field of Search: 717/124
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Computer-implemented systems and methods for testing one or more software applications. For example, a system and method can be configured to perform a comparison between test data results and benchmark test data in order to determine test result differences between the received test data results and the retrieved benchmark test data. Testing criteria is applied to the determined test result differences in order to identify test deviations from the benchmark test data.

Claims

1. A computer-implemented method for testing software applications, comprising:

receiving, using one or more data processors, test data results generated during a software application test that determines whether the software application performed intended functions;

retrieving, using the one or more data processors, benchmark test data from a data store;

determining, using the one or more data processors, a test data difference by comparing the test data results with the benchmark test data;

identifying, using the one or more data processors, a test deviation by applying a pre-determined performance criteria to the test data difference, wherein the test deviation identifies a degradation of performance of the software application, and wherein the performance criteria includes pre-specified acceptable deviations or unacceptable deviations between the test data results and the benchmark test data;

receiving, using the one or more data processors, subsequent test data results;

generating, using the one or more data processors, a forecast using one or more statistical values associated with the test data results and the subsequent test data results, wherein the forecast corresponds to a future performance of the software application;

using, using the one or more data processors, the forecast to predict when the future performance of the software application will resolve the test deviation, wherein resolving includes satisfying the pre-determined performance criteria; and

providing, using the one or more data processors, the prediction for display on a graphical interface.

2. The computer-implemented method of claim 1, further comprising:

receiving an evaluation method, wherein the evaluation method is used to compare the test data results with the benchmark test data.

3. The computer-implemented method of claim 2, wherein the evaluation method is selected from a plurality of candidate evaluation methods.

4. The computer-implemented method of claim 3, wherein the candidate evaluation methods include an absolute difference evaluation method, a percentage difference evaluation method, or a relative difference evaluation method.

5. The computer-implemented method of claim 1, further comprising:

storing the test data results in a historical data store;

receiving new test data results generated during a new software application test;

retrieving the stored test data results, wherein the test data results are used as new benchmark test data;

determining a new test data difference by comparing the new test data results with the new benchmark test data; and

identifying a new test deviation by applying the pre-determined performance criteria to the new test data difference.

6. The computer-implemented method of claim 1, further comprising:

plotting the forecast corresponding to the future performance of the software application on a graph.

7. The computer-implemented method of claim 1, wherein the test data results are filtered before being compared with the benchmark test data.

8. The computer-implemented method of claim 1, wherein the statistical values include mean, standard deviation, minimum, and maximum values.

9. A computer-implemented system for testing software applications, comprising:

one or more data processors;

one or more computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including;

receiving test data results generated during a software application test that determines whether the software application performed intended functions;

retrieving benchmark test data from a data store;

determining a test data difference by comparing the test data results with the benchmark test data;

identifying a test deviation by applying a pre-determined performance criteria to the test data difference, wherein the test deviation identifies a degradation of performance of the software application, and wherein the performance criteria includes pre-specified acceptable deviations or unacceptable deviations between the test data results and the benchmark test data;

receiving subsequent test data results;

generating a forecast using one or more statistical values associated with the test data results and the subsequent test data results, wherein the forecast corresponds to a future performance of the software application;

using the forecast to predict when the future performance of the software application will resolve the test deviation, wherein resolving includes satisfying the pre-determined performance criteria; and

providing the prediction for display on a graphical interface.

10. A computer-program product for testing software applications, tangibly embodied in a machine-readable non-transitory storage medium, including instructions configured to cause a data processing apparatus to:

receive test data results generated during a software application test that determines whether the software application performed intended functions;

retrieve benchmark test data from a data store;

determine a test data difference by comparing the test data results with the benchmark test data;

identify a test deviation by applying a pre-determined performance criteria to the test data difference, wherein the test deviation identifies a degradation of performance of the software application, and wherein the performance criteria includes pre-specified acceptable deviations or unacceptable deviations between the test data results and the benchmark test data;

receive subsequent test data results;

generate a forecast using one or more statistical values associated with the test data results and the subsequent test data results, wherein the forecast corresponds to a future performance of the software application;

use the forecast to predict when the future performance of the software application will resolve the test deviation, wherein resolving includes satisfying the pre-determined performance criteria; and

provide the prediction for display on a graphical interface.