Systems and methods for testing a software application

• US 9,111,030 B1
• Filed: 10/05/2009
• Issued: 08/18/2015
• Est. Priority Date: 10/03/2008
• Status: Active Grant

• Current Assignee: Andrei Iancu, Ashley Moody Florida Office of The Attorney General and Successors Thereto, Jeanette Nunez Lieutenant Governor of Florida and Successors Thereto, Joan D. Heuer, Laurel M Lee Florida Secretary of State and Successors Thereto, Steven Mnuchin United States Secretary of The Treasury and Successors Thereto, Timothy E Gribben Commissioner For Bureau of The Fiscal Service Agency of The United States Department of The Treasury and Successors and Assigns Thereto
• Original Assignee: Federal Home Loan Mortgage Corporation
• Inventors: Dunn, Philip, Liu, Yi
• Primary Examiner(s): Zhen, Li B
• Assistant Examiner(s): BODDEN, EVRAL E
• Application Number:
  US12/573,680
• Time in Patent Office: 2,143 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results

Abstract

Systems, methods, and computer-readable media are disclosed for testing a software application. An exemplary method includes storing a control file identifying a test case for testing a software application. A first expect result may be extracted from a device storing expected results of the software application, the first expected result being identified by the control file. A first actual result may be extracted from a device storing actual results output by the software application, the first actual result being identified by the control file. The first expected result may be compared with the first actual result to determine whether the first actual result matches the first expected result. A result file indicated whether the test case passed or failed s generated, and the test case has passed when the first actual result matches the first expected result. the result file may be stored in a storage device.

Claims

1. A computer-implemented method, performed by a processor connected to a communication network and at least one database, for testing a software application, the method comprising:

storing, in the at least one database using the processor, a control file identifying a test case for testing a software application;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing expected results of the software application, a first expected result, the first expected result being of a first format and identified by the control file;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing actual results output by the software application, a first actual result, the first actual result being of a second format and identified by the control file;

determining, using the processor, whether the first format and the second format are identical;

translating, using the processor and if the first and second formats are not identical, either the format of the first expected results to the format or the first actual result or the format of the first actual result to the format of the first expected result;

comparing, using the processor the first expected result with the first actual result to determine whether the first actual result matches the first expected result;

generating, using the processor a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result;

determining using the processor, whether the generated result file indicates that the test case passed;

determining, using the processor, whether the first expected result and

the first actual result contains no data when it is determined that the generated result file indicates that the test case passed;

updating, using the processor through the communication network, the result file to indicate that the test case failed if the first expected result and the first actual result contains no data,

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database;

storing, using the processor through the communication network, a configuration file defining a variable to have a value;

identifying, using the processor, an instance of the variable in the control file; and

interpreting, using the processor, the instance of the variable in the control file to have the value defined in the configuration file,

wherein the configuration file includes a structured query language (“

SQL”

) statement for extracting the first expected result; and

the SQL statement includes the instance of the variable.

2. The method according to claim 1, wherein the control file is a text file.

3. The method according to claim 1, wherein the result file has a name reflecting whether the test case passed or failed.

4. The method according to claim 3, wherein the name of the result file includes the string “

PASS”

when the test case passed, and the name of the result file includes the string “

FAIL”

when the test case failed.

5. The method according to claim 1, wherein the result file has a name reflecting that the test case resulted in a warning.

6. The method according to claim 1, further comprising:

storing a test case list file mapping the test case to a description of a verification point or requirement; and

generating a report including the test case, whether the test case passed or failed, and the description of the verification point or requirement.

7. The method according to claim 6, wherein the test case list file identifies a plurality of test cases including the test case identified by the control file, the method further comprising:

generating a report identifying at least one unexecuted test case from the plurality of test cases identified by the test case list file.

8. The method according to claim 1, further comprising:

generating a report indicating a number of passed test cases and a number of failed test cases for the software application.

9. The method according to claim 1, wherein the software application outputs the actual results as a spreadsheet, and the first actual result appears within a first cell of the spreadsheet.

10. The method according to claim 9, further comprising generating a spreadsheet report comprising cells, wherein the first actual result and first expected result are displayed together in a first cell of the spreadsheet report.

11. The method according to claim 10, wherein the location of the first cell of the spreadsheet report is based on the location of the first cell of the spreadsheet output by the software application.

12. The method according to claim 10, further comprising displaying an indication of whether the test case passed or failed in the first cell with the first actual result and the first expected result.

13. The method according to claim 1, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.

14. The method according to claim 1, wherein the control file identifies a plurality of test cases, and a single result file is generated to indicate the results of the plurality of test cases.

15. The method according to claim 14, wherein the plurality of test cases are identified in the control file by a naming convention that includes a character indicating a range of test cases.

16. The method according to claim 15, wherein the character is a dash “

−

”

character.

17. The method according to claim 16, wherein the naming convention uses a plus “

+”

character to append a single test case to the range indicated by the dash “

−

”

character.

18. A non-transitory computer-readable medium comprising instructions for causing a processor connected to a communication network and at least one database to execute a method for testing a software application, the method comprising:

storing in the at least one database using the processor, a control file identifying a test case for testing a software application;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing expected results of the software application, a first expected result, the first expected result being of a first format and being identified by the control file;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing actual results output by the software application, a first actual result, the first actual result being of a second format and identified by the control file;

determining, using the processor, whether the first format and the second format are identical;

translating, using the processor and if the first and second formats are not identical, either the format of the first expected results to the format or the first actual result or the format of the first actual result to the format of the first expected result;

comparing using the processor the first expected result with the first actual result to determine whether the first actual result matches the first expected result;

generating using the processor a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result;

determining using the processor, whether the generated result file indicates that the test case passed;

determining, using the processor, whether the first expected result and the first actual result contains no data when it is determined that the, generated result file indicates that the test case passed;

updating, using the processor through the communication network, the result file to indicate that the test case failed if the first of the first expected result and the first actual result contains no data;

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database;

storing, using the processor through the communication network, a configuration file defining a variable to have a value;

identifying, using the processor, an instance of the variable in the control file; and

interrupting, using the processor, the instance of the variable in the control file to have the value defined in the configuration file,

wherein the configuration file includes a structured query language (“

SQL”

) statement for extracting the first expected result; and

the SQL statement includes the instance of the variable.

19. A system for testing a software application, the system comprising:

A processor connected to a communication network and at least one database; and

at least one memory comprising instructions executable by a processor connected to a communication network and at least one database to;

store in the at least one database using the processor, a control file identifying a test case for testing a software application;

extract, using the processor through the communication network from a device connected to the at least one database and configured for storing expected results of the software application, a first expected result, the first expected result being of a first format and being identified by the control file;

extract, using the processor through the communication network from a device connected to the at least one database and configured for storing actual results output by the software application, a first actual result, the first actual result being of a second format and identified by the control file;

determining, using the processor, whether the first format and the second format are identical;

translating, using the processor and if the first and second formats are not identical, either the format of the first expected results to the format or the first actual result or the format of the first actual result to the format of the first expected result;

compare using the processor the first expected result with the first actual result to determine whether the first actual result matches the first expected result;

generate using the processor a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result;

determining using the processor, whether the generated result file indicates that the test case passed;

determining, using the processor, whether the first expected result and the first actual result contains no data when it is determined that the, generated result file indicates that the test case passed;

updating, using the processor through the communication network, the result file to indicate that the test case failed if the first of the first expected result and the first actual result contains no data,

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database;

store, using the processor through the communication network, a configuration file defining a variable to have a value;

identify, using the processor, an instance of the variable in the control file; and

interpret, using the processor, the instance of the variable in the control file to have the value defined in the configuration file,

wherein the configuration file includes a structured query language (“

SQL”

) statement for extracting the first expected result; and

the SQL statement includes the instance of the variable.