Systems and methods for testing a software application

• US 9,405,667 B1
• Filed: 07/21/2015
• Issued: 08/02/2016
• Est. Priority Date: 10/03/2008
• Status: Active Grant

• Current Assignee: Federal Home Loan Mortgage Corporation
• Original Assignee: Federal Home Loan Mortgage Corporation
• Inventors: Dunn, Philip, Liu, Yi
• Primary Examiner(s): Bodden, Evral E
• Application Number:
  US14/804,586
• Time in Patent Office: 378 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results

Abstract

Systems, methods, and computer-readable media are disclosed for testing a software application. An exemplary method includes storing a control file identifying a test case for testing a software application. A first expected result may be extracted from a device storing expected results of the software application, the first expected result being identified by the control file. A first actual result may be extracted from a device storing actual results output by the software application, the first actual result being identified by the control file. The first expected result may be compared with the first actual result to determine whether the first actual result matches the first expected result. A result file indicating whether the test case passed or failed is generated, and the test case has passed when the first actual result matches the first expected result. The result file may be stored in a storage device.

Claims

1. A computer-implemented method, performed by a processor connected to a communication network and at least one database, for testing a software application, the method comprising:

storing, in the at least one database using the processor, a control file identifying a test case for testing a software application;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing expected results of the software application, a first expected result, the first expected result being of a first format and identified by the control file;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing actual results output by the software application, a first actual result, the first actual result being of a second format and identified by the control file;

comparing, using the processor, the first expected result with the first actual result to determine whether the first actual result matches the first expected result;

generating, using the processor, a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result;

determining, using the processor, whether the generated result file indicates that the test case passed;

determining, using the processor, whether the first expected result and the first actual result contains the same amount of data when it is determined that the generated result file indicates that the test case passed;

updating, using the processor through the communication network, the result file to indicate that the test case failed if the first expected result and the first actual result do not contain the same amount of data; and

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database.

2. The method according to claim 1, wherein the control file includes an actual data identifier comprising at least one statement for extracting the first actual result.

3. The method according to claim 2, wherein the control file includes an expected data identifier comprising at least one statement for extracting the first expected result from the expected results database.

4. The method according to claim 3, further comprising:

updating, using the processor through the communication network, the result file to indicate that the test case failed if the actual data identifier and the expected data identifier are identical; and

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database.

5. The method according to claim 1, further comprising:

storing a test case list file mapping the test case to a description of a verification point or requirement; and

generating a report including the test case, whether the test case passed or failed, and the description of the verification point or requirement.

6. The method according to claim 5, wherein the test case list file identifies a plurality of test cases including the test case identified by the control file, the method further comprising:

generating a report identifying at least one unexecuted test case from the plurality of test cases identified by the test case list file.

7. The method according to claim 1, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.

8. A non-transitory computer-readable medium comprising instructions for causing a processor connected to a communication network and at least one database to execute a method for testing a software application, the method comprising:

storing, in the at least one database using the processor, a control file identifying a test case for testing a software application;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing expected results of the software application, a first expected result, the first expected result being of a first format and identified by the control file;

extracting, using the processor through the communication network from a device connected to the at least one database and configured for storing actual results output by the software application, a first actual result, the first actual result being of a second format and identified by the control file;

comparing, using the processor, the first expected result with the first actual result to determine whether the first actual result matches the first expected result;

generating, using the processor, a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result;

determining, using the processor, whether the generated result file indicates that the test case passed;

determining, using the processor, whether the first expected result and the first actual result contains the same amount of data when it is determined that the generated result file indicates that the test case passed;

updating, using the processor through the communication network, the result file to indicate that the test case failed if the first expected result and the first actual result do not contain the same amount of data; and

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database.

9. The non-transitory computer-readable medium according to claim 8, wherein the control file includes an actual data identifier comprising at least one statement for extracting the first actual result.

10. The non-transitory computer-readable medium according to claim 9, wherein the control file includes an expected data identifier comprising at least one statement for extracting the first expected result from the expected results database.

11. The non-transitory computer-readable medium according to claim 10, the method further comprising:

updating, using the processor through the communication network, the result file to indicate that the test case failed if the actual data identifier and the expected data identifier are identical; and

storing, using the processor through the communication network, the result file in a storage device connected to the at least one database.

12. The non-transitory computer-readable medium according to claim 8, the method further comprising:

storing a test case list file mapping the test case to a description of a verification point or requirement; and

generating a report including the test case, whether the test case passed or failed, and the description of the verification point or requirement.

13. The non-transitory computer-readable medium according to claim 12, wherein the test case list file identifies a plurality of test cases including the test case identified by the control file, the method further comprising:

generating a report identifying at least one unexecuted test case from the plurality of test cases identified by the test case list file.

14. The non-transitory computer-readable medium according to claim 8, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.

15. A system for testing a software application, the system comprising:

a processor having access to at least one network database;

a network interface in communication with the processor; and

a memory device in communication with the processor and configured to store instructions,

wherein, when the processor executes the instructions, the processor;

stores, in the at least one database, a control file identifying a test case for testing a software application;

extracts, through the network interface from a device connected to the at least one database and configured for storing expected results of the software application, a first expected result, the first expected result being of a first format and identified by the control file;

extracts, through the network interface from a device connected to the at least one database and configured for storing actual results output by the software application, a first actual result, the first actual result being of a second format and identified by the control file;

compares the first expected result with the first actual result to determine whether the first actual result matches the first expected result;

generates a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result;

determines whether the generated result file indicates that the test case passed;

determines whether the first expected result and the first actual result contains the same amount of data when it is determined that the generated result file indicates that the test case passed;

updates, through the network interface, the result file to indicate that the test case failed if the first expected result and the first actual result do not contain the same amount of data; and

stores, through the network interface, the result file in a storage device connected to the at least one database.

16. The system according to claim 15, wherein the control file includes an actual data identifier comprising at least one statement for extracting the first actual result.

17. The system according to claim 16, wherein the control file includes an expected data identifier comprising at least one statement for extracting the first expected result from the expected results database.

18. The system according to claim 17, wherein the processor further executes the instructions to:

update, through the network interface, the result file to indicate that the test case failed if the actual data identifier and the expected data identifier are identical; and

store, through the network interface, the result file in a storage device connected to the at least one database.

19. The system according to claim 15, wherein the processor further executes the instructions to:

store a test case list file mapping the test case to a description of a verification point or requirement; and

generate a report including the test case, whether the test case passed or failed, and the description of the verification point or requirement.

20. The system according to claim 19, wherein the test case list file identifies a plurality of test cases including the test case identified by the control file, and wherein the processor further executes the instructions to:

generate a report identifying at least one unexecuted test case from the plurality of test cases identified by the test case list file.

21. The system according to claim 15, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.