Systems and methods for testing a software application

• US 9,600,405 B1
• Filed: 07/13/2016
• Issued: 03/21/2017
• Est. Priority Date: 10/03/2008
• Status: Active Grant

• Current Assignee: Federal Home Loan Mortgage Corporation
• Original Assignee: Federal Home Loan Mortgage Corporation
• Inventors: Dunn, Philip, Liu, Yi
• Primary Examiner(s): Bodden, Evral E
• Application Number:
  US15/209,551
• Time in Patent Office: 251 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results

Abstract

Systems, methods, and computer-readable media are disclosed for testing a software application. An exemplary method includes storing a control file identifying a test case for testing a software application. A first expected result may be extracted from a device storing expected results of the software application, the first expected result being identified by the control file. A first actual result may be extracted from a device storing actual results output by the software application, the first actual result being identified by the control file. The first expected result may be compared with the first actual result to determine whether the first actual result matches the first expected result. A result file indicating whether the test case passed or failed is generated, and the test case has passed when the first actual result matches the first expected result. The result file may be stored in a storage device.

Claims

1. A computer-implemented method, performed by a processor connected to a network interface and a networked database, for testing a software application, the method comprising:

extracting, using the processor through the network interface from the networked database, a first expected result of a software application, the first expected result being identified by a control file indicating an expected result verification point of a test case for testing the software application and being generated by SQL query including logic for emulating the expected functionality of the software application to generate known good results;

extracting, using the processor through the network interface from the networked database, a first actual result output by the software application, the first actual result being identified by the control file indicating an actual result verification point of the test case for testing the software application;

comparing, using the processor, the first expected result with the first actual result and the expected result verification point with the actual result verification point to determine whether the first actual result matches the first expected result and whether the expected result verification point matches the actual result verification point;

generating, using the processor, a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result and the expected result verification point matches the actual result verification point;

determining, using the processor, if the generated result file indicates that the test case passed, whether the first expected result and the first actual result contains the same amount of data;

determining, using the processor, if the generated result file indicates that the test case passed, whether the test case ID of the expected result verification point matches the test case ID of the actual result verification point;

updating, using the processor through the network interface, the result file to indicate that the test case failed if the first expected result and the first actual result do not contain the same amount of data; and

updating, using the processor through the network interface, the result file to indicate that the test case passed if the test case ID of the expected result verification point matches the test case ID of the actual result verification point.

2. The method according to claim 1, wherein the control file identifies a plurality of test cases, and a single result file is generated to indicate the results of the plurality of test cases.

3. The method according to claim 1, wherein the control file includes an expected data identifier comprising at least one statement for extracting the first expected result from an expected results database.

4. The method according to claim 1, wherein the control file includes an actual data identifier comprising at least one statement for extracting the first actual result.

5. The method according to claim 1, further comprising:

storing a test case list file identifying a plurality of test cases including the test case identified by the control file and mapping the test case to a description of a test case verification point or requirement; and

generating a report including the test case, whether the test case passed or failed, and the description of the test case verification point or requirement.

6. The method according to claim 1, further comprising:

storing, using the processor through the network interface, the result file in a storage device connected to the networked database.

7. The method according to claim 1, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.

8. A non-transitory computer-readable medium comprising instructions for causing a processor connected to a network interface and a networked database to execute a method for testing a software application, the method comprising:

extracting, using the processor through the network interface from the networked database, a first expected result of a software application, the first expected result being identified by a control file indicating an expected result verification point of a test case for testing the software application and being generated by SQL query including logic for emulating the expected functionality of the software application to generate known good results;

extracting, using the processor through the network interface from the networked database, a first actual result output by the software application, the first actual result being identified by the control file indicating an actual result verification point of the test case for testing the software application;

comparing, using the processor, the first expected result with the first actual result and the expected result verification point with the actual result verification point to determine whether the first actual result matches the first expected result and whether the expected result verification point matches the actual result verification point;

generating, using the processor, a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result and the expected result verification point matches the actual result verification point;

determining, using the processor, if the generated result file indicates that the test case passed, whether the first expected result and the first actual result contains the same amount of data;

determining, using the processor, if the generated result file indicates that the test case passed, whether the test case ID of the expected result verification point matches the test case ID of the actual result verification point;

updating, using the processor through the network interface, the result file to indicate that the test case failed if the first expected result and the first actual result do not contain the same amount of data; and

updating, using the processor through the network interface, the result file to indicate that the test case passed if the test case ID of the expected result verification point matches the test case ID of the actual result verification point.

9. The non-transitory computer-readable medium according to claim 8, wherein the control file identifies a plurality of test cases, and a single result file is generated to indicate the results of the plurality of test cases.

10. The non-transitory computer-readable medium according to claim 8, wherein the control file includes an expected data identifier comprising at least one statement for extracting the first expected result from an expected results database.

11. The non-transitory computer-readable medium according to claim 8, wherein the control file includes an actual data identifier comprising at least one statement for extracting the first actual result.

12. The non-transitory computer-readable medium according to claim 8, the method further comprising:

storing a test case list file identifying a plurality of test cases including the test case identified by the control file and mapping the test case to a description of a test case verification point or requirement; and

generating a report including the test case, whether the test case passed or failed, and the description of the test case verification point or requirement.

13. The non-transitory computer-readable medium according to claim 8, the method further comprising:

storing, using the processor through the network interface, the result file in a storage device connected to the networked database.

14. The non-transitory computer-readable medium according to claim 8, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.

15. A system for testing a software application, the system comprising:

a processor having access to a networked database;

a network interface in communication with the processor; and

a memory device in communication with the processor and configured to store instructions,

wherein, when the processor executes the instructions, the processor;

extracts, through the network interface from the networked database, a first expected result of a software application, the first expected result being identified by a control file indicating an expected result verification point of a test case for testing the software application and being generated by SQL query including logic for emulating the expected functionality of the software application to generate known good results;

extracts, through the network interface from the networked database, a first actual result output by the software application, the first actual result being identified by the control file indicating an actual result verification point of the test case for testing the software application;

compares the first expected result with the first actual result and the expected result verification point with the actual result verification point to determine whether the first actual result matches the first expected result and whether the expected result verification point matches the actual result verification point;

generates a result file indicating whether the test case passed or failed, the test case having passed when the first actual result matches the first expected result and the expected result verification point matches the actual result verification point;

determines, if the generated result file indicates that the test case passed, whether the first expected result and the first actual result contains the same amount of data;

determines, if the generated result file indicates that the test case passed, whether the test case ID of the expected result verification point matches the test case ID of the actual result verification point;

updating, using the processor through the network interface, the result file to indicate that the test case failed if the first expected result and the first actual result do not contain the same amount of data; and

updating, using the processor through the network interface, the result file to indicate that the test case passed if the test case ID of the expected result verification point matches the test case ID of the actual result verification point.

16. The system according to claim 15, wherein the control file identifies a plurality of test cases, and a single result file is generated to indicate the results of the plurality of test cases.

17. The system according to claim 15, wherein the control file includes an expected data identifier comprising at least one statement for extracting the first expected result from an expected results database.

18. The system according to claim 15, wherein the control file includes an actual data identifier comprising at least one statement for extracting the first actual result.

19. The system according to claim 15, wherein the processor further executes the instructions to:

store a test case list file identifying a plurality of test cases including the test case identified by the control file and mapping the test case to a description of a test case verification point or requirement; and

generate a report including the test case, whether the test case passed or failed, and the description of the test case verification point or requirement.

20. The system according to claim 15, wherein the processor further executes the instructions to:

store, through the network interface, the result file in a storage device connected to the networked database.

21. The system according to claim 15, wherein the control file specifies a tolerance, and comparing the first expected result with the first actual result comprises determining whether the first actual result is within the specified tolerance of the first expected result.