Regression testing of an application that uses big data as a source of data

• US 10,642,722 B2
• Filed: 01/09/2018
• Issued: 05/05/2020
• Est. Priority Date: 01/09/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Li, Jeff J., Michelini, Vanessa V., Wang, Fang, Xu, Jia
• Primary Examiner(s): Zhen, Wei Y
• Assistant Examiner(s): Huda, Mohammed N
• Application Number:
  US15/866,086
• Publication Number:
  US 20190213117A1
• Time in Patent Office: 847 Days
• Field of Search: 717124
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

An application performs a regression test by the application processing a use case. Actual output data generated by the application processing the use case can be compared to expected output data and whether the actual output data matches the expected output data can be determined. Responsive to determining that the actual output data does not match the expected output data, the actual output data can be compared to refreshed source data and whether the actual output data matches the refreshed source data can be determined. Responsive to determining that the actual output data matches the refreshed source data, an automated remedy action can be automatically selected, and the expected output data can be updated with the refreshed source data by implementing the automated remedy action.

Claims

1. A method, comprising:

an application performing a regression test by the application processing a use case;

comparing actual output data generated by the application processing the use case to expected output data and determining whether the actual output data matches the expected output data;

responsive to the determining that the actual output data does not match the expected output data, comparing the actual output data to refreshed source data and determining whether the actual output data matches the refreshed source data; and

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting at least a first automated remedy action, and updating, using a processor, the expected output data with the refreshed source data by implementing the first automated remedy action, wherein the first automated remedy action is a new data remedy action that defines actions to be performed to update the expected output data for the use case in response to an actual output data record of the actual output data not having a matching data record in the expected output data for the use case.

2. The method of claim 1, further comprising:

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting a second automated remedy action, and updating the expected output data with the refreshed source data by implementing the second automated remedy action, wherein the second automated remedy action is a field change remedy action that defines actions to be performed to update the expected output data for the use case in response to a value of a data field in the actual output data for the use case being different than a value of a matching data field in the expected output data.

3. The method of claim 1, further comprising:

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting a second automated remedy action, and updating the expected output data with the refreshed source data by implementing the second automated remedy action, wherein the second automated remedy action is a missing data remedy action that defines actions to be performed to update the expected output data for the use case in response to an expected output data record of the expected output data not having a matching data record in the actual output data for the use case.

4. The method of claim 1, further comprising:

automatically generating regression test output capture code based, at least in part, on the expected output data, the regression test output capture code configured to perform;

creating a first data table and storing the expected output data to the first data table;

creating a second data table and storing the actual output data to the second data table; and

performing the comparing the actual output data to the expected output data;

wherein the regression test output capture code is invoked by the application during the regression test.

5. The method of claim 4, further comprising:

analyzing the use case and the expected output data;

wherein automatically generating regression test output capture code is based on the analyzing the use case and the expected output data.

6. The method of claim 4, further comprising:

instrumenting the application to invoke the regression test output capture code during the regression testing.

7. A system, comprising:

a processor programmed to initiate executable operations comprising;

an application performing a regression test by the application processing a use case;

comparing actual output data generated by the application processing the use case to expected output data and determining whether the actual output data matches the expected output data;

responsive to the determining that the actual output data does not match the expected output data, comparing the actual output data to refreshed source data and determining whether the actual output data matches the refreshed source data; and

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting at least a first automated remedy action, and updating the expected output data with the refreshed source data by implementing the first automated remedy action, wherein the first automated remedy action is a new data remedy action that defines actions to be performed to update the expected output data for the use case in response to an actual output data record of the actual output data not having a matching data record in the expected output data for the use case.

8. The system of claim 7, the executable operations further comprising:

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting a second automated remedy action, and updating the expected output data with the refreshed source data by implementing the second automated remedy action, wherein the second automated remedy action is a field change remedy action that defines actions to be performed to update the expected output data for the use case in response to a value of a data field in the actual output data for the use case being different than a value of a matching data field in the expected output data.

9. The system of claim 7, the executable operations further comprising:

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting a second automated remedy action, and updating the expected output data with the refreshed source data by implementing the second automated remedy action, wherein the second automated remedy action is a missing data remedy action that defines actions to be performed to update the expected output data for the use case in response to an expected output data record of the expected output data not having a matching data record in the actual output data for the use case.

10. The system of claim 7, the executable operations further comprising:

automatically generating regression test output capture code based, at least in part, on the expected output data, the regression test output capture code configured to perform;

creating a first data table and storing the expected output data to the first data table;

creating a second data table and storing the actual output data to the second data table; and

performing the comparing the actual output data to the expected output data;

wherein the regression test output capture code is invoked by the application during the regression test.

11. The system of claim 10, the executable operations further comprising:

analyzing the use case and the expected output data;

wherein automatically generating regression test output capture code is based on the analyzing the use case and the expected output data.

12. The system of claim 10, the executable operations further comprising:

instrumenting the application to invoke the regression test output capture code during the regression testing.

13. A computer program product, comprising:

a computer readable storage medium having program code stored thereon, the program code executable by a data processing system to initiate operations including;

an application performing a regression test by the application processing a use case;

comparing actual output data generated by the application processing the use case to expected output data and determining whether the actual output data matches the expected output data;

responsive to the determining that the actual output data does not match the expected output data, comparing the actual output data to refreshed source data and determining whether the actual output data matches the refreshed source data; and

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting at least a first automated remedy action, and updating the expected output data with the refreshed source data by implementing the first automated remedy action, wherein the first automated remedy action is a new data remedy action that defines actions to be performed to update the expected output data for the use case in response to an actual output data record of the actual output data not having a matching data record in the expected output data for the use case.

14. The computer program product of claim 13, wherein the program code is executable by the data processing system to initiate operations further comprising:

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting a second automated remedy action, and updating the expected output data with the refreshed source data by implementing the second automated remedy action, wherein the second automated remedy action is a field change remedy action that defines actions to be performed to update the expected output data for the use case in response to a value of a data field in the actual output data for the use case being different than a value of a matching data field in the expected output data.

15. The computer program product of claim 13, wherein the program code is executable by the data processing system to initiate operations further comprising:

responsive to the determining that the actual output data matches the refreshed source data, automatically selecting a second automated remedy action, and updating the expected output data with the refreshed source data by implementing the second automated remedy action, wherein the second automated remedy action is a missing data remedy action that defines actions to be performed to update the expected output data for the use case in response to an expected output data record of the expected output data not having a matching data record in the actual output data for the use case.

16. The computer program product of claim 13, wherein the program code is executable by the data processing system to initiate operations further comprising:

automatically generating regression test output capture code based, at least in part, on the expected output data, the regression test output capture code configured to perform;

creating a first data table and storing the expected output data to the first data table;

creating a second data table and storing the actual output data to the second data table; and

performing the comparing the actual output data to the expected output data;

wherein the regression test output capture code is invoked by the application during the regression test.

17. The computer program product of claim 16, wherein the program code is executable by the data processing system to initiate operations further comprising:

analyzing the use case and the expected output data;

wherein automatically generating regression test output capture code is based on the analyzing the use case and the expected output data.