Test framework for computing jobs

• US 9,298,596 B2
• Filed: 07/09/2013
• Issued: 03/29/2016
• Est. Priority Date: 07/09/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Caufield, Brian K., Sood, Ajay, Vizor, Julian J.
• Primary Examiner(s): Zhen, Wei
• Assistant Examiner(s): Chen, Zhan
• Application Number:
  US13/937,602
• Publication Number:
  US 20150020049A1
• Time in Patent Office: 994 Days
• Field of Search: 717100-178
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

According to one embodiment of the present invention, a system tests jobs in a computing environment. The system creates a test case for one or more existing executable jobs without modifying the job design or recompiling the executable itself, wherein the test case includes one or more capture points in a job flow of the executable jobs and corresponding rules for capturing data, identification of data for testing the one or more executable jobs, and rules for comparing the captured data to expected results. The system captures the data at the one or more capture points in the job flow in accordance with the test case and generates a baseline of expected results. The system executes the test case to test the one or more executable jobs by capturing data at the one or more capture points generated from execution of the one or more executable jobs and compares result data from the one or more capture points to the baseline of expected results in accordance with the comparison rules to determine a result of the test. Embodiments of the present invention further include a method and computer program product for testing jobs in a computing environment in substantially the same manners described above.

Claims

1. A computer-implemented method of testing jobs in a computing environment comprising:

creating a test case for one or more existing executable Extract, Transform, and Load (ETL) jobs comprising ETL job metadata by storing a definition of the test case as an element of the ETL job metadata without recompiling the ETL job and without introducing steps for debugging the ETL job, wherein the ETL job metadata comprises a specification of an ETL job design including operations performed by the ETL job, links between the operations and runtime parameters, wherein the test case includes one or more capture points in a job flow of the one or more executable jobs and corresponding rules for capturing data, identification of data for testing the one or more executable jobs, and rules for comparing the captured data to expected results, and wherein the capture points include at least one intermediate point of the job flow;

capturing the data at the one or more capture points in the job flow in accordance with the test case and generating a baseline of expected results; and

executing the test case to test the one or more executable jobs by capturing data at the one or more capture points generated from execution of the one or more executable jobs and comparing result data from the one or more capture points to the baseline of expected results in accordance with the comparison rules to determine a result of the test.

2. The computer-implemented method of claim 1, wherein executing the test case includes:

comparing the result data to the baseline of expected results to identify a location of a first detected difference and indicate a first point of failure for the tested one or more executable jobs.

3. The computer-implemented method of claim 1, further comprising:

exporting the test case and corresponding one or more executable jobs to a new computing environment and executing the test case to test the one or more executable jobs in the new computing environment.

4. The computer-implemented method of claim 1, wherein the one or more executable jobs are tested utilizing regression testing.

5. The computer-implemented method of claim 1, wherein the one or more executable jobs remain unchanged for the testing.

6. The computer-implemented method of claim 1, wherein the test case further includes:

a name of the test case, a configuration file, the generated baseline, and operational metadata associated with generating the baseline;

capture and compare settings including the one or more capture points and the corresponding rules for capturing data, one or more comparison points in the job flow for comparison of data, and the rules for comparing the captured data;

the identification of data for testing the one or more executable jobs including a location of source data; and

a generated sequence used to execute the test case, wherein the generated sequence comprises a job executable and a comparison operator.

7. The computer-implemented method of claim 1, wherein at least one of the one or more capture points is chosen by selecting a link in a diagram of the job flow.

8. The computer-implemented method of claim 1, wherein the data is captured by accessing the data at the one or more capture points via an automated debugger session.

9. The computer-implemented method of claim 1, wherein base operators of the one or more executable jobs capture the data at the one or more capture points according to the rules for capturing the data.

10. A system for testing jobs in a computing environment comprising:

at least one processor configured to;

create a test case for one or more existing executable Extract, Transform, and Load (ETL) jobs comprising ETL job metadata by storing a definition of the test case as an element of the ETL job metadata without recompiling the ETL job and without introducing steps for debugging the ETL job, wherein the ETL job metadata comprises a specification of an ETL job design including operations performed by the ETL job, links between the operations and runtime parameters, wherein the test case includes one or more capture points in a job flow of the one or more executable jobs and corresponding rules for capturing data, identification of data for testing the one or more executable jobs, and rules for comparing the captured data to expected results, and wherein the capture points include at least one intermediate point of the job flow;

capture the data at the one or more capture points in the job flow in accordance with the test case and generate a baseline of expected results; and

execute the test case to test the one or more executable jobs by capturing data at the one or more capture points generated from execution of the one or more executable jobs and compare result data from the one or more capture points to the baseline of expected results in accordance with the comparison rules to determine a result of the test.

11. The system of claim 10, wherein executing the test case includes:

comparing the result data to the baseline of expected results to identify a location of a first detected difference and indicate a first point of failure for the tested one or more executable jobs.

12. The system of claim 10, wherein the processor is further configured to:

export the test case and corresponding one or more executable jobs to a new computing environment and execute the test case to test the one or more executable jobs in the new computing environment.

13. The system of claim 10, wherein the one or more executable jobs are tested utilizing regression testing.

14. The system of claim 10, wherein the one or more executable jobs remain unchanged for the testing.

15. The system of claim 10, wherein the test case further includes:

a name of the test case, a configuration file, the generated baseline, and operational metadata associated with generating the baseline;

capture and compare settings including the one or more capture points and the corresponding rules for capturing data, one or more comparison points in the job flow for comparison of data, and the rules for comparing the captured data;

the identification of data for testing the one or more executable jobs including a location of source data; and

a generated sequence used to execute the test case, wherein the generated sequence comprises a job executable and a comparison operator.

16. A computer program product for testing jobs in a computing environment comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith for execution on a processing system, the computer readable program code comprising computer readable program code configured to;

create a test case for one or more existing executable Extract;

Transform, and Load (ETL) jobs comprising ETL job metadata by storing a definition of the test case as an element of the ETL job metadata without recompiling the ETL job and without introducing steps for debugging the ETL job, wherein the ETL job metadata comprises a specification of an ETL job design including operations performed by the ETL job, links between the operations and runtime parameters, wherein the test case includes one or more capture points in a job flow of the one or more executable jobs and corresponding rules for capturing data, identification of data for testing the one or more executable jobs, and rules for comparing the captured data to expected results, wherein the capture points include at least one intermediate point of the job flow;

capture the data at the one or more capture points in the job flow in accordance with the test case and generate a baseline of expected results; and

execute the test case to test the one or more executable jobs by capturing data at the one or more capture points generated from execution of the one or more executable jobs and compare result data from the one or more capture points to the baseline of expected results in accordance with the comparison rules to determine a result of the test.

17. The computer program product of claim 16, wherein executing the test case includes:

comparing the result data to the baseline of expected results to identify a location of a first detected difference and indicate a first point of failure for the tested one or more executable jobs.

18. The computer program product of claim 16, further comprising computer readable program code configured to:

export the test case and corresponding one or more executable jobs to a new computing environment and execute the test case to test the one or more executable jobs in the new computing environment.

19. The computer program product of claim 16, wherein the one or more executable jobs remain unchanged for the testing.

20. The computer program product of claim 16, wherein the test case further includes:

a name of the test case, a configuration file, the generated baseline, and operational metadata associated with generating the baseline;

capture and compare settings including the one or more capture points and the corresponding rules for capturing data, one or more comparison points in the job flow for comparison of data, and the rules for comparing the captured data;

the identification of data for testing the one or more executable jobs including a location of source data; and

a generated sequence used to execute the test case, wherein the generated sequence comprises a job executable and a comparison operator.