System for transform generation

• US 10,817,503 B2
• Filed: 06/19/2017
• Issued: 10/27/2020
• Est. Priority Date: 12/10/2012
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Studer, Scott, Wholey, III, Joseph Skeffington, Weisman, Amit
• Primary Examiner(s): Uddin, Md I
• Application Number:
  US15/626,713
• Publication Number:
  US 20180067982A1
• Time in Patent Office: 1,226 Days
• Field of Search: 707718, 707752, 707756, 707755
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24534 Query rewriting; Transformation
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

This specification describes technologies relating to generating transforms based on rule sets. In general, one aspect described in this specification can be embodied in methods that include receiving a rule set including execution cases, where at least one execution case in the rule set includes one or more trigger conditions and a specification of an output that is to be generated when the one or more trigger conditions are all satisfied. The methods may further include generating a control structure including a sequence of rows corresponding to one or more execution cases in the rule set. Each row may include a sequence of one or more trigger conditions and information specifying the output for a corresponding execution case. For at least one of the trigger conditions, when the trigger condition is failed, the control structure may direct processing to skip at least one row in the sequence of rows.

Claims

1. A method of encoding a rule set for execution on a data processing system, the method comprising, with at least one computing device:

encoding, using at least one processor, a rule set for transforming data, the rule set defining a sequence of execution cases, wherein each execution case in the sequence of the execution cases is associated with one or more trigger conditions and an output to be produced when the one or more trigger conditions for a corresponding execution case are all satisfied, said encoding comprising;

generating a control structure, using the at least one processor, based at least in part on the rule set, the control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions that identify which of the plurality of trigger conditions to evaluate according to a logical result of evaluating a different one of the plurality of trigger conditions on values of the data being transformed,

wherein the control structure includes a first group of one or more trigger conditions associated with a first execution case, a second group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the first execution case, and a third group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the second execution case, and

wherein the control structure includes at least one logical association between a trigger condition of the first group and a trigger condition of the third group such that the second group of one or more trigger conditions is selectively skipped depending upon values of the data being transformed;

receiving input data; and

checking trigger conditions of the control structure against the input data in a sequence determined using the control structure.

2. The method of claim 1, further including:

storing and/or transmitting data based on an output specified by the control structure.

3. The method of claim 1, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

4. The method of claim 1, wherein the plurality of trigger conditions specified by the control structure consist of unique trigger conditions from the rule set.

5. The method of claim 1, further including:

determining a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

6. The method of claim 1, further including:

updating the execution time for a trigger condition in a list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sorting pointers to trigger conditions in the control structure based on the updated execution time.

7. The method of claim 1, further including:

determining a logical sequence of the plurality of trigger conditions of the control structure based on failure rates for the trigger conditions.

8. The method of claim 1, further including:

updating a failure rate for a trigger condition in a list of unique trigger conditions based on whether the trigger condition is satisfied by a record in the input data; and

determining a logical sequence of the plurality of trigger conditions of the control structure based on the updated failure rate.

9. The method of claim 1, wherein the rule set is specified through a graphical user interface.

10. The method of claim 1, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

11. The method of claim 1, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

12. The method of claim 1, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.

13. A computer system comprising:

at least one processor;

at least one user interface device; and

at least one computer readable medium comprising processor-executable instructions that, when executed, cause the at least one processor to;

encode a rule set for transforming data, the rule set defining a sequence of execution cases, wherein each execution case in the sequence of the execution cases is associated with one or more trigger conditions and an output to be produced when the one or more trigger conditions for a corresponding execution case are all satisfied, said encoding comprising;

generating a control structure, using the at least one processor, based at least in part on the rule set, the control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions that identify which of the plurality of trigger conditions to evaluate according to a logical result of evaluating a different one of the plurality of trigger conditions on values of the data being transformed,

wherein the control structure includes a first group of one or more trigger conditions associated with a first execution case, a second group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the first execution case, and a third group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the second execution case, and

wherein the control structure includes at least one logical association between a trigger condition of the first group and a trigger condition of the third group such that whether trigger conditions of the second group of one or more trigger conditions are evaluated is selectively skipped based depending upon values of the data being transformed;

receive input data; and

check trigger conditions of the control structure against the input data in a sequence determined using the control structure.

14. The computer system of claim 13, wherein the processor-executable instructions further cause the at least one processor to:

store and/or transmit data based on an output specified by the control structure.

15. The computer system of claim 13, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

16. The computer system of claim 13, wherein the plurality of trigger conditions specified by the control structure consist of unique trigger conditions from the rule set.

17. The computer system of claim 13, wherein the processor-executable instructions further cause the at least one processor to:

determining a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

18. The computer system of claim 13, wherein the processor-executable instructions further cause the at least one processor to:

update the execution time for a trigger condition in a list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sort pointers to trigger conditions in the control structure based on the updated execution time.

19. The computer system of claim 13, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

20. The computer system of claim 13, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

21. The computer system of claim 13, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.

22. A computer system comprising:

at least one processor;

means for encoding a rule set for transforming data, the rule set defining a sequence of execution cases, wherein each execution case in the sequence of the execution cases is associated with one or more trigger conditions and an output to be produced when the one or more trigger conditions for a corresponding execution case are all satisfied;

means for generating a control structure based at least in part on the rule set, the control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions that identify which of the plurality of trigger conditions to evaluate according to a logical result of evaluating a different one of the plurality of trigger conditions on values of the data being transformed,

wherein the control structure includes a first group of one or more trigger conditions associated with a first execution case, a second group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the first execution case, and a third group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the second execution case, and

wherein the control structure includes at least one logical association between a trigger condition of the first group and a trigger condition of the third group such that the second group of one or more trigger conditions is selectively skipped depending upon values of the data being transformed;

means for receiving input data; and

means for checking trigger conditions of the control structure against the input data in a sequence determined using the control structure.

23. A method of executing an encoded rule set, the rule set defining a sequence of execution cases, wherein each execution case in the sequence of the execution cases is associated with one or more trigger conditions and an output to be produced when the one or more trigger conditions for a corresponding execution case are all satisfied, the method comprising, with a data processing system:

receiving input data;

checking trigger conditions of a control structure against the input data in a sequence determined using the control structure, wherein the control structure specifies;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions that identify which of the plurality of trigger conditions to evaluate according to a logical result of evaluating a different one of the plurality of trigger conditions on values of the data being transformed,

wherein the control structure includes a first group of one or more trigger conditions associated with a first execution case, a second group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the first execution case, and a third group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the second execution case, and

wherein the control structure includes at least one logical association between a trigger condition of the first group and a trigger condition of the third group such that the second group of one or more trigger conditions is selectively skipped depending upon values of the data being transformed; and

storing and/or transmitting data based on an output specified by the control structure.

24. The method of claim 23, further including:

determining a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

25. The method of claim 23, further including:

updating the execution time for a trigger condition in a list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sorting pointers to trigger conditions in the control structure based on the updated execution time.

26. The method of claim 23, further including:

determining a logical sequence of the plurality of trigger conditions of the control structure based on failure rates for the trigger conditions.

27. The method of claim 23, further including:

updating a failure rate for a trigger condition in a list of unique trigger conditions based on whether the trigger condition is satisfied by a record in the input data; and

determining a logical sequence of the plurality of trigger conditions of the control structure based on the updated failure rate.

28. The method of claim 23, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.

29. At least one non-transitory computer readable medium comprising processor-executable instructions that, when executed, cause the at least one processor to:

encode a rule set for transforming data, the rule set defining a sequence of execution cases, wherein each execution case in the sequence of the execution cases is associated with one or more trigger conditions and an output to be produced when the one or more trigger conditions for a corresponding execution case are all satisfied, said encoding comprising;

generating a control structure, using the at least one processor, based at least in part on the rule set, the control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions that identify which of the plurality of trigger conditions to evaluate according to a logical result of evaluating a different one of the plurality of trigger conditions on values of the data being transformed,

wherein the control structure includes a first group of one or more trigger conditions associated with a first execution case, a second group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the first execution case, and a third group of one or more trigger conditions associated with an execution case appearing later in the sequence of execution cases than the second execution case, and

wherein the control structure includes at least one logical association between a trigger condition of the first group and a trigger condition of the third group such that whether trigger conditions of the second group of one or more trigger conditions are evaluated is selectively skipped based depending upon values of the data being transformed;

receive input data; and

check trigger conditions of the control structure against the input data in a sequence determined using the control structure.

30. The at least one non-transitory computer readable medium of claim 29, wherein the processor-executable instructions further cause the at least one processor to store and/or transmit data based on an output specified by the control structure.

31. The at least one non-transitory computer readable medium of claim 29, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

32. The at least one non-transitory computer readable medium of claim 29, wherein the plurality of trigger conditions specified by the control structure consist of unique trigger conditions from the rule set.

33. The at least one non-transitory computer readable medium of claim 29, wherein the processor-executable instructions further cause the at least one processor to:

determine a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

34. The at least one non-transitory computer readable medium of claim 29, wherein the processor-executable instructions further cause the at least one processor to:

update the execution time for a trigger condition in a list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sort pointers to trigger conditions in the control structure based on the updated execution time.

35. The at least one non-transitory computer readable medium of claim 29, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

36. The at least one non-transitory computer readable medium of claim 29, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

37. The at least one non-transitory computer readable medium of claim 29, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.