System for transform generation

• US 9,703,822 B2
• Filed: 08/02/2013
• Issued: 07/11/2017
• Est. Priority Date: 12/10/2012
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Studer, Scott, Wholey, III, Joseph Skeffington, Weisman, Amit
• Primary Examiner(s): Uddin, MD. I
• Application Number:
  US13/958,037
• Publication Number:
  US 20140164410A1
• Time in Patent Office: 1,439 Days
• Field of Search: 707718, 707752, 707756
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24534 Query rewriting; Transformation
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

This specification describes technologies relating to generating transforms based on rule sets. In general, one aspect described in this specification can be embodied in methods that include receiving a rule set including execution cases, where at least one execution case in the rule set includes one or more trigger conditions and a specification of an output that is to be generated when the one or more trigger conditions are all satisfied. The methods may further include generating a control structure including a sequence of rows corresponding to one or more execution cases in the rule set. Each row may include a sequence of one or more trigger conditions and information specifying the output for a corresponding execution case. For at least one of the trigger conditions, when the trigger condition is failed, the control structure may direct processing to skip at least one row in the sequence of rows.

Claims

1. A method, performed by one or more data processing apparatus, for including:

encoding a rule set for transforming data, said encoding comprising;

receiving a rule set including a sequence of execution cases, each execution case in the sequence of the execution cases including one or more trigger conditions and a specification of an output that is to be generated when the one or more trigger conditions for a corresponding execution case are all satisfied; and

generating a control structure using at least one processor based at least in part on the rule set, the control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions, the logical associations specifying at least, for a first trigger condition of the plurality of trigger conditions;

a second trigger condition of the plurality of trigger conditions to be evaluated when evaluation of the first trigger condition by the at least one processor determines the first trigger condition to be false,

wherein the first trigger condition is included in a first execution case of the sequence of execution cases, and

wherein the second trigger condition is included in a second execution case of the sequence of execution cases, the second execution case being non-sequential to the first execution case in the sequence of execution cases,

wherein the control structure comprises;

a first group of one or more trigger conditions associated with a first set of execution cases;

a second group of one or more trigger conditions associated with execution cases appearing earlier in the sequence of execution cases than the first set of execution cases; and

a third group of one or more trigger conditions associated with execution cases appearing later in the sequence of execution cases than the first set of execution cases, and

wherein the control structure is configured such that, when trigger conditions of the second group and the third group are evaluated, whether trigger conditions of the first group of one or more trigger conditions are evaluated is based upon values of the data being transformed.

2. The method of claim 1, further including:

receiving input data;

checking trigger conditions against the input data in a sequence determined using the control structure; and

storing or transmitting data based on an output specified by the control structure.

3. The method of claim 1, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

4. The method of claim 1, wherein the plurality of trigger conditions specified by the control structure are unique trigger conditions from the rule set.

5. The method of claim 1, wherein the control structure specifies logical associations between trigger conditions and unique outputs from the rule set.

6. The method of claim 1, further including:

determining a logical sequence of trigger conditions of the control structure that are associated with a first execution case based on a trigger condition associated with a second execution case to which processing will be directed when the trigger condition associated with a second execution case fails during processing of data.

7. The method of claim 1, further including:

determining a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

8. The method of claim 1, further including:

receiving input data;

checking trigger conditions against the input data in a sequence determined using the control structure;

updating the execution time for a trigger condition in the list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sorting pointers to trigger conditions in the control structure based on the updated execution time.

9. The method of claim 1, further including:

determining a logical sequence of the plurality of trigger conditions of the control structure based on failure rates for the trigger conditions.

10. The method of claim 1, further including:

receiving input data;

checking trigger conditions against the input data in a sequence determined using the control structure;

updating the failure rate for a trigger condition in a list of unique trigger conditions based on whether the trigger condition is satisfied by a record in the input data; and

determining a logical sequence of the plurality of trigger conditions of the control structure based on the updated failure rate.

11. The method of claim 1, wherein the control structure further includes a portion of code that directs processing to one of the plurality of trigger conditions of the control structure that is included in a fourth execution case when all of the trigger conditions included in a third execution case are evaluated to be true.

12. The method of claim 1, wherein the rule set is specified through a graphical user interface.

13. The method of claim 1, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

14. The method of claim 1, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

15. The method of claim 1, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.

16. Software stored on a non-transitory computer-readable medium the software including instructions that, when executed by a computing system comprising at least one processor, cause the computing system to:

encode a rule set for transforming data, said encoding comprising causing the computer system to;

receive a rule set including a sequence of execution cases, each execution case in the sequence of the execution cases including one or more trigger conditions and a specification of an output that is to be generated when the one or more trigger conditions for a corresponding execution case are all satisfied; and

generate, based at least in part on the rule set, a control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions, the logical associations specifying at least, for a first trigger condition of the plurality of trigger conditions;

a second trigger condition of the plurality of trigger conditions to be evaluated when evaluation of the first trigger condition by the at least one processor determines the first trigger condition to be false,

wherein the first trigger condition is included in a first execution case of the sequence of execution cases, and

wherein the second trigger condition is included in a second execution case of the sequence of execution cases, the second execution case being non-sequential to the first execution case in the sequence of execution cases,

wherein the control structure comprises;

a first group of one or more trigger conditions associated with a first set of execution cases;

a second group of one or more trigger conditions associated with execution cases appearing earlier in the sequence of execution cases than the first set of execution cases; and

a third group of one or more trigger conditions associated with execution cases appearing later in the sequence of execution cases than the first set of execution cases, and

wherein the control structure is configured such that, when trigger conditions of the second group and the third group are evaluated, whether trigger conditions of the first group of one or more trigger conditions are evaluated is based upon values of the data being transformed.

17. The medium of claim 16, including instructions for causing a computing system to:

receive input data;

check trigger conditions against the input data in a sequence determined using the control structure; and

store or transmit data based on an output specified by the control structure.

18. The medium of claim 16, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

19. The medium of claim 16, wherein the plurality of trigger conditions specified by the control structure are unique trigger conditions from the rule set.

20. The medium of claim 16, wherein the control structure specifies logical associations between trigger conditions and unique outputs from the rule set.

21. The medium of claim 16, including instructions for causing a computing system to:

determine a logical sequence of trigger conditions of the control structure that are associated with a first execution case based on a trigger condition associated with a second execution case to which processing will be directed when the trigger condition associated with a second execution case fails during processing of data when a trigger condition in the sequence fails during processing of data.

22. The medium of claim 16, including instructions for causing a computing system to:

determine a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

23. The medium of claim 22, including instructions for causing a computing system to:

receive input data;

check trigger conditions against the input data in a sequence determined using the control structure;

update the execution time for a trigger condition in the list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sort pointers to trigger conditions in the control structure based on the updated execution time.

24. The medium of claim 16, including instructions for causing a computing system to:

determine a logical sequence of the plurality of trigger conditions of the control structure based on failure rates for the trigger conditions.

25. The medium of claim 24, including instructions for causing a computing system to:

receive input data;

check trigger conditions against the input data in a sequence determined using the control structure;

update the failure rate for a trigger condition in a list of unique trigger conditions based on whether the trigger condition is satisfied by a record in the input data; and

determine a logical sequence of the plurality of trigger conditions of the control structure based on the updated failure rate.

26. The medium of claim 16, wherein the control structure further includes a portion of code that directs processing to one of the plurality of trigger conditions of the control structure that is included in a fourth execution case when all of the trigger conditions included in a third execution case are evaluated to be true.

27. The medium of claim 16, wherein the rule set is specified through a graphical user interface.

28. The medium of claim 16, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

29. The medium of claim 16, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

30. The medium of claim 16, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.

31. A computing system for encoding a rule set for transforming data, the computing system including:

an input device or port configured to receive a rule set including a sequence of execution cases, each execution case in the sequence of the execution cases including one or more trigger conditions and a specification of an output that is to be generated when the one or more trigger conditions for a corresponding execution case are all satisfied; and

at least one processor configured to perform operations, the operations including generating, based at least in part on the rule set, a control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions, the logical associations specifying at least, for a first trigger condition of the plurality of trigger conditions;

a second trigger condition of the plurality of trigger conditions to be evaluated when evaluation of the first trigger condition by the at least one processor determines the first trigger condition to be false,

wherein the first trigger condition is included in a first execution case of the sequence of execution cases, and

wherein the second trigger condition is included in a second execution case of the sequence of execution cases, the second execution case being non-sequential to the first execution case in the sequence of execution cases,

wherein the control structure comprises;

a first group of one or more trigger conditions associated with a first set of execution cases;

a second group of one or more trigger conditions associated with execution cases appearing earlier in the sequence of execution cases than the first set of execution cases; and

a third group of one or more trigger conditions associated with execution cases appearing later in the sequence of execution cases than the first set of execution cases, and

wherein the control structure is configured such that, when trigger conditions of the second group and the third group are evaluated, whether trigger conditions of the first group of one or more trigger conditions are evaluated is based upon values of the data being transformed.

32. The system of claim 31, including:

an input device or port configured to receive input data;

at least one processor configured to perform operations, the operations including checking trigger conditions against the input data in a sequence determined using the control structure; and

an output device or port configured to transmit data based on an output specified by the control structure.

33. The system of claim 31, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

34. The system of claim 31, wherein the plurality of trigger conditions specified by the control structure are unique trigger conditions from the rule set.

35. The system of claim 31, wherein the control structure specifies logical associations between trigger conditions and unique outputs from the rule set.

36. The system of claim 31, wherein the operations further include:

determining a logical sequence of trigger conditions of the control structure that are associated with a first execution case based on a trigger condition associated with a second execution case to which processing will be directed when the trigger condition associated with a second execution case fails during processing of data.

37. The system of claim 31, wherein the operations further include:

determining a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

38. The system of claim 37, wherein the operations further include:

receiving input data;

checking trigger conditions against the input data in a sequence determined using the control structure;

updating the execution time for a trigger condition in the list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

sorting pointers to trigger conditions in the control structure based on the updated execution time.

39. The system of claim 31, wherein the operations further include:

determining a logical sequence of the plurality of trigger conditions of the control structure based on failure rates for the trigger conditions.

40. The system of claim 39, wherein the operations further include:

receiving input data;

checking trigger conditions against the input data in a sequence determined using the control structure;

updating the failure rate for a trigger condition in a list of unique trigger conditions based on whether the trigger condition is satisfied by a record in the input data; and

determining a logical sequence of the plurality of trigger conditions of the control structure based on the updated failure rate.

41. The system of claim 31, wherein the control structure further includes a portion of code that directs processing to one of the plurality of trigger conditions of the control structure that is included in a fourth execution case when all of the trigger conditions included in a third execution case are evaluated to be true.

42. The system of claim 31, wherein the rule set is specified through a graphical user interface.

43. The system of claim 31, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

44. The system of claim 31, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

45. The system of claim 31, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.

46. A computing system for encoding a rule set for transforming data, the computing system including:

an input device or port configured to receive a rule set including a sequence of execution cases, each execution case in the sequence of the execution cases including one or more trigger conditions and a specification of an output that is to be generated when the one or more trigger conditions for a corresponding execution case are all satisfied; and

means for generating, based at least in part on the rule set, a control structure specifying;

a plurality of trigger conditions of the one or more trigger conditions associated with execution cases of the sequence of execution cases; and

logical associations between pairs of the plurality of trigger conditions, the logical associations specifying at least, for a first trigger condition of the plurality of trigger conditions;

a second trigger condition of the plurality of trigger conditions to be evaluated when evaluation of the first trigger condition by the at least one processor determines the first trigger condition to be false,

wherein the first trigger condition is included in a first execution case of the sequence of execution cases, and

wherein the second trigger condition is included in a second execution case of the sequence of execution cases, the second execution case being non-sequential to the first execution case in the sequence of execution cases,

wherein the control structure comprises;

a first group of one or more trigger conditions associated with a first set of execution cases;

a second group of one or more trigger conditions associated with execution cases appearing earlier in the sequence of execution cases than the first set of execution cases; and

a third group of one or more trigger conditions associated with execution cases appearing later in the sequence of execution cases than the first set of execution cases, and

wherein the control structure is configured such that, when trigger conditions of the second group and the third group are evaluated, whether trigger conditions of the first group of one or more trigger conditions are evaluated is based upon values of the data being transformed.

47. The system of claim 46, including:

an input device or port configured to receive input data;

at least one processor configured to perform operations, the operations including checking trigger conditions against the input data in a sequence determined using the control structure; and

an output device or port configured to transmit data based on an output specified by the control structure.

48. The system of claim 46, wherein at least one of the one or more trigger conditions associated with execution cases of the sequence of execution cases is not specified by the control structure.

49. The system of claim 46, wherein the plurality of trigger conditions specified by the control structure are unique trigger conditions from the rule set.

50. The system of claim 46, wherein the control structure specifies logical associations between trigger conditions and unique outputs from the rule set.

51. The system of claim 46, including:

means for determining a logical sequence of the plurality of trigger conditions of the control structure based on a position of each trigger condition in the sequence of execution cases.

52. The system of claim 46, including:

means for determining a logical sequence of the plurality of trigger conditions of the control structure based on execution times for the trigger conditions.

53. The system of claim 52, including:

an input device or port configured to receive input data;

at least one processor configured to perform operations, the operations including;

checking trigger conditions against the input data in a sequence determined using the control structure; and

updating the execution time for a trigger condition in the list of unique trigger conditions based on the time it takes to execute the trigger condition with the input data; and

means for sorting pointers to trigger conditions in the control structure based on the updated execution time.

54. The system of claim 46, including:

means for determining a logical sequence of the plurality of trigger conditions of the control structure based on failure rates for the trigger conditions.

55. The system of claim 54, wherein the operations further include:

an input device or port configured to receive input data;

at least one processor configured to perform operations, the operations including;

checking trigger conditions against the input data in a sequence determined using the control structure; and

updating the failure rate for a trigger condition in a list of unique trigger conditions based on whether the trigger condition is satisfied by a record in the input data; and

means for determining a logical sequence of the plurality of trigger conditions of the control structure based on the updated failure rate.

56. The system of claim 46, wherein the control structure further includes a portion of code that directs processing to one of the plurality of trigger conditions of the control structure that is included in a fourth execution case when all of the trigger conditions included in a third execution case are evaluated to be true.

57. The system of claim 46, wherein the rule set is specified through a graphical user interface.

58. The system of claim 46, wherein at least two trigger conditions for an execution case in the rule set are combined and represented by a single trigger condition in the control structure.

59. The system of claim 46, wherein at least two outputs for different execution cases in the rule set are combined and represented by a single output expression in the control structure.

60. The system of claim 46, wherein the control structure is an acyclic directed graph with nodes corresponding to the plurality of trigger conditions and vertices corresponding to the logical associations between pairs of the plurality of trigger conditions.