Impact analysis

• US 10,176,234 B2
• Filed: 06/12/2015
• Issued: 01/08/2019
• Est. Priority Date: 11/05/2014
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Gould, Joel, Studer, Scott
• Primary Examiner(s): Nguyen, Phong H
• Application Number:
  US14/738,232
• Publication Number:
  US 20160125057A1
• Time in Patent Office: 1,306 Days
• Field of Search: 707756
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for impact analysis. One of the methods includes receiving information about at least two logical datasets, the information identifying, for each logical dataset, a field in that logical dataset and format information about that field. The method includes receiving information about a transformation identifying a first logical dataset from which the transformation is to receive data and a second logical dataset to which the transformed data is provided. The method includes receiving one or more proposed changes to at least one of the fields. The method includes analyzing the proposed changes based on information about the transformation and information about the first logical dataset and the second logical dataset. The method includes calculating metrics of the proposed change based on the analysis. The method also includes storing information about the metrics.

Claims

1. A computer-implemented method, the method including:

receiving data lineage information for multiple components representing at least two logical datasets and a transformation, with the data lineage information identifying a first logical dataset from which the transformation is to receive data and a second logical data dataset to which transformed data is to be provided, the transformation represented by a component in the data lineage information including one or more rules to be applied to data from the first logical dataset, and with the data lineage information identifying paths and flow traces of data though the multiple components;

receiving data specifying one or more proposed changes to a field in the first logical dataset, a field in the second logical dataset, or the transformation;

analyzing the multiple components according to the data lineage information to identify each component affected by the one or more proposed changes;

for at least one identified component identified as affected by the one or more proposed changes,

generating an impact metric representing a number of times a field specified by the one or more proposed changes is referenced within the at least one identified component;

determining, based on the generated impact metric, an impact of implementing the one or more proposed changes to one or more of the multiple components; and

storing information about the impact metric.

2. The method of claim 1, wherein the impact metric provides a measure of a direct impact of implementing the one or more proposed changes to the one or more of the multiple components.

3. The method of claim 1, wherein the impact metric provides a measure of an indirect impact of implementing the one or more proposed changes to the one or more of the multiple components.

4. The method of claim 1, further comprising associating a cost to a proposed change based on one or more impact metrics.

5. The method of claim 1, further including:

for that at least one component identified that represents the transformation, generating annotated code that includes the impact metric;

compiling the annotated code into an executable program; and

at least partially based on execution of the executable program, determining the impact of the one or more proposed changes to one or more of the multiple components.

6. The method of claim 1, further including:

selecting from data storage stored items of annotated code for one or more of the multiple components in the data lineage information, which are affected by the one or more proposed changes;

compiling the annotated code of one or more of the stored items into a transform; and

processing the transform to determine a risk output of a corresponding impact metric.

7. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving a data lineage information for multiple components representing at least two logical datasets and a transformation, with the data lineage information identifying a first logical dataset describing characteristics of a first physical dataset from which the transformation is to receive data and a second logical dataset describing characteristics of a second physical dataset to which transformed data is to be provided, the transformation represented by a component in the data lineage information including one or more rules to be applied to data from the first logical dataset, and with the data lineage information identifying paths and flow traces of data though the multiple components;

receiving data specifying one or more proposed changes to a field in the first logical dataset, a field in the second logical dataset, or the transformation;

analyzing the multiple components according to the data lineage information to identify each component affected by the one or more proposed changes;

for at least one component identified as affected by the one or more proposed changes,

generating an impact metric representing a number of times a field specified by the one or more proposed changes is referenced within the at least one identified component;

determining, based on the generated impact metric, an impact of implementing the one or more proposed changes to one or more of the multiple components; and

storing information about the impact metric.

8. The system of claim 7, wherein the impact metric provides a measure of a direct impact of implementing the one or more proposed changes to the one or more of the multiple components.

9. The system of claim 7, wherein the impact metric provides a measure of an indirect impact of implementing the one or more proposed changes to the one or more of the multiple components.

10. The system of claim 7, wherein a proposed change is one of the group consisting of a change in format of a field in a dataset or a change in a transformation.

11. The system of claim 7, wherein the operations further comprise associating a cost to a proposed change based on one or more impact metrics.

12. A system comprising:

one or more processor devices; and

memory operatively coupled to the one or more processor devices, storing a computer program that configures the system to;

receive data lineage information for multiple components representing at least first and second logical datasets and a transformation, with the data lineage information identifying a first physical dataset from which the transformation is to receive data and a second physical data dataset to which transformed data is to be provided, the transformation represented by a component in the data lineage information including one or more rules to be applied to data from the first logical dataset, and with the data lineage information identifying paths and flow traces of data though the multiple components;

receive data specifying one or more proposed changes to a field in the first logical dataset, the second logical dataset, or the transformation;

analyze the multiple components according to the data lineage information to identify each component affected by the one or more proposed changes;

for at least one identified component identified as affected by the one or more proposed changes,

generate an impact metric representing a number of times a field specified by the one or more proposed changes is referenced within the at least one identified component;

determine, based on the generated impact metric, an impact of implementing one or more of the multiple components; and

store information about the impact metric.

13. A computer storage medium encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving data lineage information for multiple components representing at least two logical datasets and a transformation, with the data lineage information identifying a first logical dataset describing characteristics of a first physical dataset from which the transformation is to receive data and a second logical dataset describing characteristics of a second physical dataset to which transformed data is to be provided, the transformation represented by a component in the data lineage information including one or more rules to be applied to data from the first logical dataset, and with the data lineage information identifying paths and flow traces of data though the multiple components;

receiving data specifying one or more proposed changes to a field in the first logical dataset, a field in the second logical dataset, or the transformation;

analyzing the multiple components according to the data lineage information to identify each component affected by the one or more proposed changes;

for at least one component identified that represents a transformation,

for at least one identified component identified as affected by the one or more proposed changes,

generating an impact metric representing a number of times a field specified by the one or more proposed changes is referenced within the at least one identified component;

determining, based on the generated impact metric, an impact of implementing the one or more proposed changes to one or more of the multiple components; and

storing information about the impact metric.

14. The medium of claim 13, where the impact metric provides a measure of a direct impact of implementing the one or more proposed changes to the one or more of the multiple components.

15. The medium of claim 13, where the impact metric provides a measure an indirect impact of implementing the one or more proposed changes to the one or more of the multiple components.

16. The medium of claim 13, wherein a proposed change is one of the group consisting of a change in format of a field in a dataset or a change in a transformation.

17. The medium of claim 13, further comprising associating a cost to a proposed change based on one or more impact metrics.