DATA ASSET RECONSTRUCTION

• US 20160179630A1
• Filed: 12/19/2014
• Published: 06/23/2016
• Est. Priority Date: 12/19/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Nizri, Ortal, Halberstadt, Benjamin, Sharf, Erel, Steinhart, Shlomo
• Granted Patent: US 9,886,353 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1402 Saving, restoring, recovering or retrying
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

According to an aspect, data asset reconstruction includes receiving a data lineage for a data asset, where the data lineage identifies a data source connected to the data asset by an intermediate process. It is determined whether the intermediate process can be used to reconstruct a lost data asset. Based on determining that the intermediate process can be used to reconstruct the lost data asset, reconstructing the lost data asset with the intermediate process and the data source.

Claims

1. A system for data asset reconstruction, comprising:

a memory having computer readable instructions; and

a processor for executing the computer readable instructions, the computer readable instructions including;

receiving a data lineage for a data asset, wherein the data lineage identifies a data source connected to the data asset by an intermediate process;

determining whether the intermediate process can be used to reconstruct a lost data asset; and

based on determining that the intermediate process can be used to reconstruct the lost data asset, reconstructing the lost data asset with the intermediate process and the data source.

2. The system recited in claim 1, wherein:

the data source is upstream from the lost data asset in the data lineage; and

the intermediate process connecting the data source to the lost data asset is a process that was applied to the data source to yield the data asset.

3. The system recited in claim 2, wherein the computer instructions further include, based on the intermediate process being rerunnable, reconstructing the lost data asset by rerunning the intermediate process using the data from the data source to yield the lost data asset.

4. The system recited in claim 3, wherein the intermediate process is rerunnable if the process did not change the data source or if the intermediate process is a known one-to-one function.

5. The system recited in claim 1, wherein:

the data source is downstream from the data asset in the data lineage; and

the intermediate process connecting the data source to the data asset is a process that was applied to the data asset to yield the data source.

6. The system recited in claim 5, wherein the computer instructions further include, based on the intermediate process being reversible, reconstructing the lost data asset by reversing the intermediate process from the data source to yield the lost data asset.

7. The system recited in claim 1, wherein the lost data asset is a first lost data asset, the data source is a first data source, the intermediate process is a first intermediate process, the data lineage is a first data lineage, and the computer instructions further include:

identifying a second lost data asset of the first data source upon which reconstruction of the first lost data asset depends;

receiving a second data lineage for the second lost data asset, wherein the second data lineage identifies at least a second data source connected to the second lost data asset by a second intermediate process; and

reconstructing the second lost data asset with the second intermediate process and the second data source prior to reconstructing the first lost data asset.

8. The system recited in claim 1, wherein the computer instructions further include reconstructing a portion of the lost data asset even if a full version of the lost data asset cannot be reconstructed.

9. A method for data asset reconstruction comprising:

obtaining, by a processor, a data lineage for a data asset at a memory location, wherein the data lineage identifies a data source related to the data asset by an intermediate process;

analyzing, by the processor, the intermediate process to determine whether it can be used to reconstruct a lost data asset; and

based on determining that the intermediate process can be used to reconstruct the lost data asset, reconstructing, the processor, the lost data asset with the intermediate process and the data source.

10. The method recited in claim 9, wherein:

the data source is upstream from the data asset in the data lineage;

the intermediate process connecting the data source to the data asset is a process that was applied to the data source to yield the lost data asset.

11. The method recited in claim 10, wherein, based on the intermediate process being rerunnable, reconstructing the lost data asset by rerunning the intermediate process using the data from the data source to yield the lost data asset.

12. The method recited in claim 11, wherein the intermediate process is rerunnable if the intermediate process did not change the data source or if the intermediate process is a known one-to-one function.

13. The method recited in claim 9, wherein:

the data source is downstream from the data asset in the data lineage; and

the intermediate process connecting the data source to the data asset is a process that was applied to the data asset to yield the data source.

14. The method recited in claim 13, wherein based on the intermediate process being reversible, reconstructing the lost data asset by reversing the intermediate process from the data source to yield the lost data asset.

15. The method recited in claim 14, wherein the intermediate process is reversible if the intermediate process did not use a function that lacked a corresponding compensating function.

16. The method recited in claim 14, wherein reversing the intermediate process from the data source comprises:

based on determining that a function was used as part of the intermediate process to generate the data source, a compensating function of the function is applied to the data source; and

based on determining that more than one function was used as part of the intermediate process to generate the data source, the order of the more than one functions is reversed to establish a new order and the compensating function for each of the more than one function are applied to the data source in the new order.

17. The method recited in claim 9, wherein the lost data asset is a first lost data asset, the data source is a first data source, the intermediate process is a first intermediate process, the data lineage is a first data lineage further comprising:

identifying a second lost data asset of the first data source upon which reconstruction of the first lost data asset depends;

receiving a second data lineage for the second lost data asset, wherein the second data lineage identifies at least a second data source connected to the second lost data asset by a second intermediate process; and

reconstructing the second lost data asset with the second intermediate process and the second data source prior to reconstructing the first lost data asset.

18. A computer program product comprising:

a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to perform a method comprising;

receiving a data lineage for a data asset, wherein the data lineage identifies a data source connected to the data asset by an intermediate process;

determining whether the intermediate process can be used to reconstruct a lost data asset; and

based on determining that the intermediate process can be used to reconstruct the lost data asset, reconstructing the lost data asset with the intermediate process and the data source.

19. The computer program product of claim 18, wherein:

the data source is upstream from the lost data asset in the data lineage;

the intermediate process connecting the data source to the lost data asset is a process that was applied to the data source to yield the data asset;

the intermediate process is rerunnable if the process did not change the data source or if the intermediate process is a known one-to-one function; and

based on the intermediate process being rerunnable, the method further comprises reconstructing the lost data asset by rerunning the intermediate process using the data from the data source to yield the lost data asset.

20. The computer program product of claim 18, wherein

the data source is downstream from the data asset in the data lineage;

and

the intermediate process connecting the data source to the data asset is a process that was applied to the data asset to yield the data source; and

the method further comprises based on determining that a function was used as part of the intermediate process to generate the data source, a compensating function of the function is applied to the data source; and

.