Data lineage in an intelligent data integration system

• US 9,372,903 B1
• Filed: 06/05/2013
• Issued: 06/21/2016
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Monsanto Technology LLC (Bayer AG)
• Original Assignee: Monsanto Technology LLC (Bayer AG)
• Inventors: Richt, Ryan Jerry, Taylor, Christopher Allen
• Primary Examiner(s): Jami, Hares
• Application Number:
  US13/910,302
• Time in Patent Office: 1,112 Days
• Field of Search: 707/602, 707/736, 707/756, 707/803, 707/804, 717/116
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G16B 30/00 ICT specially adapted for sequence analysis involving nucleotides or amino acids
• CPC Class Codes: G16B 50/00 ICT programming tools or database systems specially adapted for bioinformatics

Abstract

A computer-executed method includes storing in a data store data attributes, data objects, and a data analysis tool (DAT). Each data object has an attribute set and an identifier set. The method includes identifying each data object that has an attribute set comprising a data attribute matching each reference data attribute associated with the DAT, and selecting an identified data object as an input data object for the DAT. The DAT generates a new data object as a function of the input data object, which includes analyzing the input data object with reference to an auxiliary data object and creating an identifier set for the new data object that includes an identifier of the new data object for distinguishing the new data object from each other data object in the data store, and the identifier of the auxiliary data object. The new data object is stored in the data store.

Claims

1. A computer-executed method for associating data objects, the method comprising:

storing in a data store a plurality of data attributes for defining data objects;

storing in the data store a plurality of data objects, each data object of the plurality having an attribute set comprising one or more data attributes of the plurality of stored data attributes, and each data object is associated with an identifier set comprising at least an identifier of the data object for distinguishing the data object from each other data object of the plurality of data objects;

storing in the data store a data analysis tool (DAT), said DAT associated with at least one reference data attribute of the plurality of stored data attributes, and associated with at least one auxiliary data object of the stored plurality of data objects;

identifying each data object of the plurality of stored data objects that has an attribute set comprising a data attribute that matches each at least one reference data attribute associated with the DAT;

selecting at least one identified data object to be an input data object for the DAT;

generating by the DAT a new data object as a function of the input data object for the DAT, wherein said generating comprises analyzing the input data object with reference to the at least one auxiliary data object and creating an identifier set for the new data object that includes an identifier of the new data object for distinguishing the new data object from each other data object of the plurality of data objects, and the identifier of the at least one auxiliary data object;

storing the new data object as a data object of the plurality of data objects in the data store;

identifying, as a primary data object, each data object in the data store that satisfies user-defined search criteria;

identifying, as an attenuated data object, each data object in the data store that has an identifier set which includes the identifier of a primary data object; and

rendering each primary data object and each attenuated data object as search results for the user-defined search criteria.

2. The computer-executed method of claim 1 wherein said creating an identifier set for the new data object comprises creating an identifier set for the new data object that further includes the identifier of the input data object for the DAT from which the new data object was generated.

3. The computer-executed method of claim 1 wherein said creating an identifier set for the new data object comprises creating an identifier set for the new data object that further includes each identifier included in the identifier set of the input data object for the DAT from which the new data object was created.

4. The computer-executed method of claim 1 wherein the identifier set associated with a data object is stored in said data object.

5. The computer-executed method of claim 1 wherein the identifier set associated with a data object is stored in the data store separately from said data object.

6. The computer-executed method of claim 1 wherein said generating a new data object further comprises establishing an attribute set for the new data object comprising one or more data attributes of the plurality of stored data attributes.

7. The computer-executed method of claim 6 wherein said DAT is a first DAT, and said at least one reference data attribute associated with the first DAT is an at least one first reference data attribute, and said new data object is a first new data object, and wherein the computer-executed method of claim 5 further comprises:

storing in the data store a second DAT, said second DAT associated with at least one second reference data attribute of the plurality of stored data attributes, wherein the data attribute set of the first new data object comprises a data attribute that matches each at least one second reference data attribute;

identifying the first new data object of the plurality of stored data objects as having a data attribute set comprising a data attribute that matches each at least one second reference data attribute;

selecting the first new data object as an input data object for the second DAT;

generating by the second DAT a second new data object as a function of the input data object for the second DAT, wherein said generating comprises creating an identifier set for the second new data object that includes an identifier for the second new data object for distinguishing the second new data object from each other data object of the plurality of data objects; and

storing the second new data object as a data object of the plurality of data object in the data store.

8. The computer-executed method of claim 7 wherein said creating an identifier set for the first new data object comprises creating an identifier set for the first new data object that further includes each identifier included in the identifier set of the input data object for the first DAT from which the first new data object was created, and wherein said creating an identifier set for the second new data object comprises creating an identifier set for the second new data object that further includes each identifier included in the identifier set of the second input data object for the second DAT from which the second new data object was created.

9. The computer-executed method of claim 7 wherein said creating an identifier set for the first new data object comprises creating an identifier set for the first new data object that further includes the identifier of the input data object for the first DAT from which the first new data object was generated, and wherein said creating an identifier set for the second new data object comprises creating an identifier set for the second new data object that further includes the identifier of the input data object for the second DAT from which the second new data object was generated.

10. The computer-executed method of claim 1 wherein the identifier of the at least one auxiliary data object has a role name associated therewith, said role name providing a description of the reference to said at least one auxiliary data object by the DAT.

11. A system for associating data objects, the system comprising:

at least one processor;

at least one data storage device storing computer-executable instructions for execution by the processor;

said data storage device;

storing in a data store a plurality of data attributes for defining data objects;

storing in the data store a plurality of data objects, each data object of the plurality having an attribute set comprising one or more data attributes of the plurality of stored data attributes, and each data object is associated with an identifier set comprising at least an identifier of the data object for distinguishing the data object from each other data object of the plurality of data objects; and

storing in the data store a data analysis tool (DAT), said DAT associated with at least one reference data attribute of the plurality of stored data attributes, and associated with at least one auxiliary data object of the stored plurality of data objects;

said computer-executable instructions comprising;

instructions for identifying each data object of the plurality of stored data objects that has an attribute set comprising a data attribute that matches each at least one reference data attribute associated with the DAT;

instructions for selecting at least one identified data object to be an input data object for the DAT;

instructions for generating by the DAT a new data object as a function of the input data object for the DAT, wherein said generating comprises analyzing the input data object with reference to the at least one auxiliary data object and creating an identifier set for the new data object that includes an identifier of the new data object for distinguishing the new data object from each other data object of the plurality of data objects, and the identifier of the at least one auxiliary data object;

instructions for storing the new data object as a data object of the plurality of data objects in the data store,

instructions for identifying, as a primary data object, each data object in the data store that satisfies user-defined search criteria;

instructions for identifying, as an attenuated data object, each data object in the data store that has an identifier set which includes the identifier of a primary data object; and

instructions for rendering each primary data object and each attenuated data object as search results for the user-defined search criteria.

12. The system of claim 11 wherein said creating an identifier set for the new data object comprises creating an identifier set for the new data object that further includes the identifier of the input data object for the DAT from which the new data object was generated.

13. The system of claim 11 wherein said creating an identifier set for the new data object comprises creating an identifier set for the new data object that further includes each identifier included in the identifier set of the input data object for the DAT from which the new data object was created.

14. A tangible, non-transitory storage medium storing processor-executable instructions for associating data objects, said storage medium storing:

instructions for storing in a data store a plurality of data attributes for defining data objects;

instructions for storing in the data store a plurality of data objects, each data object of the plurality having an attribute set comprising one or more data attributes of the plurality of stored data attributes, and each data object is associated with an identifier set comprising at least an identifier of the data object for distinguishing the data object from each other data object of the plurality of data objects;

instructions for storing in the data store a data analysis tool (DAT), said DAT associated with at least one reference data attribute of the plurality of stored data attributes, and associated with at least one auxiliary data object of the stored plurality of data objects;

instructions for identifying each data object of the plurality of stored data objects that has an attribute set comprising a data attribute that matches each at least one reference data attribute associated with the DAT;

instructions for selecting at least one identified data object to be an input data object for the DAT;

instructions for generating by the DAT a new data object as a function of the input data object for the DAT, wherein said generating comprises analyzing the input data object with reference to the at least one auxiliary data object and creating an identifier set for the new data object that includes an identifier of the new data object for distinguishing the new data object from each other data object of the plurality of data objects, and the identifier of the at least one auxiliary data object;

instructions for storing the new data object as a data object of the plurality of data objects in the data store;

instructions for identifying, as a primary data object, each data object in the data store that satisfies user-defined search criteria;

instructions for identifying, as an attenuated data object, each data object in the data store that has an identifier set which includes the identifier of a primary data object; and

instructions for rendering each primary data object and each attenuated data object as search results for the user-defined search criteria.

15. The tangible, non-transitory storage medium of claim 14 wherein said creating an identifier set for the new data object comprises creating an identifier set for the new data object that further includes the identifier of the input data object for the DAT from which the new data object was generated.

16. The tangible, non-transitory storage medium of claim 14 wherein said creating an identifier set for the new data object comprises creating an identifier set for the new data object that further includes each identifier included in the identifier set of the input data object for the DAT from which the new data object was created.

17. A computer-executed method for associating data objects, the method comprising:

storing in a data store a plurality of data attributes for defining data objects;

storing in the data store a plurality of data objects, each data object of the plurality having an attribute set comprising one or more data attributes of the plurality of stored data attributes, and each data object is associated with an identifier set comprising at least an identifier of the data object for distinguishing the data object from each other data object of the plurality of data objects;

storing in the data store a first data analysis tool (DAT), said DAT associated with at least one first reference data attribute of the plurality of stored data attributes, and associated with at least one auxiliary data object of the stored plurality of data objects;

identifying each data object of the plurality of stored data objects that has an attribute set comprising a data attribute that matches each at least one first reference data attribute associated with the first DAT;

selecting at least one identified data object to be an input data object for the first DAT;

generating by the first DAT a first new data object as a function of the input data object for the first DAT, wherein said generating comprises analyzing the input data object with reference to the at least one auxiliary data object and creating an identifier set for the first new data object that includes an identifier of the first new data object for distinguishing the first new data object from each other data object of the plurality of data objects, and the identifier of the at least one auxiliary data object, wherein said generating a first new data object further comprises establishing an attribute set for the first new data object comprising one or more data attributes of the plurality of stored data attributes;

storing the first new data object as a data object of the plurality of data objects in the data store;

storing in the data store a second DAT, said second DAT associated with at least one second reference data attribute of the plurality of stored data attributes, wherein the data attribute set of the first new data object comprises a data attribute that matches each at least one second reference data attribute;

identifying the first new data object of the plurality of stored data objects as having a data attribute set comprising a data attribute that matches each at least one second reference data attribute;

selecting the first new data object as an input data object for the second DAT;

generating by the second DAT a second new data object as a function of the input data object for the second DAT, wherein said generating comprises creating an identifier set for the second new data object that includes an identifier for the second new data object for distinguishing the second new data object from each other data object of the plurality of data objects; and

storing the second new data object as a data object of the plurality of data object in the data store.