Intelligent data integration system

• US 9,262,469 B1
• Filed: 04/23/2013
• Issued: 02/16/2016
• Est. Priority Date: 04/23/2012
• Status: Active Grant

• Current Assignee: Monsanto Technology LLC (Bayer AG)
• Original Assignee: Monsanto Technology LLC (Bayer AG)
• Inventors: Taylor, Christopher Allen, Richt, Ryan Jerry
• Primary Examiner(s): HASAN, SYED HAROON
• Application Number:
  US13/868,258
• Time in Patent Office: 1,029 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

Data objects stored in a data store include data attribute(s) and associated value(s) for the attributes. Data analysis tools (DATs) stored in a data store are associated with reference data attribute(s). The data objects are identified by one or more DATs based on each reference data attribute(s) of a corresponding DAT matching one of the data attribute(s) of the corresponding data object(s) and independent of the value for the data attribute(s). The DATs generate an additional data object as a function of the identified data object, and the additional data object is stored in the data store.

Claims

1. A computer-executed method for self-assembling a plurality of individual data analysis tools into a workflow, the method comprising:

storing in a data store a plurality of data attributes for defining one or more data objects;

storing in the data store a first data object having a first attribute set comprising at least one first data attribute of the plurality of stored data attributes and a value for said at least one first data attribute;

storing in the data store a first data analysis tool (DAT), said first DAT associated with at least one first reference data attribute of the plurality of stored data attributes;

identifying said first data object based on each at least one first reference data attribute matching one of the at least one first data attribute of the first data attribute set and independent of the value for the at least one first data attribute of the first attribute set;

generating by said first DAT a second data object as a function of the identified first data object, wherein said second data object comprises a second attribute set comprising at least one second data attribute of the plurality of stored data attributes, and a value for said at least one second data attribute of the second attribute set;

storing the second data object in the data store;

storing in the data store a second data analysis tool (DAT), said second DAT associated with at least one second reference data attribute of the plurality of stored data attributes;

identifying said second data object based on each at least one second reference data attribute matching one of the at least second first data attribute of the second data attribute set and independent of the value for the at least one second data attribute of the second attribute set;

generating by said second DAT a third data object as a function of the identified second data object, wherein said third data object comprises a third data attribute set comprising at least one third data attribute of the plurality of stored data attributes, and a value for said at least one third data attribute of the third attribute set; and

storing said third data object in the data store;

storing in the data store a plurality of other first data objects, each of the plurality of other first data objects having an attribute set comprising at least one data attribute of the plurality of stored data attributes and a value for said at least one data attribute;

identifying said other first data objects based on each at least one first reference data attribute matching one of the at least one data attribute of each data attribute set of the other first data objects and independent of the value for the at least one data attribute of each data attribute set of the other first data objects; and

defining a collection of identified first data objects that includes the identified first data object and each of the identified plurality of the other first data objects.

2. The method of claim 1 wherein said identifying said first data object comprises determining that each of the at least one first reference data attribute is identical to or has a semantic lexical relationship with one of the at least one first reference data attribute.

3. The method of claim 1 further comprising:

mapping a plurality of the stored data attributes together;

determining whether each at least one first reference data attribute is mapped to one of the at least one first data attribute, wherein the first data attribute matches the first reference data attribute when the first data attribute is determined to be mapped to the first reference data attribute.

4. The method of claim 3 wherein mapping a plurality of the stored data attributes together comprises mapping a plurality of the stored data attributes together based on said plurality of the stored data attributes having a semantic lexical relationship.

5. The method of claim 1 wherein the first DAT is associated with the at least one first reference data attribute and an other first reference data attribute, and wherein said identifying said first data object comprises determining that a Boolean combination of the at least one first reference data attribute and the other first reference data attribute matches the at least one first data attribute of the first data attribute set.

6. The method of claim 1 wherein the first data attribute set comprises the at least one first data attribute and an other first data attribute, and wherein the first DAT is associated with the at least one first reference data attribute and an other first reference data attribute, and wherein said identifying said first data object comprises determining that a Boolean combination of the first reference data attribute and the other first reference data attribute matches the at least one first data attribute and the other first data attribute, in combination.

7. The method of claim 1 wherein said storing a first DAT in the data store comprises:

validating said first DAT, wherein said validating includes verifying that the at least one first data reference attribute associated with said first DAT is one of the plurality of data attributes stored in the data store;

compiling said first DAT; and

uploading the validated and compiled first DAT to the data store.

8. The method of claim 1 wherein said first DAT is associated with a first reference data object, said first reference data object having the at least one first reference data attribute.

9. The method of claim 8 wherein said first reference data object is identical to the first data object and each at least one first reference data attribute is identical to one of the at least one first data attribute.

10. The method of claim 8 wherein said first reference data object is different from the first data object and each at least one first reference data attribute matches one of the at least one first data attribute.

11. The method of claim 8 wherein said identifying said first data object comprises determining that each of the at least one first reference data attribute is identical to or has a semantic lexical relationship with one of the at least one first reference data attribute.

12. The method of claim 8 further comprising storing a first reference data object class in the data store, said first reference data object class defining requirements for instances of first reference data objects, wherein said storing a first DAT in the data store comprises:

validating said first DAT, wherein said validating includes verifying that the at least one first reference data object associated with said first DAT meets the requirements for the stored first reference object class; and

compiling said first DAT;

uploading the validated and compiled first DAT to the data store.

13. The method of claim 1 further comprising:

adding a new data object to the data store, said new data object having a particular set of data attributes;

in response to said adding the new data object to the data store;

determining whether each at least one first reference data attribute matches one of the data attributes of the new data object; and

determining whether each at least one second reference data attribute matches one of the data attributes of the new data object.

14. The method of claim 13 further comprising:

receiving a user-provided value;

querying the data store as a function of the user-provided value to identify data attributes having values corresponding to the user-provided value; and

generating a result set as a function of the querying, said result set including data objects having the identified data attributes.

15. The method of claim 1 wherein the identified first data objects of the collection are approximately ordered for analysis by the first DAT based on an order in which the identified first data objects of the collection were stored in the data store.

16. The method of claim 1 wherein the identified first data objects of the collection are approximately ordered for analysis by the first DAT based on an order in which the identified first data objects were identified.

17. The method of claim 1 wherein the identified first data objects of the collection are approximately ordered for analysis by the first DAT based on one of a size required for the storing each of the identified first data objects in the data store, and a total number of prior accesses by a DAT for each of the identified first data objects.

18. The method of claim 1 further comprising:

receiving a user-provided value;

querying the data store as a function of the user-provided value to identify data attributes having values corresponding to the user-provided value; and

generating a result set as a function of the querying, said result set including data objects having the identified data attributes;

wherein said receiving, querying, and generating are performed for each of a plurality of user-provided values.

19. The method of claim 18 wherein the identified first data objects of the collection are approximately ordered for analysis by the first DAT based on, for each identified first data object of the collection, a cumulative total number of generated result sets in which said identified first data object is included.

20. The method of claim 18 further comprising:

defining a display set of data objects, wherein said display set is a sub-set of the generated result set; and

rendering the display set of data objects for displaying to a user.

21. The method of claim 20 wherein the identified first data objects of the collection are approximately ordered for analysis by the first DAT based on, for each identified first data object of the collection, a cumulative total number of display sets in which the identified first data object is included.

22. The method of claim 20 further comprising receiving user-feedback for one or more data objects in the rendered display set.

23. The method of claim 22 wherein the identified first data objects of the collection are approximately ordered for analysis by the first DAT based on the received-user feedback.

24. The method of claim 1 further comprising iteratively selecting one identified first data object from the collection of identified first data objects for analysis by the first DAT, wherein the generating by said first DAT a second object as a function of the identified first data object comprises generating by said first DAT, for each iteratively selected one identified first data object, a second data object as a function of the selected one identified first data object.

25. The method of claim 24 further comprising in response to selecting the one identified first data object from the collection of identified first data objects, logically removing the selected one identified first data object from the collection of identified first data objects.

26. The method of claim 24 further comprising iteratively identifying an additional other first data object and adding the identified additional first data object to the collection of identified first data objects.

27. The method of claim 26 wherein the collection of identified first data objects has a maximum size.

28. The method of claim 27 further comprising adjusting the maximum size of the collection of identified first data objects as a function of at least one of a rate that the first additional first data objects are added to the collection of identified first data objects, and a rate that the identified first data objects of the collection of identified first data objects are selected for analysis by the first DAT.

29. The method of claim 26 wherein adding the identified additional first data object to the collection of identified first data objects comprises concurrently adding the identified additional first data object to the collection of identified first data objects.

30. The method of claim 1 further comprising assigning a sequence number to each of the identified first data objects of the collection of identified data objects, wherein the assigned sequence number defines an approximate order for analysis by the first DAT.

31. The method of claim 30 wherein at least two of the identified first data objects of the collection of identified first data objects have the same assigned sequence number.

32. A system for self-assembling a plurality of individual data analysis tools into a workflow, the system comprising:

at least one processor;

at least one data storage device storing computer executable instructions for execution by the processor;

said data storage device;

storing a plurality of data attributes for defining one or more data objects;

storing in the data store a first data object having a first attribute set comprising at least one first data attribute of the plurality of stored data attributes and a value for said at least one first data attribute;

storing in the data store a first data analysis tool (DAT), said first DAT associated with at least one first reference data attribute of the plurality of stored data attributes;

said computer executable instructions comprising;

instructions for identifying said first data object based on each at least one first reference data attribute matching one of the at least one first data attribute of the first data attribute set and independent of the value for the at least one first data attribute of the first attribute set;

instructions for generating by said first DAT a second data object as a function of the identified first data object, wherein said second data object comprises a second attribute set comprising at least one second data attribute of the plurality of stored data attributes, and a value for said at least one second data attribute of the second attribute set;

instructions for storing the second data object in the data store;

instructions for storing in the data store a second data analysis tool (DAT), said second DAT associated with at least one second reference data attribute of the plurality of stored data attributes;

instructions for identifying said second data object based on each at least one second reference data attribute matching one of the at least one second data attribute of the second data attribute set and independent of the value for the at least one second data attribute of the first attribute set;

instructions for generating by said second DAT a third data object as a function of the identified second data object, wherein said third data object comprises a third attribute set comprising at least one third data attribute of the plurality of stored data attributes, and a value for said at least one third data attribute of the third attribute set;

instructions for storing said third data object in the data store;

instructions for adding a new data object to the data store, said new data object having a particular set of data attributes;

in response to said adding the new data object to the data store;

instructions for determining whether each at least one first reference data attribute matches one of the data attributes of the new data object, and

instructions for determining whether each at least one second reference data attribute matches one of the data attributes of the new data object; and

instructions for assigning a sequence number to each of the identified first data objects of the collection of identified first data objects, wherein the assigned sequence number defines an approximate order for analysis by the first DAT.

33. The system of claim 32, wherein said instructions for identifying said first data object comprises instructions for determining that each of the at least one first reference data attribute is identical to or has a semantic lexical relationship with one of the at least one first reference data attribute.

34. The system of claim 32, said computer executable instructions further comprising:

instructions for mapping a plurality of the stored data attributes together;

instructions for determining whether each at least one first reference data attribute is mapped to one of the at least one first data attribute, wherein the first data attribute matches the first reference data attribute when the first data attribute is determined to be mapped to the first reference data attribute.

35. The system of claim 32, wherein said first DAT is associated with a first reference data object, said first reference data object having the at least one first reference data attribute.

36. The system of claim 32, said computer executable instructions further comprising:

instructions for receiving a user-provided value;

instructions for querying the data store as a function of the user-provided value to identify data attributes having values corresponding to the user-provided value; and

instructions for generating a result set as a function of the querying, said result set including data objects having the identified data attributes;

wherein said receiving, querying, and generating instructions are executed by the one or more processors for each of a plurality of user-provided values.

37. The system of claim 32 wherein at least two of the identified first data objects of the collection of identified first data objects have the same assigned sequence number.

38. A tangible, non-transitory storage medium storing processor-executable instructions for self-assembling a plurality of individual data analysis tools into a workflow, said storage medium storing:

instructions for storing a plurality of data attributes for describing/defining one or more data objects in a storage device;

instructions for storing in a data store a first data object having a first attribute set comprising at least one first data attribute of the plurality of stored data attributes and a value for said at least one first data attribute;

instructions for storing in the data store a first data analysis tool (DAT), said first DAT associated with at least one first reference data attribute of the plurality of stored data attributes;

instructions for identifying said first data object based on each at least one first reference data attribute matching one of the at least one first data attribute of the first data attribute set and independent of the value for the at least one first data attribute of the first attribute set;

instructions for generating by said first DAT a second data object as a function of the identified first data object, wherein said second data object comprises a second attribute set comprising at least one second data attribute of the plurality of stored data attributes, and a value for said at least one second data attribute of the second attribute set;

instructions for storing the second data object in the data store;

instructions for storing in the data store a second data analysis tool (DAT), said second DAT associated with at least one second reference data attribute of the plurality of stored data attributes;

instructions for identifying said second data object based on each at least one second reference data attribute matching one of the at least second first data attribute of the second data attribute set and independent of the value for the at least one second data attribute of the second attribute set;

instructions for generating by said second DAT a third data object as a function of the identified second data object, wherein said third data object comprises a third attribute set comprising at least one third data attribute of the plurality of stored data attributes, and a value for said at least one third data attribute of the third attribute set;

instructions for storing said third data object in the data store;

instructions for adding a new data object to the data store, said new data object having a particular set of data attributes;

in response to said adding the new data object to the data store;

instructions for determining whether each at least one first reference data attribute matches one of the data attributes of the new data object, and

instructions for determining whether each at least one second reference data attribute matches one of the data attributes of the new data object;

instructions for receiving a user-provided value;

instructions for querying the data store as a function of the user-provided value to identify data attributes having values corresponding to the user-provided value;

instructions for generating a result as a function of the querying, said result set including data objects having the identified data attributes, wherein said receiving, querying, and generating instructions are executed by the one or more processors for each of a plurality of user-provided values; and

instructions for iteratively selecting one identified first data object from the collection of identified first data objects for analysis by the first DAT, wherein the instructions for generating by said first DAT a second object as a function of the identified first data object comprises instructions for generating by said first DAT, for each iteratively selected one identified first data object, a second data object as a function of the selected one identified first data object.

39. The storage medium of claim 38, wherein said instructions for identifying said first data object comprises instructions for determining that each of the at least one first reference data attribute is identical to or has a semantic lexical relationship with one of the at least one first reference data attribute.

40. The storage medium of claim 38, the storage medium further storing:

instructions for mapping a plurality of the stored data attributes together;

instructions for determining whether each at least one first reference data attribute is mapped to one of the at least one first data attribute, wherein the first data attribute matches the first reference data attribute when the first data attribute is determined to be mapped to the first reference data attribute.

41. The storage medium of claim 38, wherein said first DAT is associated with a first reference data object, said first reference data object having the at least one first reference data attribute.

42. The storage medium of claim 38, the storage medium further storing:

instructions for assigning a sequence number to each of the identified first data objects of the collection of identified first data objects, wherein the assigned sequence number defines an approximate order for analysis by the first DAT.

43. The storage medium of claim 42 wherein at least two of the identified first data objects of the collection of identified first data objects have the same assigned sequence number.

44. A system for self-assembling a plurality of individual data analysis tools into a workflow, the system comprising:

at least one processor;

at least one data storage device storing computer executable instructions for execution by the processor;

said data storage device;

storing a plurality of data attributes for defining one or more data objects;

storing in the data store a first data object having a first attribute set comprising at least one first data attribute of the plurality of stored data attributes and a value for said at least one first data attribute;

storing in the data store a first data analysis tool (DAT), said first DAT associated with at least one first reference data attribute of the plurality of stored data attributes;

said computer executable instructions comprising;

instructions for identifying said first data object based on each at least one first reference data attribute matching one of the at least one first data attribute of the first data attribute set and independent of the value for the at least one first data attribute of the first attribute set;

instructions for generating by said first DAT a second data object as a function of the identified first data object, wherein said second data object comprises a second attribute set comprising at least one second data attribute of the plurality of stored data attributes, and a value for said at least one second data attribute of the second attribute set;

instructions for storing the second data object in the data store;

instructions for storing in the data store a second data analysis tool (DAT), said second DAT associated with at least one second reference data attribute of the plurality of stored data attributes;

instructions for identifying said second data object based on each at least one second reference data attribute matching one of the at least one second data attribute of the second data attribute set and independent of the value for the at least one second data attribute of the first attribute set;

instructions for generating by said second DAT a third data object as a function of the identified second data object, wherein said third data object comprises a third attribute set comprising at least one third data attribute of the plurality of stored data attributes, and a value for said at least one third data attribute of the third attribute set;

instructions for storing said third data object in the data store; and

instructions for iteratively selecting one identified first data object from the collection of identified first data objects for analysis by the first DAT, wherein the instructions for generating by said first DAT a second object as a function of the identified first data object comprises instructions for generating by said first DAT, for each iteratively selected one identified first data object, a second data object as a function of the selected one identified first data object.