DATA VIRTUALIZATION ACROSS HETEROGENEOUS FORMATS

• US 20160055184A1
• Filed: 08/25/2014
• Published: 02/25/2016
• Est. Priority Date: 08/25/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: FOKOUE-NKOUTCHE, Achille Belly, SRINIVAS, Kavitha, HASSANZADEH, Oktie, KEMENTSIETSIDIS, Anastasios, WARD, Michael J.
• Granted Patent: US 10,740,304 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof

Abstract

Various embodiments virtualize data across heterogeneous formats. In one embodiment, a plurality of heterogeneous data sources is received as input. A local schema graph including a set of attribute nodes and a set of type nodes is generated for each of the plurality of heterogeneous data sources. A global schema graph is generated based on each local schema graph that has been generated. The global schema graph comprises each of the local schema graphs and at least one edge between at least one of two or more attributes nodes and two or more type nodes from different local schema graphs. The edge indicates a relationship between the data sources represented by the different local schema graphs comprising the two or more attributes nodes based on a computed similarity between at least one value associated with each of the two or more attributes nodes.

Claims

1. A method for virtualizing data across heterogeneous formats, the method comprising:

receiving, as input, a plurality of heterogeneous data sources;

generating, for each of the plurality of heterogeneous data sources, a local schema graph comprising a set of attribute nodes and a set of type nodes, wherein an attribute node corresponds to a schema element in the heterogeneous data source comprising a domain with at least one value and is annotated with the value in the local schema graph, and wherein a type node corresponds to a schema element in the heterogeneous data source whose domain is defined recursively through at least one of one or more attribute nodes and one or more other type nodes; and

generating a global schema graph based on each local schema graph that has been generated, wherein the global schema graph comprises each of the local schema graphs and at least one edge between at least one of two or more attributes nodes and two or more type nodes from different local schema graphs, and wherein the edge indicates a relationship between the data sources represented by the different local schema graphs comprising the two or more attributes nodes based on a computed similarity between at least one value associated with each of the two or more attributes nodes.

2. The method of claim 1, wherein each of the plurality of heterogeneous data sources comprises at least one of:

structured data;

semi-structured data;

relational data; and

Resource Description Framework-based data.

3. The method of claim 1, wherein generating the local schema graph comprises:

determining that the heterogeneous data source comprises one of structured and semi-structured data;

identifying a plurality of elements in the one of structured and semi-structured data and a value associated with each element;

generating at least one root node, wherein the root node is a type node and represents the heterogeneous data source;

generating a node for each of the plurality of elements, wherein the node is one of an attribute node and a type node; and

annotating each attribute node with the value associated with the element represented by the attribute node.

4. The method of claim 1, wherein generating the local schema graph comprises:

determining that the heterogeneous data source comprises relational data;

identifying at least one table in the relational data;

identifying each column within the table;

generating a type node representing the table;

generating an attribute node representing each column within the table;

annotating each attribute with at least at least one value in a record of the table; and

generating an edge between each attribute node and the type node.

5. The method of claim 4, wherein generating the local schema graph further comprises:

identifying at least one additional table in the relational data;

generating at least one additional type node representing the additional table; and

generating an edge between the type node and the additional type node based on at least one foreign key relationship between the table and the additional table.

6. The method of claim 1, wherein generating the local schema graph further comprises:

determining that the heterogeneous data source comprises Resource Description Framework-based data;

identifying at least one class in the Resource Description Framework-based data;

identifying each property with at least one value in the Resource Description Framework-based data associated with the class;

generating a type node for the class;

generating an attribute node for each property that has been identified; and

annotating each attribute node with the at least one value associated with the property represented by the attribute node.

7. The method of claim 1, further comprising:

tokenizing and indexing each value from each local schema graph; and

associating a uniform resource identifier with each value from each local schema graph based on the tokenizing and indexing.

8. The method of claim 1, wherein generating a global schema graph further comprises:

annotating the edge with a measure of the similarity.

9. The method of claim 1, wherein generating the local schema graph further comprises:

associating at least one uniform resource identifier to each of the set of attribute nodes and the set of type nodes.

10. An information processing system for virtualizing data across heterogeneous formats, the information processing system comprising:

a memory;

a processor communicatively coupled to the memory; and

a data processor communicatively coupled to the memory and the processor, wherein the data processor is configured to perform a method comprising;

receiving, as input, a plurality of heterogeneous data sources;

generating, for each of the plurality of heterogeneous data sources, a local schema graph comprising a set of attribute nodes and a set of type nodes, wherein an attribute node corresponds to a schema element in the heterogeneous data source comprising a domain with at least one value and is annotated with the value in the local schema graph, and wherein a type node corresponds to a schema element in the heterogeneous data source whose domain is defined recursively through at least one of one or more attribute nodes and one or more other type nodes; and

generating a global schema graph based on each local schema graph that has been generated, wherein the global schema graph comprises each of the local schema graphs and at least one edge between at least one of two or more attributes nodes and two or more type nodes from different local schema graphs, and wherein the edge indicates a relationship between the data sources represented by the different local schema graphs comprising the two or more attributes nodes based on a computed similarity between at least one value associated with each of the two or more attributes nodes.

11. The information processing system of claim 10, wherein generating the local schema graph comprises:

determining that the heterogeneous data source comprises one of structured and semi-structured data;

identifying a plurality of elements in the one of structured and semi-structured data and a value associated with each element;

generating at least one root node, wherein the root node is a type node and represents the heterogeneous data source;

generating a node for each of the plurality of elements, wherein the node is one of an attribute node and a type node; and

annotating each attribute node with the value associated with the element represented by the attribute node.

12. The information processing system of claim 10, wherein generating the local schema graph comprises:

determining that the heterogeneous data source comprises relational data;

identifying at least one table in the relational data;

identifying each column within the table;

generating a type node representing the table;

generating an attribute node representing each column within the table;

annotating each attribute with at least at least one value in a record of the table; and

generating an edge between each attribute node and the type node.

13. The information processing system of claim 12, wherein generating the local schema graph further comprises:

identifying at least one additional table in the relational data;

generating at least one additional type node representing the additional table; and

generating an edge between the type node and the additional type node based on at least one foreign key relationship between the table and the additional table.

14. The information processing system of claim 10, wherein generating the local schema graph further comprises:

determining that the heterogeneous data source comprises Resource Description Framework-based data;

identifying at least one class in the Resource Description Framework-based data;

identifying each property with at least one value in the Resource Description Framework-based data associated with the class;

generating a type node for the class;

generating an attribute node for each property that has been identified; and

annotating each attribute node with the at least one value associated with the property represented by the attribute node.

15. The information processing system of claim 10, wherein the method further comprises:

tokenizing and indexing each value from each local schema graph; and

associating a uniform resource identifier with each value from each local schema graph based on the tokenizing and indexing.

16. A computer program product for virtualizing data across heterogeneous formats, the computer program product comprising:

a storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

receiving, as input, a plurality of heterogeneous data sources;

generating, for each of the plurality of heterogeneous data sources, a local schema graph comprising a set of attribute nodes and a set of type nodes, wherein an attribute node corresponds to a schema element in the heterogeneous data source comprising a domain with at least one value and is annotated with the value in the local schema graph, and wherein a type node corresponds to a schema element in the heterogeneous data source whose domain is defined recursively through at least one of one or more attribute nodes and one or more other type nodes; and

generating a global schema graph based on each local schema graph that has been generated, wherein the global schema graph comprises each of the local schema graphs and at least one edge between at least one of two or more attributes nodes and two or more type nodes from different local schema graphs, and wherein the edge indicates a relationship between the data sources represented by the different local schema graphs comprising the two or more attributes nodes based on a computed similarity between at least one value associated with each of the two or more attributes nodes.

17. The computer program product of claim 16, wherein generating the local schema graph comprises:

determining that the heterogeneous data source comprises one of structured and semi-structured data;

identifying a plurality of elements in the one of structured and semi-structured data and a value associated with each element;

generating at least one root node, wherein the root node is a type node and represents the heterogeneous data source;

generating a node for each of the plurality of elements, wherein the node is one of an attribute node and a type node; and

annotating each attribute node with the value associated with the element represented by the attribute node.

18. The computer program product of claim 16, wherein generating the local schema graph comprises:

determining that the heterogeneous data source comprises relational data;

identifying at least one table in the relational data;

identifying each column within the table;

generating a type node representing the table;

generating an attribute node representing each column within the table;

annotating each attribute with at least at least one value in a record of the table; and

generating an edge between each attribute node and the type node.

19. The computer program product of claim 18, wherein generating the local schema graph further comprises:

identifying at least one additional table in the relational data;

generating at least one additional type node representing the additional table; and

generating an edge between the type node and the additional type node based on at least one foreign key relationship between the table and the additional table.

20. The computer program product of claim 16, wherein generating the local schema graph further comprises:

determining that the heterogeneous data source comprises Resource Description Framework-based data;

identifying at least one class in the Resource Description Framework-based data;

identifying each property with at least one value in the Resource Description Framework-based data associated with the class;

generating a type node for the class;

generating an attribute node for each property that has been identified; and

annotating each attribute node with the at least one value associated with the property represented by the attribute node.