Systems and methods for query evaluation over distributed linked data stores

• US 10,031,922 B2
• Filed: 07/10/2015
• Issued: 07/24/2018
• Est. Priority Date: 03/24/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fokoue-Nkoutche, Achille B., Kementsietsidis, Anastasios, Kotoulas, Spyros, Rafique, Muhammad Mustafa
• Primary Examiner(s): Featherstone, Mark D
• Assistant Examiner(s): Forouharnejad, Faezeh
• Application Number:
  US14/796,776
• Publication Number:
  US 20160283551A1
• Time in Patent Office: 1,110 Days
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/24535 of sub-queries or views
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A method for query evaluation comprises receiving a query over a set of distributed data sources, decomposing the query into a set of sub-queries of the query, evaluating each sub-query in the set of sub-queries with respect to each data source in the set of distributed data sources, wherein evaluating comprises determining which data sources in the set of distributed data sources are capable of answering each sub-query and at what cost, computing a set of distributed plans by composing one or more of the sub-queries in one or more of the data sources, evaluating each plan in the set of distributed plans, selecting a sub-set of plans from the set of distributed plans to be executed for responding to the query, executing the selected sub-set of plans, and returning results of the query.

Claims

1. An apparatus, comprising:

at least one memory; and

at least one processing device operatively coupled to the at least one memory and configured to;

implement a data query evaluation system at each data source of a set of distributed and networked linked data sources, wherein the distributed and network linked data sources respectively correspond to a plurality of nodes connected over a network, wherein the data query evaluation system at each data source is configured to;

receive a query over the set of distributed and network linked data sources;

decompose the query into a set of sub-queries of the query;

evaluate each sub-query in the set of sub-queries with respect to each data source in the set of distributed and network linked data sources, wherein evaluating comprises determining which data sources in the set of distributed and network linked data sources are capable of answering each sub-query, wherein each data source of the set of distributed and networked linked data sources dynamically determines whether it is capable of answering each sub-query;

summarize results of the evaluation of each sub-query;

exchange between the distributed and networked linked data sources the summarized results of the evaluation, wherein each data source of the set of distributed and networked linked data sources dynamically provides the summarized results;

execute, in parallel by the distributed and networked linked data sources, a distributed query evaluation process to compute a plurality of combinations of query results from the distributed and networked linked data sources which provide an answer to the query;

compute a set of distributed plans by composing one or more of the sub-queries in one or more of the distributed and networked linked data sources;

evaluate each plan in the set of distributed plans;

select a sub-set of plans from the set of distributed plans to be executed for responding to the query;

execute the selected sub-set of plans, wherein executing the selected sub-set of plans comprises;

an exchange of data between each of the distributed and networked linked data sources that are part of the selected sub-set of plans;

executing a plurality of sub-queries in respective ones of the distributed and networked linked data sources that are part of the selected sub-set of plans; and

forwarding results of the execution of one or more given sub-queries of the plurality sub-queries to one or more data sources of the distributed and networked linked data sources that are part of the selected sub-set of plans, and which did not perform the one or more given sub-queries, wherein the results comprise answers and partial answers to the given sub-queries; and

return results of the query;

wherein the implementing of the data query evaluation system at each data source is performed without;

(i) transmission of queried data to a central repository;

(ii) using a central federator for coordinating and merging the partial answers from the one or more data sources of the distributed and networked linked data sources that are part of the selected sub-set of plans; and

(iii) using a global schema to evaluate the query.

2. The apparatus according to claim 1, wherein the summarized results include an identification of local statistics and remote statistics for each data source.

3. The apparatus according to claim 1, wherein the at least one processing device is further configured to determine a nature of information being provided by a data source in response to a composed sub-query for that data source, and determine a cost associated with obtaining the information.

4. The apparatus according to claim 3, wherein the cost is based on at least one of memory and time needed for the composed sub-query.

5. The apparatus according to claim 1, wherein evaluating each plan comprises comparing the set of distributed plans with each other with respect to a cost of each plan.

6. The apparatus according to claim 1, wherein the one or more of the sub-queries are composed in each data source determined to be capable of answering a sub-query.

7. The apparatus according to claim 1, wherein the query comprises a graph query and the data sources comprise graph data sources.

8. The apparatus according to claim 1, wherein the at least one processing device is further configured to compile results of the executing of the selected sub-set of plans.

9. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by at least one processor to cause the at least one processor to perform a method comprising:

implementing a data query evaluation system at each data source of a set of distributed and networked linked data sources, wherein the distributed and network linked data sources respectively correspond to a plurality of nodes connected over a network, wherein the data query evaluation system at each data source is configured for;

receiving a query over the set of distributed and network linked data sources;

decomposing the query into a set of sub-queries of the query;

evaluating each sub-query in the set of sub-queries with respect to each data source in the set of distributed and network linked data sources, wherein evaluating comprises determining which data sources in the set of distributed and network linked data sources are capable of answering each sub-query, wherein each data source of the set of distributed and networked linked data sources dynamically determines whether it is capable of answering each sub-query;

summarizing results of the evaluating of each sub-query;

exchanging between the distributed and networked linked data sources the summarized results of the evaluating, wherein each data source of the set of distributed and networked linked data sources dynamically provides the summarized results;

executing, in parallel by the distributed and networked linked data sources, a distributed query evaluation process to compute a plurality of combinations of query results from the distributed and networked linked data sources which provide an answer to the query;

computing a set of distributed plans by composing one or more of the sub-queries in one or more of the distributed and networked linked data sources;

evaluating each plan in the set of distributed plans;

selecting a sub-set of plans from the set of distributed plans to be executed for responding to the query;

executing the selected sub-set of plans, wherein executing the selected sub-set of plans comprises;

exchange of data between each of the distributed and networked linked data sources that are part of the selected sub-set of plans;

executing a plurality of sub-queries in respective ones of the distributed and networked linked data sources that are part of the selected sub-set of plans; and

forwarding results of the execution of one or more given sub-queries of the plurality sub-queries to one or more data sources of the distributed and networked linked data sources that are part of the selected sub-set of plans, and which did not perform the one or more given sub-queries, wherein the results comprise answers and partial answers to the given sub-queries; and

returning results of the query;

wherein the implementing of the data query evaluation system at each data source is performed without;

(i) transmission of queried data to a central repository;

(ii) using a central federator for coordinating and merging the partial answers from the one or more data sources of the distributed and networked linked data sources that are part of the selected sub-set of plans; and

(iii) using a global schema to evaluate the query.