SOURCE QUERY CACHING AS FAULT PREVENTION FOR FEDERATED QUERIES

• US 20150242464A1
• Filed: 02/24/2014
• Published: 08/27/2015
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Original Assignee: Red Hat, Inc. (International Business Machines Corporation)
• Inventors: Nguyen, Filip, Elis, Filip
• Granted Patent: US 10,114,874 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0775 Content or structure details of the error report, e.g. specific table structure, specific error fields
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 11/30 Monitoring
• CPC Class Codes: G06F 16/24539 using cached or materialised query results
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

An example system for processing a federated query includes a query proxy that receives a federated query including a plurality of source queries and receives an indication that a failed set of one or more source queries failed to execute successfully. Each source query is specific to an autonomous data source belonging to a federation. The system also includes a data federation engine that identifies a plurality of autonomous data sources to which to send the plurality of source queries. The plurality of autonomous data sources belong to the federation. The system further includes a query fail analyzer that updates a data structure to reflect the unsuccessful execution of one or more source queries of the failed set.

Claims

1. A method of processing a federated query, comprising:

receiving at a query proxy a federated query including a plurality of source queries, each source query of the plurality of source queries being specific to an autonomous data source belonging to a federation;

identifying a plurality of target autonomous data sources to which to send the plurality of source queries, the plurality of target autonomous data sources belonging to the federation;

receiving an indication that a failed set of one or more source queries of the plurality of source queries failed to execute successfully; and

after receiving the indication, updating a data structure to reflect the unsuccessful execution of one or more source queries of the failed set.

2. The method of claim 1, further including:

determining whether one or more source queries of the failed set of source queries is stored in the data structure,

wherein when a first subset of the failed set is determined to not be stored in the data structure, the updating includes inserting into the data structure one or more entries including a source query of the first subset and metadata of the source query, the metadata including a federated query identifier and a data source identifier, the federated query identifier identifying the federated query, and the data source identifier identifying a target data source to which the source query was sent, and

when a second subset of the failed set is determined to be stored in the data structure, the updating includes updating metadata of the one or more entries corresponding to the second subset stored in the data structure, wherein the updating metadata includes updating a number of times the respective source query has failed.

3. The method of claim 2, further including:

determining a third subset of one or more failed source queries stored in the data structure, the determining being based on the metadata, and each source query of the third subset including one or more source tables; and

for each source query of the third subset;

submitting the respective source query to a target data source specific to the respective source query; and

retrieving a result of the respective source query from the target data source; and

storing the result and respective source query into the cache.

4. The method of claim 3, wherein for each source query of the third subset, the method further includes:

identifying a source table included in the respective source query;

determining a cache table name based on a name of the identified source table, wherein the storing includes storing the result in a table identified by the cache table name.

5. The method of claim 4, wherein the determining a cache table name includes identifying a source table name of the identified source table and adding a suffix to the source table name.

6. The method of claim 4, wherein the receiving at a query proxy a federated query includes receiving the first federated query from a client, the method further including:

determining whether one or more data sources of the plurality of target autonomous data sources is unavailable;

when one or more data sources of the plurality of target autonomous data sources is determined to be unavailable;

obtaining a second federated query, the second federated query being different from the first query and including a second plurality of source queries, wherein at least one source query of the second plurality of source queries is specific to a target data source that is not a target data source of the first plurality of source queries; and

sending a communication that causes each source query included in the second plurality of source queries to be sent to a target data source specific to the respective source query;

receiving one or more results of the executed second plurality of source queries; and

sending the result to the client, the result being responsive to the first federated query.

7. The method of claim 6, wherein the second federated query includes one or more rewritten names of source tables included in the first federated query, and wherein the one or more rewritten names of source tables corresponds to information stored in tables that are unavailable.

8. The method of claim 4, wherein the obtaining a second federated query includes for each unavailable data source:

identifying one or more source queries embedded in the first federated query that is specific to the respective unavailable data source;

rewriting each of the identified one or more source queries embedded in the first federated query; and

replacing the identified one or more source queries in the first federated query with one or more corresponding rewritten source queries, wherein the obtaining a second federated query includes generating the second federated query based on the identifying, rewriting, and replacing.

9. The method of claim 8, wherein the rewriting includes changing one or more source table names included in the identified one or more source queries.

10. The method of claim 9, wherein the changing one or more source table names includes appending a suffix to the respective source table name.

11. The method of claim 1, wherein the receiving a federated query includes receiving a federated query from a client, the method further including:

after the receiving an indication, sending to the client a message indicating that execution of the federated query was unsuccessful.

12. The method of claim 1, wherein the receiving at a query proxy a federated query includes intercepting the federated query.

13. A system for processing a federated query, comprising:

a query proxy that receives a federated query including a plurality of source queries and receives an indication that a failed set of one or more source queries of the plurality of source queries failed to execute successfully, wherein each source query of the plurality of source queries is specific to an autonomous data source belonging to a federation;

a data federation engine that identifies a plurality of autonomous data sources to which to send the plurality of source queries, wherein the plurality of autonomous data sources belong to the federation;

a query fail analyzer that updates a data structure to reflect the unsuccessful execution of one or more source queries of the failed set; and

a federated server coupled over a network, wherein the federated server executes at least one of the query proxy, the data federation engine, and the query fail analyzer.

14. The system of claim 13, further including:

a periodic cache manager that determines, based on the metadata, a third subset of one or more failed source queries stored in the data structure, wherein each source query of the third subset includes one or more source tables, and wherein for each source query of the third subset, the periodic cache manager submits the respective source query to a target data source specific to the respective source query, retrieves a result of the respective source query from the target data source, and stores the result and respective source query into the cache.

15. The system of claim 14, wherein for each source query of the third subset, the periodic cache manager identifies a source table included in the respective source query and determines a cache table name based on a name of the identified source table, and wherein the periodic cache manager stores the result in a table identified by the cache table name.

16. The system of claim 14, wherein the federated query is sent from a client, and wherein the query proxy determines whether one or more data sources of the plurality of target autonomous data sources is unavailable, the system further including:

a query rewriter that when one or more data sources of the plurality of target autonomous data sources is determined to be unavailable, the query rewriter obtains a second federated query, wherein the second federated query is different from the first query and includes a second plurality of source queries, and wherein at least one source query of the second plurality of source queries is specific to a target data source that is not a target data source of the first plurality of source queries,

wherein the query proxy sends a communication that causes each source query included in the second plurality of source queries to be sent to a target data source specific to the respective source query, receives one or more results of the executed second plurality of source queries, and sends the result to the client, wherein the result is responsive to the first federated query.

17. The system of claim 16, wherein the second federated query includes one or more rewritten names of source tables included in the first federated query, and wherein the one or more rewritten names of source tables corresponds to information stored in tables that are unavailable.

18. The system of claim 17, wherein the query rewriter identifies one or more source queries embedded in the first federated query that is specific to the respective unavailable autonomous data source, rewrites each of the identified one or more source queries embedded in the first federated query, and replaces the identified one or more source queries in the first federated query with one or more corresponding rewritten source queries to determine the second federated query.

19. The method of claim 18, wherein the query rewriter changes one or more source table names included in the identified one or more source queries.

20. A machine-readable medium comprising a plurality of machine-readable instructions that when executed by one or more processors is adapted to cause the one or more processors to perform a method comprising:

receiving a federated query including a plurality of source queries, each source query of the plurality of source queries being specific to an autonomous data source belonging to a federation;

identifying a plurality of target autonomous data sources to which to send the plurality of source queries, the plurality of target autonomous data sources belonging to the federation;

receiving an indication that a failed set of one or more source queries of the plurality of source queries failed to execute successfully; and

after receiving the indication, updating a data structure to reflect the unsuccessful execution of one or more source queries of the failed set.