QUERYING A DATA SOURCE ON A NETWORK

• US 20160239582A1
• Filed: 06/26/2015
• Published: 08/18/2016
• Est. Priority Date: 02/18/2015
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Schechter, Ian, Allin, Glenn John
• Granted Patent: US 10,417,281 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence

Abstract

Among other things, we describe a computer-implemented method, performed by a data processing system, of executing a computer program based on a query that is expressed in accordance with a query language applicable to a relational database, the computer program executed based at least in part on data stored in a tangible, non-transitory computer-readable medium, the executing including receiving a SQL query, where the SQL query includes an identifier associated with a resource that is external to the data processing system, generating a computer program based on the SQL query, and executing the computer program, causing transmitting one or more instructions to the resource, the instructions defining operations other than operations of the SQL query, and receiving data from the resource in response to the instructions.

Claims

1. A computer-implemented method, performed by a data processing system, of executing a computer program based on a query that is expressed in accordance with a query language applicable to a relational database, the computer program executed based at least in part on data stored in a tangible, non-transitory computer-readable medium, the executing including:

receiving a SQL query, where the SQL query includes an identifier associated with a resource that is external to the data processing system;

generating a computer program based on the SQL query, including;

identifying configuration data associated with the resource, the configuration data specifying at least one value used in communicating with the resource; and

executing the computer program, based on the configuration data, the execution of the computer program causing;

transmitting one or more instructions to the resource, the instructions defining operations other than operations of the SQL query, and

receiving data from the resource in response to the instructions.

2. The method of claim 1 in which the portion of the computer program includes executable instructions that define a manner in which the resource is accessed, where the executable instructions operate based on the configuration data provided to the portion of the computer program.

3. The method of claim 1 in which the configuration data is specified in a catalog that can be updated based on a change to a data format used by the resource.

4. The method of claim 1 including

generating parameter values based on the configuration data;

and

providing the parameter values to a portion of the computer program, the portion being capable of communicating with the resource;

the execution of the computer program being based on the parameter values.

5. The method of claim 4 in which generating parameter values based on the configuration data includes executing a parameter generator which generates a parameter file in a format readable by the portion of the computer program, and

in which providing the parameter values to the portion of the computer program includes making the parameter file available to the portion of the computer program.

6. The method of claim 1 in which the SQL query includes a SELECT statement that includes an argument, where at least a portion of the argument corresponds to the identifier associated with the resource.

7. The method of claim 1 in which the computer program includes components representing operations of the SQL query.

8. The method of claim 1 in which the computer program is a dataflow graph and the portion of the computer program is a subgraph of the dataflow graph.

9. The method of claim 1 in which the resource is accessed using an application programming interface (API) exposed by the resource.

10. The method of claim 9 including

causing functions of the API to be executed in response to receiving the SQL query,

the functions of the API being executable at the resource, and the instructions transmitted to the resource causing the functions of the API to be executed.

11. The method of claim 1 including formatting the data received from the external resource in the form of a database table.

12. The method of claim 1 in which the resource is not a relational database management system.

13. The method of claim 1 in which the instructions are transmitted to a facility of the resource that does not return results in response to a SQL query.

14. The method of claim 1 in which the data received from the resource in response to the instructions includes data specified by the SQL query.

15. The method of claim 1 including identifying records and fields in the data received from the resource in response to the instructions, the records and fields identified based on a record format associated with the resource that is external to the data processing system.

16. The method of claim 1 in which generating the computer program based on the SQL query includes:

identifying a data source based on the SQL query;

identifying an executing system other than a system managing a relational database;

generating a request to a query planner based on the query;

providing the request to the query planner;

receiving a query plan generated by the query planner based on the request, the query plan including a description of one or more steps to be performed by a system managing a relational database;

generating a data structure instantiating a dataflow graph that includes;

a first node that represents at least one operation to be executed,

the first node associated with information usable by an executing system to invoke executable program code to perform the operation,

the first node associated with information usable by an executing system to make data available to the program code, and

the operation chosen based on an step described by the query plan, and

at least one link that represents output data of an operation of the first node being provided as input data to an operation of a second node of the dataflow graph.

17. A data processing system capable of executing a computer program based on a query that is expressed in accordance with a query language applicable to a relational database, the computer program executed based at least in part on data stored in a tangible, non-transitory computer-readable medium, the data processing system configured to perform operations including:

receiving a SQL query, where the SQL query includes an identifier associated with a resource that is external to the data processing system;

generating a computer program based on the SQL query, including;

identifying configuration data associated with the resource, the configuration data specifying at least one value used in communicating with the resource; and

executing the computer program, based on the configuration data, the execution of the computer program causing;

transmitting one or more instructions to the resource, the instructions defining operations other than operations of the SQL query, and

receiving data from the resource in response to the instructions.

18. A non-transitory computer readable storage device storing instructions that enable a data processing system to execute a computer program based on a query that is expressed in accordance with a query language applicable to a relational database, the computer program executed based at least in part on data stored in a tangible, non-transitory computer-readable medium, the instructions causing the data processing system to perform operations including:

receiving a SQL query, where the SQL query includes an identifier associated with a resource that is external to the data processing system;

generating a computer program based on the SQL query, including;

identifying configuration data associated with the resource, the configuration data specifying at least one value used in communicating with the resource; and

executing the computer program, based on the configuration data, the execution of the computer program causing;

transmitting one or more instructions to the resource, the instructions defining operations other than operations of the SQL query, and

receiving data from the resource in response to the instructions.

19. A data processing system capable of executing a computer program based on a query that is expressed in accordance with a query language applicable to a relational database, the computer program executed based at least in part on data stored in a tangible, non-transitory computer-readable medium, the data processing system including:

means for receiving a SQL query, where the SQL query includes an identifier associated with a resource that is external to the data processing system;

means for generating a computer program based on the SQL query, including;

identifying configuration data associated with the resource, the configuration data specifying at least one value used in communicating with the resource; and

means for executing the computer program, based on the configuration data, the execution of the computer program causing;

transmitting one or more instructions to the resource, the instructions defining operations other than operations of the SQL query, and

receiving data from the resource in response to the instructions.