Platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization

• US 10,438,013 B2
• Filed: 02/22/2017
• Issued: 10/08/2019
• Est. Priority Date: 06/19/2016
• Status: Active Grant

• Current Assignee: Data.World, Inc.
• Original Assignee: Data.World, Inc.
• Inventors: Jacob, Bryon Kristen, Griffith, David Lee, Le, Triet Minh, Reynolds, Shad William, Keen, Arthur Albert
• Primary Examiner(s): Bullock, Joshua
• Application Number:
  US15/439,911
• Publication Number:
  US 20170364694A1
• Time in Patent Office: 958 Days
• Field of Search: 707770
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/2423 Interactive query statement specification based on a database schema
• CPC Class Codes: G06F 16/24534 Query rewriting; Transformation
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

Various techniques are described for platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization, including receiving at a dataset access platform a query formatted according to a first data schema, generating a copy of the query, saving the query and the copy to a datastore, parsing the copy of the query in the first schema using an inference engine, determining whether the query comprises data associated with an access control condition associated with accessing the dataset, the access control condition being configured to indicate whether the query is permitted to access the dataset, and rewriting, using a proxy server, the copy of the query in a second schema, and optimizing the rewriting by identifying a database engine to execute the query and including other data converted into another triple associated with an attribute of the query.

Claims

1. A method, comprising:

receiving a query at a dataset access platform, the query being formatted according to a first data schema including a structured relational-based format, the query comprising data associated with a request to access a dataset;

generating a copy of the query, the copy being identified as a master and configured to be stored in a datastore;

parsing the copy of the query in the first schema, the parsing being performed by an inference engine configured to identify the dataset, to infer an attribute associated with the query, and to generate one or more data links between the dataset and another dataset accessible by the dataset access platform;

rewriting, using a proxy server, the copy of the query in a second schema including a triples-based format and, if the attribute indicates the query is configured to provide authentication data to access the dataset, the rewriting comprising converting the copy of the query into a triple and converting the attribute into another triple; and

optimizing the rewriting, the optimizing comprising identifying a database engine to execute the query and converting other data to a further triple, the other data and the further triple being associated with a path configured to route the query or the copy of the query in the second schema including the triples-based format from the dataset access platform to retrieve query results from a target database configured to store the dataset as graph-based data.

2. The method of claim 1, wherein the converting the copy and the converting the attribute are performed using a framework.

3. The method of claim 1, further comprising determining whether the attribute comprises an access control condition associated with accessing the dataset.

4. The method of claim 1, wherein the inference engine is configured to generate a graph associated with the dataset, the another dataset, the data links, and the one or more property paths, the graph being stored in the datastore or another datastore.

5. The method of claim 1, wherein the database engine is configured to be topologically internal to a data network associated with the dataset access platform.

6. The method of claim 1, wherein the optimizing also comprises determining one or more property paths to the another dataset.

7. The method of claim 6, wherein the one or more property paths are determined by performing a comparison of another attribute associated with each of the one or more property paths to a threshold to identify an optimal path to run the query or the copy of the query.

8. The method of claim 1, wherein the access control condition is configured to authenticate the query to permit access to the dataset.

9. The method of claim 1, wherein the triple and the another triple are configured to be included in a rewritten query generated from the rewriting, the rewritten query being configured to be processed by one or more endpoints associated with the dataset access platform.

10. The method of claim 9, wherein at least one of the one or more endpoints comprise the database engine, the database engine being configured to execute the query, the copy of the query, or the rewritten query using the framework to perform data communication with the dataset or another data network topologically associated with the dataset.

11. The method of claim 1, wherein the database engine is configured to communicate data between the dataset access platform and another data network associated with the dataset.

12. The method of claim 1, wherein the database engine is configured to use the triple, the another triple, and the further triple to execute the copy to access the dataset and, after accessing the dataset, returning responsive data to the dataset access platform.

13. The method of claim 1, wherein the first schema including the structured relational-based format includes SQL query data.

14. The method of claim 1, wherein parsing the copy of the query comprises:

parsing the query into a data structure including an abstract syntax tree associated with a target query language.

15. A non-transitory computer readable medium having one or more computer program instructions configured to perform a method, the method comprising:

receiving a query at a dataset access platform, the query being formatted according to a first data schema including a structured relational-based format, the query comprising data associated with a request to access a dataset;

generating a copy of the query, the copy being identified as a master and configured to be stored in a datastore;

parsing the copy of the query in the first schema, the parsing being performed by an inference engine configured to identify the dataset, to infer an attribute associated with the query, and to generate one or more data links between the dataset and another dataset accessible by the dataset access platform;

rewriting, using a proxy server, the copy of the query in a second schema including a triples-based format and, if the attribute indicates the query is configured to provide authentication data to access the dataset, the rewriting comprising converting the copy of the query into a triple and converting the attribute into another triple; and

optimizing the rewriting, the optimizing comprising identifying a database engine to execute the query and converting other data to a further triple, the other data and the further triple being associated with a path configured to route the query or the copy of the query in the second schema including the triples-based format from the dataset access platform to retrieve query results from a target database configured to store the dataset as graph-based data.