PLATFORM MANAGEMENT OF INTEGRATED ACCESS OF PUBLIC AND PRIVATELY-ACCESSIBLE DATASETS UTILIZING FEDERATED QUERY GENERATION AND QUERY SCHEMA REWRITING OPTIMIZATION

• US 20170364539A1
• Filed: 02/22/2017
• Published: 12/21/2017
• Est. Priority Date: 06/19/2016
• Status: Active Grant

• Current Assignee: Data.World, Inc.
• Original Assignee: Data.World, Inc.
• Inventors: Jacob, Bryon Kristen, Griffith, David Lee, Le, Triet Minh, Reynolds, Shad William, Keen, Arthur Albert
• Granted Patent: US 10,452,975 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/24547 Optimisations to support specific applications; Extensibility of optimisers
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

Various techniques are described for platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization, including receiving at a dataset access platform a query formatted according to a first data schema, generating a copy of the query, saving the query and the copy to a datastore, parsing the copy of the query in the first schema using an inference engine, determining whether the query comprises data associated with an access control condition associated with accessing the dataset, the access control condition being configured to indicate whether the query is permitted to access the dataset, and rewriting, using a proxy server, the copy of the query in a second schema by converting the copy of the query into a triple associated with the query and another triple associated with the access control condition.

Claims

1. A method, comprising:

receiving at a dataset access platform a query formatted using a first schema, the query comprising data associated with a request to access a dataset;

generating a copy of the query;

saving the query and the copy to a datastore, the query being identified as a master;

parsing the copy of the query in the first schema, the parsing being performed by an inference engine to infer the first schema and an attribute associated with the query, and to generate a graph having one or more data links between the dataset and another dataset accessible by the dataset access platform;

determining, during the parsing, whether the query comprises other data associated with an access control condition associated with accessing the dataset, the access control condition being configured to indicate whether the query is configured to provide authentication data to access the dataset; and

rewriting, using a proxy server, the copy of the query in a second schema, if the access control condition indicates the query is configured to authenticate access to the dataset, the rewriting comprising converting, using a framework, the copy of the query into a triple associated with the query and another triple associated with the other data, the triple and the another triple being included in a rewritten query directed to one or more endpoints associated with the dataset access platform.

2. The method of claim 1, further comprising creating a triple file associated with the triple and the another triple, the triple file also including other data associated with a graph, the graph being associated with the query and the attribute.

3. The method of claim 1, further comprising creating a triple file associated with the triple and the another triple, the triple file also being configured to indicate one or more property paths associated with the dataset and linked data associated with the dataset.

4. The method of claim 1, wherein the query comprises a pre-signed uniform resource locator address.

5. The method of claim 1, wherein the query comprises a pre-signed uniform resource indicator.

6. The method of claim 1, wherein the proxy server is configured to convert the query from the first schema to the second schema using a protocol, the protocol being SPARQL.

7. The method of claim 1, wherein the rewriting is performed using resource description framework to model one or more data relationships of the query.

8. The method of claim 1, wherein the query comprises a publicly-accessible dataset and a privately accessible dataset, the privately-accessible dataset being configured to permit access if the access control condition is equal to another condition

9. The method of claim 8, wherein the another condition is configured to be controlled by the privately-accessible dataset.

10. The method of claim 1, wherein the rewriting of the copy of the query is configured to generate a federated query that is configured to access one or more endpoints of the dataset access platform and one or more database engines associated with the dataset.

11. The method of claim 1, wherein the attribute, during inference, indicates the query does not include an access control condition configured to access the dataset.

12. The method of claim 1, wherein the attribute, during inference, indicates the dataset is privately-accessible and the access control condition, is configured to access the dataset.

13. The method of claim 1, further comprising federating the copy of the query rewritten in the second schema, the federated query comprising a triple pattern generated using the second schema and other data associated with accessing one or more access-controlled datasets, the one or more access-controlled datasets being stored on one or more data networks.

14. The method of claim 13, wherein each of the one or more endpoints is configured to access at least one of the one or more data networks.

15. The method of claim 13, wherein each of the one or more endpoints is configured to access at least one of the one or more access-controlled datasets.

16. A system, comprising:

a database configured to store data associated with a request to access a dataset and other data associated with an access control condition associated with accessing the dataset; and

an application platform comprising logic configured to receive a query formatted using a first schema, the query comprising the data, to generate a copy of the query, to save the query and the copy to a datastore, the query being identified as a master, to parse the copy of the query in the first schema, the parsing being performed by an inference engine, the parsing being configured to identify the dataset, to infer an attribute associated with the query, and to generate one or more data links between the dataset and another dataset accessible by the dataset access platform, to determine, during the parsing, whether the query comprises the other data, the access control condition being configured to indicate whether the query is configured to provide authentication data to access the dataset, and to rewrite, using a proxy server, the copy of the query in a second schema, if the access control condition indicates the query is configured to authenticate access to the dataset, the rewriting comprising converting, using a framework, the copy of the query into a triple associated with the query and another triple associated with the other data, the triple and the another triple being included in a rewritten query directed to one or more endpoints associated with the dataset access platform.

17. The system of claim 16, wherein the database comprises one or more datastores.

18. The system of claim 16, wherein the inference engine is configured to determine one or more linked datasets that are linked to the dataset requested to be accessed by the query.

19. The system of claim 16, wherein the inference engine is configured to determine one or more linked datasets that are logically linked by one or more property paths to the dataset requested to be accessed by the query, and storing one or more triples associated with the one or more linked datasets, the attribute, and the one or more property paths within a triple file comprising the triple and the another triple, the triple file being configured to be stored in one or more other datastores.

20. A non-transitory computer readable medium having one or more computer program instructions configured to perform a method, the method comprising:

receiving at a dataset access platform a query formatted using a first schema, the query comprising data associated with a request to access a dataset;

generating a copy of the query;

saving the query and the copy to a datastore, the query being identified as a master;

parsing the copy of the query in the first schema, the parsing being performed by an inference engine, the parsing being configured to identify the dataset, to infer an attribute associated with the query, and to generate one or more data links between the dataset and another dataset accessible by the dataset access platform;

determining, during the parsing, whether the query comprises other data associated with an access control condition associated with accessing the dataset, the access control condition being configured to indicate whether the query is configured to provide authentication data to access the dataset; and

rewriting, using a proxy server, the copy of the query in a second schema, if the access control condition indicates the query is configured to authenticate access to the dataset, the rewriting comprising converting, using a framework, the copy of the query into a triple associated with the query and another triple associated with the other data, the triple and the another triple being included in a rewritten query directed to one or more endpoints associated with the dataset access platform.