QUERY GENERATION FOR COLLABORATIVE DATASETS

• US 20170364553A1
• Filed: 06/19/2016
• Published: 12/21/2017
• Est. Priority Date: 06/19/2016
• Status: Active Grant

• Current Assignee: Data.World, Inc.
• Original Assignee: Data.World, Inc.
• Inventors: Jacob, Bryon Capacity, Griffith, David Lee, Le, Triet Minh, Loyens, Jon, Hurt, Brett A., Keen, Arthur Albert
• Granted Patent: US 10,324,925 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24535 of sub-queries or views
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

Various embodiments relate generally to data science and data analysis, computer software and systems, and wired and wireless network communications to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby a collaborative data layer and associated logic facilitate, for example, efficient access to, and implementation of, collaborative datasets. In some examples, a method may include receiving data representing a query of a consolidated dataset that may include datasets formatted atomized datasets, analyzing the query to classify portions of the query to form classified query portions, partitioning the query into sub-queries as a function of a classification type for each of the classified query portions, and retrieving data representing a query result from distributed data repositories.

Claims

1. A method comprising:

receiving, into a collaborative dataset consolidation system, data representing a query of a consolidated dataset comprising a plurality of datasets formatted as plurality of atomized datasets;

analyzing the query to classify portions of the query to form classified query portions;

partitioning the query into a plurality of sub-queries as a function of a classification type for each of the classified query portions; and

retrieving data representing a query result from distributed data repositories.

2. The method of claim 1 further comprising:

rewriting the query as a plurality of sub-queries.

3. The method of claim 1 wherein receiving the data representing the query comprises:

receiving data representing a federated query.

4. The method of claim 3 wherein receiving the data representing the federated query comprises:

receiving requests to access different computer-based repositories.

5. The method of claim 1 further comprising:

identifying a number of atomized datasets associated with the query.

6. The method of claim 5 further comprising:

implementing per-dataset permissions to determine whether a user account is authorized to query each of the atomized datasets.

7. The method of claim 5 further comprising:

identifying an identifier associated with the query;

determining authorization data between the identifier and each of atomized datasets in the number of atomized datasets; and

granting authorization to apply the query to the number of atomized datasets based on authorization to each of the atomized datasets.

8. The method of claim 1 wherein partitioning the query into the plurality of sub-queries comprises:

identifying the classification type of a sub-query identifies a type of repository; and

transmitting the sub-query to a repository identified as the type of repository.

9. The method of claim 8 wherein the type of repository is a function of an architecture influencing operational characteristics of the repository.

10. The method of claim 1 wherein partitioning the query into the plurality of sub-queries comprises:

identifying the classification type of a sub-query identifies an external repository that is external to the collaborative dataset consolidation system; and

transmitting the sub-query to the external repository.

11. The method of claim 10 wherein identifying the classification type identifies the external repository comprises:

determine that the query includes a dataset disposed in the external repository and linked to other atomized datasets associated with the query.

12. The method of claim 1 wherein partitioning the query into the plurality of sub-queries comprises:

identifying the classification type of a sub-query identifies a type of query;

identifying a repository to receive the sub-query having the type of query; and

transmitting the sub-query to a repository having the repository.

13. The method of claim 12 wherein identifying the classification type identifies the external repository comprises:

determine that the query includes a dataset disposed in the external repository and linked to other atomized datasets associated with the query.

14. The method of claim 1 wherein retrieving the data representing the query result from distributed data repositories comprises:

retrieving the data representing the query result from triplestore data repositories.

15. The method of claim 1 wherein receiving the data representing the query comprises:

receiving the query from a computing device associated with data representing a user account.

16. The method of claim 1 wherein receiving the data representing the query comprises:

receiving the query from a computing device associated with an external computing system hosting an external dataset linked to one of the plurality of atomized datasets.

17. The method of claim 16 wherein receiving the query from the external computing system comprises:

performing a query-level authorization process.

18. The method of claim 1 wherein retrieving the data representing the query result from distributed data repositories comprises:

retrieving the data representing the query result from triplestore data repositories.