Dataset analysis and dataset attribute inferencing to form collaborative datasets

• US 10,452,677 B2
• Filed: 06/19/2016
• Issued: 10/22/2019
• Est. Priority Date: 06/19/2016
• Status: Active Grant

• Current Assignee: Data.World, Inc.
• Original Assignee: Data.World, Inc.
• Inventors: Jacob, Bryon Kristen, Griffith, David Lee, Le, Triet Minh, Loyens, Jon, Hurt, Brett A., Keen, Arthur Albert
• Primary Examiner(s): Mackes, Kris E
• Assistant Examiner(s): Htay, Lin Lin M
• Application Number:
  US15/186,516
• Publication Number:
  US 20170364570A1
• Time in Patent Office: 1,220 Days
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Various embodiments relate generally to data science and data analysis, computer software and systems, and wired and wireless network communications to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby a collaborative data layer and associated logic facilitate, for example, efficient access to, and implementation of, collaborative datasets. In some examples, a method may include receiving a dataset having a data format into a dataset ingestion controller configured to form a collaborative dataset, interpreting data of the dataset against data classifications at an inference engine to derive at least an inferred attribute, associating the data with annotative data identifying the inferred attribute, and converting the dataset at a format converter to form an atomized dataset.

Claims

1. A method comprising:

receiving data representing a plurality of datasets each dataset having a different data format into a dataset ingestion controller configured to form a collaborative dataset, at least one dataset being associated with an identifier;

interpreting a subset of data of the dataset against one or more data classifications at an inference engine to derive at least an inferred attribute for the subset of data;

identifying that the subset of data of the at least one dataset omits an association to annotative data representing an annotative description;

deducing a data classification for the subset of data of the dataset to form a deduced data classification;

analyzing other annotative data associated with equivalently classified data in other datasets;

associating the subset of the data with the annotative data based on the deduced data classification to identify the inferred attribute;

converting the plurality of datasets from the different data formats at a format converter to form an atomized dataset having a specific format;

identifying a subset of datasets;

linking a subset of atomized data points in the atomized dataset to the subset of datasets;

forming an enriched graph data structure accessible in association with the identifier;

deriving a supplemental subset of data based on one or more subsets of the at least one dataset;

associating the supplemental subset of data to the at least one dataset; and

converting the supplemental subset of data to form a supplemental atomized dataset including the atomized dataset.

2. The method of claim 1 further comprising:

consolidating the atomized dataset with a graph data structure stored in a repository.

3. The method of claim 2 wherein consolidating the atomized dataset comprises:

linking associations between one or more atomized data points of the atomized dataset and one or more other atomized data points of other atomized datasets.

4. The method of claim 3 further comprising:

retrieving data representing other datasets for the other atomized datasets from external computing systems that host the other datasets, which have different data formats; and

converting the data representing the other datasets from the different data formats to the specific format to form the other atomized datasets.

5. The method of claim 1 wherein identifying the subset of datasets comprises:

identifying a subset of datasets hosted external to the collaborative dataset consolidation system.

6. The method of claim 1 wherein identifying the subset of datasets comprises:

identifying data representing descriptive information describing a usage of the subset of datasets; and

identifying links atomized data points among datasets to form the subset of datasets based on the usage data.

7. The method of claim 1 wherein identifying the subset of datasets comprises:

identifying data representing descriptive information describing a user account attribute associated with the subset of datasets; and

identifying links atomized data points among datasets to form the subset of datasets based on the user account attribute.

8. The method of claim 1 wherein receiving the data representing the dataset associated with the identifier:

receiving the data representing the dataset associated with an account identifier.

9. The method of claim 1 further comprising:

limiting access to a portion of data in a graph data structure as a function of the identifier,

wherein the portion of data in the graph data structure includes the atomized dataset and other atomized datasets stored in a repository.

10. The method of claim 1 wherein interpreting the subset of data against the one or more data classifications comprises:

interpreting the subsets of data against the one or more data classifications associated with other subsets of data in the dataset.

11. The method of claim 1 wherein interpreting the subset of data against the one or more data classifications comprises:

interpreting the subsets of data against the one or more data classifications associated with data stored in a repository including a graph data structure.

12. The method of claim 1 wherein interpreting the subset of data against the one or more data classifications comprises:

generating a data signal to request input data from a user interface;

transmitting the data signal to the user interface; and

receiving response data specifying the annotative data identifying an attribute as the inferred attribute.

13. The method of claim 1 further comprising:

loading the atomized dataset into a data repository configured to facilitate access to the atomized dataset.

14. The method of claim 13 wherein loading the atomized dataset into the repository comprises:

loading the atomized dataset into a triplestore data repository.

15. A method comprising:

receiving data representing a plurality of datasets, each dataset having a different data format, into a dataset ingestion controller configured to form a collaborative dataset, at least one dataset being associated with a user account identifier;

converting the plurality of datasets from the different data formats at a format converter to form an atomized dataset having a specific format;

identifying a subset of datasets that includes portions that are associated with at least a portion of the dataset;

identifying that the portion of the dataset omits an association to annotative data representing an annotative description;

deducing a data classification for the portion of the dataset to form a deduced data classification;

analyzing other annotative data associated with equivalently classified data in other datasets;

linking at a data enrichment manager a subset of atomized data points associated with the subset of datasets to atomized data points of the atomized dataset based on the deduced data classification;

converting the subset of datasets including the portion of the dataset to form a subset of atomized datasets having the specific format;

forming a graph data structure accessible in association with the user account identifier, the graph data structure including data from the atomized dataset and from the subset of atomized datasets;

identifying the subset of datasets;

linking the subset of atomized data points in the atomized dataset to the subset of datasets;

forming an enriched graph data structure accessible in association with the user account identifier;

deriving a supplemental subset of data based on one or more subsets of the at least one dataset;

associating the supplemental subset of data to the at least one dataset; and

converting the supplemental subset of data to form a supplemental atomized dataset including the atomized dataset.

16. The method of claim 15 wherein linking the subset of atomized data points comprising:

forming an atomized consolidated dataset based on differently-formatted datasets.

17. The method of claim 15 comprises:

identifying a first data classification of the dataset is equivalent to a second data classification of at least one subset of the subset of datasets.