LAYERED DATA GENERATION AND DATA REMEDIATION TO FACILITATE FORMATION OF INTERRELATED DATA IN A SYSTEM OF NETWORKED COLLABORATIVE DATASETS

• US 20190050445A1
• Filed: 03/20/2018
• Published: 02/14/2019
• Est. Priority Date: 06/19/2016
• Status: Active Grant

• Current Assignee: Data.World, Inc.
• Original Assignee: Data.World, Inc.
• Inventors: Griffith, David Lee, Jacob, Bryon Kristen, Reynolds, Shad William
• Granted Patent: US 11,036,716 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24564 Applying rules; Deductive queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection

Abstract

Various embodiments relate generally to data science and data analysis, computer software and systems, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby logic is configured to remediate anomalies in a data set originating in a first format prior to enrichment and conversion into a second format that facilitates forming collaborative dataset and, for example, interrelations among a system of networked collaborative datasets, whereby, at least in some implementations, data interrelations between different formats may be disposed in one or more data layers (e.g., layered data files and/or data arrangements). In some examples, a method may include analyzing data to detect a non-compliant data attribute, detecting a condition based on the non-compliant data attribute, invoking an action to modify a subset of data, and generating a graph data arrangement linkable to other graph data arrangements to form a collaborative dataset.

Claims

1. A method comprising:

receiving data representing a subset of data disposed in data fields of a data arrangement;

retrieving data representing a data attribute with which to analyze data from the data arrangement;

analyzing the subset of data to detect a non-compliant data attribute;

detecting a condition based on the non-compliant data attribute for the subset of data;

invoking an action to modify the subset of data to form a modified subset of the data directed to affecting the condition; and

generating a graph data arrangement including links with modified subset of the data, the graph data arrangement being linkable to other graph data arrangements as a collaborative dataset.

2. The method of claim 1 further comprises:

generating data to present one or more user inputs in a user interface to initiate execution of instructions to resolve the condition; and

receiving a response to at least one of the one or more user inputs to invoke the action.

3. The method of claim 2 wherein retrieving the data representing the data attribute comprises:

receiving data representing a configurable data attribute for the subset of data, the configurable data attribute including a configurable data property and a configurable value with which to analyze data from the data arrangement.

4. The method of claim 1 further comprises:

identifying the action relative to a plurality of actions to resolve the condition; and

executing instructions to invoke the action to resolve the condition.

5. The method of claim 1 wherein analyzing to detect the non-compliant data attribute further comprises:

identifying a data characteristic specifying a portion of the subset of data as including restricted data.

6. The method of claim 5 further comprising:

masking the portion of the subset of data.

7. The method of claim 6 wherein masking the portion of the subset of data comprises:

encrypting the portion of the subset of data.

8. The method of claim 1 wherein analyzing to detect the non-compliant data attribute further comprises:

characterizing the subset of data to identify a pattern of data; and

matching the pattern of data against the data fields to identify a noncompliant portion of the data.

9. The method of claim 1 wherein analyzing to detect the non-compliant data attribute further comprises:

characterizing the subset of data to identify a pattern of data; and

matching the pattern of data against the data arrangement to identify a noncompliant portion of the data arrangement.

10. The method of claim 1 wherein generating the graph data arrangement further comprises:

detecting an anomalous condition.

11. The method of claim 10 wherein detecting the anomalous condition comprises:

detecting an anomalous value associated with the subset of data.

12. The method of claim 10 wherein detecting the anomalous value comprises:

determining a non-value.

13. The method of claim 10 wherein detecting the anomalous value comprises:

determining a value is out-of-range compliance.

14. The method of claim 1 wherein generating the graph data arrangement comprises:

forming referential data to associate a datum in the graph data arrangement to a portion of data in a tabular data arrangement,

wherein data operations using the tabular data arrangement is applied on the datum in the graph data arrangement.

15. The method of claim 1 wherein the data arrangement comprises:

a tabular data arrangement.

16. An apparatus comprising:

a memory including executable instructions; and

a processor, responsive to executing the instructions, is configured to;

receive data representing a subset of data disposed in data fields of a data arrangement;

retrieve data representing a data attribute with which to analyze data from the data arrangement;

analyze the subset of data to detect a non-compliant data attribute;

detect a condition based on the non-compliant data attribute for the subset of data;

invoke an action to modify the subset of data to form a modified subset of the data directed to affecting the condition; and

generate a graph data arrangement including links with modified subset of the data, the graph data arrangement being linkable to other graph data arrangements as a collaborative dataset.

17. The apparatus of claim 16 wherein the processor is further configured to:

generate data to present one or more user inputs in a user interface to initiate execution of instructions to resolve the condition; and

receive a response to at least one of the one or more user inputs to invoke the action.

18. The apparatus of claim 17 wherein a subset of the instructions to cause the processor to retrieve the data representing the data attribute further causes the processor to:

receive data representing a configurable data attribute for the subset of data, the configurable data attribute including a configurable data property and a configurable value with which to analyze data from the data arrangement.

19. The apparatus of claim 16 wherein the processor is further configured to:

identify the action relative to a plurality of actions to resolve the condition; and

execute instructions to invoke the action to resolve the condition.

20. The apparatus of claim 16 wherein a subset of the instructions to cause the processor to analyze the subset of data to detect the non-compliant data attribute further causes the processor to:

identify a data characteristic specifying a portion of the subset of data as including restricted data; and

mask the portion of the subset of data.