Intelligent data curation

• US 10,552,739 B1
• Filed: 07/05/2019
• Issued: 02/04/2020
• Est. Priority Date: 10/15/2018
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Rausch, Nancy Anne, Barney, Roger Jay, Trawinski, John P.
• Primary Examiner(s): Khuu, Hien D
• Application Number:
  US16/503,742
• Time in Patent Office: 214 Days
• Field of Search: 706 12
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 9/30 Arrangements for executing machine instructions, e.g. instruction decode for executing microinstructions G06F9/22
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An apparatus includes a processor to: provide a set of feature routines to a set of processor cores to detect features of a data set distributed thereamong; generate metadata indicative of the detected features; generate context data indicative of contextual aspects of the data set; provide the metadata and context data to each processor core, and distribute a set of suggestion models thereamong to enable derivation of a suggested subset of data preparation operations to be suggested to be performed on the data set; transmit indications of the suggested subset to a viewing device, and receive therefrom indications of a selected subset of data preparation operations selected to be performed; compare the selected and suggested subsets; and in response to differences therebetween, re-train at least one suggestion model of the set of suggestion models based at least on the combination of the metadata, context data and selected subset.

Claims

1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:

receive an indication of availability of a data set, wherein the indication of availability of the data set comprises an indication of at least one of a contextual aspect of the data set from among a pre-selected set of contextual aspects, a structural feature of the data set from among a pre-selected set of structural features, or a data feature of a data value of the data set from among a pre-selected set of data features;

coordinate a performance of a distribution of data set portions of the data set among a set of processor cores;

provide a set of feature routines to each processor core of the set of processor cores to enable each processor core of the set of processor cores to execute instructions of each feature routine to detect a structural feature of the corresponding data set portion or a data feature of data values of the corresponding data set portion;

receive indications of detected structural features and detected data features of the set of data set portions from the set of processor cores;

generate, for the data set, metadata indicative of the detected structural features and the detected data features;

generate, for the data set, context data indicative of the set of contextual aspects of the data set;

provide the metadata and context data to each processor core of the set of processor cores;

distribute a set of suggestion models among the set of processor cores to provide each processor core of the set of processor cores with a different suggestion model from among the set of suggestion models, and to enable the set of processor cores to employ the set of suggestion models to derive a suggested subset of data preparation operations of a set of data preparation operations to be suggested to be performed on the data set, wherein;

each suggestion model comprises a pre-selected type of model previously trained to determine whether to suggest that a corresponding data preparation operation of the set of data preparation operations be performed on the data set based on the metadata and context data;

receive indications of the suggested subset from the set of processor cores;

transmit an indication of the suggested subset to a viewing device to enable a presentation of the suggested subset;

receive, from the viewing device, an indication of a selected subset of the set of data preparation operations selected to be performed;

compare the selected subset to the suggested subset to determine whether there is a difference between the suggested and selected subsets; and

in response to a determination that there is a difference between the suggested and selected subsets, re-train at least one suggestion model of the set of suggestion models based at least on a combination of the metadata, the context data and the selected subset.

2. The apparatus of claim 1, wherein the processor is caused to provide an indication of the selected subset to each processor core of the selected set of processor cores to enable the set of processor cores to perform the selected subset of data preparation operations on the data set.

3. The apparatus of claim 1, wherein:

the at least one suggestion model comprises neural network configuration data that comprises an indication of at least one of a quantity of neurons, a quantity of rows of neurons, a set of connections among neurons, a trigger function to be implemented by at least one neuron, weights and biases of the trigger function; and

use of the at least one model by at least one processor core of the set of processor cores to determine whether to suggest that the corresponding data preparation operation be performed on the data set comprises use of the suggestion model by the at least one processor core to configure a neural network to implement a set of classifiers.

4. The apparatus of claim 3, wherein the re-training of the at least one suggestion model comprises use, by the at least one processor core, of at least the combination of the metadata, the context data and selected subset to perform backpropagation on the neural network to change at least one of the weights or the biases.

5. The apparatus of claim 1, wherein:

the set of processor cores is distributed among a set of node devices;

the processor is caused to perform operations comprising;

receive indications of availability of each node device of a plurality of node devices; and

select the set of node devices from among the plurality of node devices based on the received indications of availability;

the coordination of the performance of distribution of the data set portions of the data set among the set of processor cores comprises a coordination of a performance of a distribution of the data set portions among the set of node devices from at least one storage device through a network;

the provision of the set of feature routines to each processor core of the set of processor cores comprises transmission of the set of feature routines to each node device of the set of node devices;

the provision of the metadata and context data to each processor core of the set of processor cores comprises transmission of the metadata and context data to each node device of the set of node devices; and

the distribution of the set of suggestion models among the set of processor cores comprises the distribution of the set of suggestion models among the set of node devices to provide each node device of the set of node devices with a different suggestion model from among the set of suggestion models.

6. The apparatus of claim 5, wherein:

each node device of the set of node devices stores the corresponding data set portion distributed to the node device after the detection of structural features or data features of the corresponding data portion; and

the processor is caused to transmit, to each node device of the set of node devices, an indication of the selected subset to enable the node device to perform the selected subset of the set of data preparation operations on the corresponding data set portion.

7. The apparatus of claim 5, wherein:

the pre-selected set of structural features of the data set comprises a size of the data set;

the indication of availability of the data set comprises an indication of the size of the data set; and

the processor is caused to perform operations comprising;

derive a quantity of node devices based on the size of the data set; and

select the quantity of node devices from among the plurality of node devices as the set of node devices.

8. The apparatus of claim 5, wherein the processor is caused to perform operations comprising:

receive an indication from each node device of the plurality of node devices of a type of processor of the node device; and

select the set of feature routines to transmit to each node device of the set of node devices based on the type of processor of the node device.

9. The apparatus of claim 5, wherein the processor is caused to perform operations comprising:

receive update data comprising a feature routine;

coordinate a distribution of training data set portions of a training data set among the set of node devices;

transmit the set of feature routines that includes the received feature routine to each node device of the set of node devices to enable each node device of the set of node devices to detect features of the corresponding training data set portion;

receive indications of detected features of the set of training data set portions from the set of node devices;

generate, for the training data set, a portion of training metadata indicative of the detected features;

transmit, to each node device of the set of node devices, the portion of the training metadata, a portion of training context data and a set of action indications, wherein;

the portion of the training context data is indicative of a set of contextual aspects of the training data set; and

the set of action indications is indicative of a subset of data preparation actions of a set of data preparation actions to be suggested to be performed on the training data set based on the portion of the training metadata and the portion of the training context data;

distribute the set of suggestion models among the set of node devices to provide each node device of the set of node devices with a different suggestion model from among the set of suggestion models, and to enable a re-training of the set of suggestion models by the set of node devices;

receive, from each node device, a corresponding suggestion model of the set of suggestion models after the re-training; and

store the received suggestion models as replacements for the set of suggestion models.

10. The apparatus of claim 1, wherein:

the pre-selected set of contextual aspects comprises at least one of;

an indication of an identity of a source of the data set;

an indication of a location associated with the source;

an indication of an industry associated with the source;

an indication of a time or date of receipt of the data set;

an indication of a user of the other device;

an indication of a location associated with the user;

an indication of an industry associated with the user;

or

an indication of a time or date of receipt of a request from the user to access the data set;

the pre-selected set of structural features comprises at least one of;

an indication of a size of the data set;

an indication of a time or date of generation of the data set;

an indication of a language of the data set;

an indication of an organization of data values within the data set;

or

an indication of a number of dimensions of indexing of the data set; and

the pre-selected set of data features comprises at least one of;

an indication of at least one data type within the data set;

an indication of at least one data format within the data set;

or

an indication of a minimum, a maximum, a mean or an average of a data value of the data set.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor to perform operations comprising:

receive an indication of availability of a data set, wherein the indication of availability of the data set comprises an indication of at least one of a contextual aspect of the data set from among a pre-selected set of contextual aspects, a structural feature of the data set from among a pre-selected set of structural features, or a data feature of a data value of the data set from among a pre-selected set of data features;

coordinate a performance of a distribution of data set portions of the data set among a set of processor cores;

provide a set of feature routines to each processor core of the set of processor cores to enable each processor core of the set of processor cores to execute instructions of each feature routine to detect a structural feature of the corresponding data set portion or a data feature of data values of the corresponding data set portion;

receive indications of detected structural features and detected data features of the set of data set portions from the set of processor cores;

generate, for the data set, metadata indicative of the detected structural features and the detected data features;

generate, for the data set, context data indicative of the set of contextual aspects of the data set;

provide the metadata and context data to each processor core of the set of processor cores;

distribute a set of suggestion models among the set of processor cores to provide each processor core of the set of processor cores with a different suggestion model from among the set of suggestion models, and to enable the set of processor cores to employ the set of suggestion models to derive a suggested subset of data preparation operations of a set of data preparation operations to be suggested to be performed on the data set, wherein;

each suggestion model comprises a pre-selected type of model previously trained to determine whether to suggest that a corresponding data preparation operation of the set of data preparation operations be performed on the data set based on the metadata and context data;

receive indications of the suggested subset from the set of processor cores;

transmit an indication of the suggested subset to a viewing device to enable a presentation of the suggested subset;

receive, from the viewing device, an indication of a selected subset of the set of data preparation operations selected to be performed;

compare the selected subset to the suggested subset to determine whether there is a difference between the suggested and selected subsets; and

in response to a determination that there is a difference between the suggested and selected subsets, re-train at least one suggestion model of the set of suggestion models based at least on a combination of the metadata, the context data and the selected subset.

12. The computer-program product of claim 11, wherein the processor is caused to provide an indication of the selected subset to each processor core of the selected set of processor cores to enable the set of processor cores to perform the selected subset of data preparation operations on the data set.

13. The computer-program product of claim 11, wherein:

the at least one suggestion model comprises neural network configuration data that comprises an indication of at least one of a quantity of neurons, a quantity of rows of neurons, a set of connections among neurons, a trigger function to be implemented by at least one neuron, weights and biases of the trigger function; and

use of the at least one model by at least one processor core of the set of processor cores to determine whether to suggest that the corresponding data preparation operation be performed on the data set comprises use of the suggestion model by the at least one processor core to configure a neural network to implement a set of classifiers.

14. The computer-program product of claim 13, wherein the re-training of the at least one suggestion model comprises use, by the at least one processor core, of at least the combination of the metadata, the context data and selected subset to perform backpropagation on the neural network to change at least one of the weights or the biases.

15. The computer-program product of claim 11, wherein:

the set of processor cores is distributed among a set of node devices;

the processor is caused to perform operations comprising;

receive indications of availability of each node device of a plurality of node devices; and

select the set of node devices from among the plurality of node devices based on the received indications of availability;

the coordination of the performance of distribution of the data set portions of the data set among the set of processor cores comprises a coordination of a performance of a distribution of the data set portions among the set of node devices from at least one storage device through a network;

the provision of the set of feature routines to each processor core of the set of processor cores comprises transmission of the set of feature routines to each node device of the set of node devices;

the provision of the metadata and context data to each processor core of the set of processor cores comprises transmission of the metadata and context data to each node device of the set of node devices; and

the distribution of the set of suggestion models among the set of processor cores comprises the distribution of the set of suggestion models among the set of node devices to provide each node device of the set of node devices with a different suggestion model from among the set of suggestion models.

16. The computer-program product of claim 15, wherein:

each node device of the set of node devices stores the corresponding data set portion distributed to the node device after the detection of structural features or data features of the corresponding data portion; and

the processor is caused to transmit, to each node device of the set of node devices, an indication of the selected subset to enable the node device to perform the selected subset of the set of data preparation operations on the corresponding data set portion.

17. The computer-program product of claim 15, wherein:

the pre-selected set of structural features of the data set comprises a size of the data set;

the indication of availability of the data set comprises an indication of the size of the data set; and

the processor is caused to perform operations comprising;

derive a quantity of node devices based on the size of the data set; and

select the quantity of node devices from among the plurality of node devices as the set of node devices.

18. The computer-program product of claim 15, wherein the processor is caused to perform operations comprising:

receive an indication from each node device of the plurality of node devices of a type of processor of the node device; and

select the set of feature routines to transmit to each node device of the set of node devices based on the type of processor of the node device.

19. The computer-program product of claim 15, wherein the processor is caused to perform operations comprising:

receive update data comprising a feature routine;

coordinate a distribution of training data set portions of a training data set among the set of node devices;

transmit the set of feature routines that includes the received feature routine to each node device of the set of node devices to enable each node device of the set of node devices to detect features of the corresponding training data set portion;

receive indications of detected features of the set of training data set portions from the set of node devices;

generate, for the training data set, a portion of training metadata indicative of the detected features;

transmit, to each node device of the set of node devices, the portion of the training metadata, a portion of training context data and a set of action indications, wherein;

the portion of the training context data is indicative of a set of contextual aspects of the training data set; and

the set of action indications is indicative of a subset of data preparation actions of a set of data preparation actions to be suggested to be performed on the training data set based on the portion of the training metadata and the portion of the training context data;

distribute the set of suggestion models among the set of node devices to provide each node device of the set of node devices with a different suggestion model from among the set of suggestion models, and to enable a re-training of the set of suggestion models by the set of node devices;

receive, from each node device, a corresponding suggestion model of the set of suggestion models after the re-training; and

store the received suggestion models as replacements for the set of suggestion models.

20. The computer-program product of claim 11, wherein:

the pre-selected set of contextual aspects comprises at least one of;

an indication of an identity of a source of the data set;

an indication of a location associated with the source;

an indication of an industry associated with the source;

an indication of a time or date of receipt of the data set;

an indication of a user of the other device;

an indication of a location associated with the user;

an indication of an industry associated with the user;

or

an indication of a time or date of receipt of a request from the user to access the data set;

the pre-selected set of structural features comprises at least one of;

an indication of a size of the data set;

an indication of a time or date of generation of the data set;

an indication of a language of the data set;

an indication of an organization of data values within the data set;

or

an indication of a number of dimensions of indexing of the data set; and

the pre-selected set of data features comprises at least one of;

an indication of at least one data type within the data set;

an indication of at least one data format within the data set;

or

an indication of a minimum, a maximum, a mean or an average of a data value of the data set.

21. A computer-implemented method comprising:

receiving, by a processor, an indication of availability of a data set, wherein the indication of availability of the data set comprises an indication of at least one of a contextual aspect of the data set from among a pre-selected set of contextual aspects, a structural feature of the data set from among a pre-selected set of structural features, or a data feature of a data value of the data set from among a pre-selected set of data features;

coordinating, by the processor, a performance of a distribution of data set portions of the data set among a set of processor cores;

providing, by the processor, a set of feature routines to each processor core of the set of processor cores to enable each processor core of the set of processor cores to execute instructions of each feature routine to detect a structural feature of the corresponding data set portion or a data feature of data values of the corresponding data set portion;

receiving, by the processor, indications of detected structural features and detected data features of the set of data set portions from the set of processor cores;

generating, by the processor, for the data set, metadata indicative of the detected structural features and the detected data features;

generating, by the processor, for the data set, context data indicative of the set of contextual aspects of the data set;

providing, by the processor, the metadata and context data to each processor core of the set of processor cores;

distributing, by the processor, a set of suggestion models among the set of processor cores to provide each processor core of the set of processor cores with a different suggestion model from among the set of suggestion models, and to enable the set of processor cores to employ the set of suggestion models to derive a suggested subset of data preparation operations of a set of data preparation operations to be suggested to be performed on the data set, wherein;

each suggestion model comprises a pre-selected type of model previously trained to determine whether to suggest that a corresponding data preparation operation of the set of data preparation operations be performed on the data set based on the metadata and context data;

receiving, by the processor, indications of the suggested subset from the set of processor cores;

transmitting, by the processor and via a network, an indication of the suggested subset to a viewing device to enable a presentation of the suggested subset;

receiving, by the processor, and from the viewing device via the network, an indication of a selected subset of the set of data preparation operations selected to be performed;

comparing, by the processor, the selected subset to the suggested subset to determine whether there is a difference between the suggested and selected subsets; and

in response to a determination that there is a difference between the suggested and selected subsets, re-training, by the processor, at least one suggestion model of the set of suggestion models based at least on a combination of the metadata, the context data and the selected subset.

22. The computer-implemented method of claim 21, further comprising providing, by the processor, an indication of the selected subset to each processor core of the selected set of processor cores to enable the set of processor cores to perform the selected subset of data preparation operations on the data set.

23. The computer-implemented method of claim 21, wherein:

the at least one suggestion model comprises neural network configuration data that comprises an indication of at least one of a quantity of neurons, a quantity of rows of neurons, a set of connections among neurons, a trigger function to be implemented by at least one neuron, weights and biases of the trigger function; and

use of the at least one model by at least one processor core of the set of processor cores to determine whether to suggest that the corresponding data preparation operation be performed on the data set comprises use of the suggestion model by the at least one processor core to configure a neural network to implement a set of classifiers.

24. The computer-implemented method of claim 23, wherein the re-training of the at least one suggestion model comprises use, by the at least one processor core, of at least the combination of the metadata, the context data and selected subset to perform backpropagation on the neural network to change at least one of the weights or the biases.

25. The computer-program product of claim 21, wherein:

the set of processor cores is distributed among a set of node devices;

the method further comprises;

receiving, by the processor, indications of availability of each node device of a plurality of node devices; and

selecting, by the processor, the set of node devices from among the plurality of node devices based on the received indications of availability;

the coordination of the performance of distribution of the data set portions of the data set among the set of processor cores comprises a coordination of a performance of a distribution of the data set portions among the set of node devices from at least one storage device through the network;

the provision of the set of feature routines to each processor core of the set of processor cores comprises transmission of the set of feature routines to each node device of the set of node devices;

the provision of the metadata and context data to each processor core of the set of processor cores comprises transmission of the metadata and context data to each node device of the set of node devices; and

the distribution of the set of suggestion models among the set of processor cores comprises the distribution of the set of suggestion models among the set of node devices to provide each node device of the set of node devices with a different suggestion model from among the set of suggestion models.

26. The computer-implemented method of claim 25, wherein:

each node device of the set of node devices stores the corresponding data set portion distributed to the node device after the detection of structural features or data features of the corresponding data portion; and

the method further comprises transmitting, by the processor, to each node device of the set of node devices, an indication of the selected subset to enable the node device to perform the selected subset of the set of data preparation operations on the corresponding data set portion.

27. The computer-implemented method of claim 25, wherein:

the pre-selected set of structural features of the data set comprises a size of the data set;

the indication of availability of the data set comprises an indication of the size of the data set; and

the method further comprises;

deriving, by the processor, a quantity of node devices based on the size of the data set; and

selecting, by the processor, the quantity of node devices from among the plurality of node devices as the set of node devices.

28. The computer-implemented method of claim 25, further comprising:

receiving, by the processor, an indication from each node device of the plurality of node devices of a type of processor of the node device; and

selecting, by the processor, the set of feature routines to transmit to each node device of the set of node devices based on the type of processor of the node device.

29. The computer-implemented method of claim 25, further comprising:

receiving, by the processor, update data comprising a feature routine;

coordinating, by the processor and through the network, a distribution of training data set portions of a training data set among the set of node devices;

transmitting, by the processor, the set of feature routines that includes the received feature routine to each node device of the set of node devices to enable each node device of the set of node devices to detect features of the corresponding training data set portion;

receiving, by the processor, indications of detected features of the set of training data set portions from the set of node devices;

generating, by the processor, for the training data set, a portion of training metadata indicative of the detected features;

transmitting, by the processor, to each node device of the set of node devices, the portion of the training metadata, a portion of training context data and a set of action indications, wherein;

the portion of the training context data is indicative of a set of contextual aspects of the training data set; and

the set of action indications is indicative of a subset of data preparation actions of a set of data preparation actions to be suggested to be performed on the training data set based on the portion of the training metadata and the portion of the training context data;

distributing, by the processor, the set of suggestion models among the set of node devices to provide each node device of the set of node devices with a different suggestion model from among the set of suggestion models, and to enable a re-training of the set of suggestion models by the set of node devices;

receiving, by the processor, and from each node device, a corresponding suggestion model of the set of suggestion models after the re-training; and

storing the received suggestion models as replacements for the set of suggestion models.

30. The computer-implemented method of claim 21, wherein:

the pre-selected set of contextual aspects comprises at least one of;

an indication of an identity of a source of the data set;

an indication of a location associated with the source;

an indication of an industry associated with the source;

an indication of a time or date of receipt of the data set;

an indication of a user of the other device;

an indication of a location associated with the user;

an indication of an industry associated with the user;

or

an indication of a time or date of receipt of a request from the user to access the data set;

the pre-selected set of structural features comprises at least one of;

an indication of a size of the data set;

an indication of a time or date of generation of the data set;

an indication of a language of the data set;

an indication of an organization of data values within the data set;

or

an indication of a number of dimensions of indexing of the data set; and

the pre-selected set of data features comprises at least one of;

an indication of at least one data type within the data set;

an indication of at least one data format within the data set;

or

an indication of a minimum, a maximum, a mean or an average of a data value of the data set.