Data analytics lifecycle automation

• US 9,098,617 B1
• Filed: 09/27/2012
• Issued: 08/04/2015
• Est. Priority Date: 09/27/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Todd, Stephen, Dietrich, David, Pauley, Wayne A. Jr., Baldwin, Rhonda
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Pellett, Daniel
• Application Number:
  US13/628,589
• Time in Patent Office: 1,041 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/301 where the computing system is a virtual computing platform, e.g. logically partitioned systems virtual machines G06F9/45533; logical partitioning of resources G06F9/5077
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 16/212 with details for data modelling support
• CPC Class Codes: G06N 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

An initial data analytic plan for analyzing a given data set associated with a given data problem is defined. At least a portion of original data in the given data set is conditioned to generate conditioned data. At least one model is selected to analyze at least one of the original data and the conditioned data. The at least one selected model is executed on at least one of a portion of the original data and a portion of the conditioned data. Results of the model execution are communicated to at least one entity, the results comprising a refined data analytic plan for analyzing the given data set. One or more computing resources are provisioned to implement the refined data analytic plan. The defining, conditioning, selecting, executing, communicating and provisioning steps are performed on one or more processing elements associated with a computing system and automate a data analytics lifecycle.

Claims

1. A method comprising:

defining an initial data analytic plan for analyzing a given data set associated with a given data problem;

conditioning at least a portion of original data in the given data set to generate conditioned data;

selecting at least one model to analyze at least one of the original data and the conditioned data;

executing the at least one selected model on at least one of a portion of the original data and a portion of the conditioned data to confirm an adequacy of the at least one selected model;

communicating results of the model execution to at least one entity, the results comprising a refined data analytic plan for analyzing the given data set; and

provisioning, via generating and deploying, one or more computing resources to implement the refined data analytic plan;

wherein the defining, conditioning, selecting, executing, communicating and provisioning steps correspond to respective phases of a data analytics lifecycle;

wherein the method further comprises;

providing, to a user during multiple ones of the respective phases prior to the phase corresponding to the provisioning, an inventory of one or more computing resources to implement the at least one selected model; and

changing, by the user during one or more of the respective phases prior to the phase corresponding to the provisioning, the at least one selected model based on the provided inventory;

wherein the step of conditioning at least a portion of original data in the given data set to generate conditioned data comprises creating a separate analytics environment used to condition the portion of the original data, the separate analytics environment is created to have a capacity that is a selectable multiple of a capacity associated with the original data in the given data set, and one of one or more selectable multiples of the capacity are selected by the user between any two of the respective phases to dynamically provision and adjust the capacity of the separate analytics environment;

wherein, in response to the user returning to at least one previous phase of the data analytics lifecycle and altering the previous phase, one or more subsequent phases of the data analytics lifecycle are automatically updated based on the user-altered previous phase; and

wherein the defining, conditioning, selecting, executing, communicating, provisioning, providing, and changing steps are performed on one or more processing elements associated with a computing system and automate the data analytics lifecycle.

2. The method of claim 1, wherein the step of defining an initial data analytic plan for analyzing a given data set further comprises developing at least one hypothesis relating to the given data problem.

3. The method of claim 1, wherein the step of conditioning at least a portion of original data in the given data set to generate conditioned data further comprises transforming at least a portion of the original data utilizing one or more transformation algorithms.

4. The method of claim 2, wherein the step of selecting at least one model to analyze at least one of the original data and the conditioned data further comprises selecting a model that one of proves and disproves the at least one hypothesis developed in the initial data analytic plan defining step.

5. The method of claim 1, wherein the step of executing the at least one selected model on at least one of a portion of the original data and a portion of the conditioned data to confirm an adequacy of the at least one selected model further comprises updating the initial data analytic plan to generate a refined data analytic plan.

6. The method of claim 1, wherein the step of communicating results of the model execution to at least one entity further comprises masking at least a portion of the results, prior to presentation, based on one or more privacy considerations.

7. The method of claim 1, wherein the step of provisioning one or more computing resources to implement the refined data analytic plan further comprises provisioning one or more virtual resources to form a provisioned system that is usable to analyze subsequent data sets associated with the given data problem.

8. A computer program product comprising a non-transitory processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by the one or more processing elements of the computing system implement the steps of the method of claim 1.

9. The method of claim 1, wherein the inventory of the one or more computing resources is provided to the user via a user interface.

10. The method of claim 1, wherein the at least one selected model is changed by the user via a user interface.

11. The method of claim 1, wherein the one of the one or more selectable multiples of the capacity are selected by the user via a user interface.

12. An apparatus comprising:

a memory; and

at least one processor operatively coupled to the memory and configured to;

define an initial data analytic plan for analyzing a given data set associated with a given data problem;

condition at least a portion of original data in the given data set to generate conditioned data;

select at least one model to analyze at least one of the original data and the conditioned data;

execute the at least one selected model on at least one of a portion of the original data and a portion of the conditioned data to confirm an adequacy of the at least one selected model;

communicate results of the model execution to at least one entity, the results comprising a refined data analytic plan for analyzing the given data set; and

provision, via generating and deploying, one or more computing resources to implement the refined data analytic plan;

wherein the defining, conditioning, selecting, executing, communicating and provisioning operations automate and correspond to respective phases of a data analytics lifecycle; and

wherein the at least one processor is further configured to;

provide, to a user during multiple ones of the respective phases prior to the phase corresponding to the provisioning, an inventory of one or more computing resources to implement the at least one selected model; and

change, based on input from the user, during one or more of the respective phases prior to the phase corresponding to the provisioning, the at least one selected model based on the provided inventory;

wherein conditioning at least a portion of original data in the given data set to generate conditioned data comprises creating a separate analytics environment used to condition the portion of the original data, the separate analytics environment is created to have a capacity that is a selectable multiple of a capacity associated with the original data in the given data set, and one of one or more selectable multiples of the capacity are selected by the user between any two of the respective phases to dynamically provision and adjust the capacity of the separate analytics environment; and

wherein, in response to the user returning to at least one previous phase of the data analytics lifecycle and altering the previous phase, one or more subsequent phases of the data analytics lifecycle are automatically updated based on the user-altered previous phase.

13. The apparatus of claim 12, wherein defining an initial data analytic plan for analyzing a given data set further comprises developing at least one hypothesis relating to the given data problem.

14. The apparatus of claim 12, wherein conditioning at least a portion of original data in the given data set to generate conditioned data further comprises transforming at least a portion of the original data utilizing one or more transformation algorithms.

15. The apparatus of claim 13, wherein selecting at least one model to analyze at least one of the original data and the conditioned data further comprises selecting a model that one of proves and disproves the at least one hypothesis developed in the initial data analytic plan defining step.

16. The apparatus of claim 12, wherein executing the at least one selected model on at least one of a portion of the original data and a portion of the conditioned data to confirm an adequacy of the at least one selected model further comprises updating the initial data analytic plan to generate a refined data analytic plan.

17. The apparatus of claim 12, wherein communicating results of the model execution to at least one entity further comprises masking at least a portion of the results, prior to presentation, based on one or more privacy considerations.

18. The apparatus of claim 12, wherein provisioning one or more computing resources to implement the refined data analytic plan further comprises provisioning one or more virtual resources to form a provisioned system that is usable to analyze subsequent data sets associated with the given data problem.

19. A system comprising:

a memory;

one or more processing elements operatively coupled to the memory;

a graphical user interface;

a discovery phase module for defining an initial data analytic plan for analyzing a given data set associated with a given data problem;

a data preparation phase module for conditioning at least a portion of original data in the given data set to generate conditioned data;

a model planning phase module for selecting at least one model to analyze at least one of the original data and the conditioned data;

a model building phase module for executing the at least one selected model on at least one of a portion of the original data and a portion of the conditioned data to confirm an adequacy of the at least one selected model;

a results communication phase module for communicating results of the model execution to at least one entity, the results comprising a refined data analytic plan for analyzing the given data set; and

an operationalizing phase module for provisioning, via generating and deploying, one or more computing resources to implement the refined data analytic plan;

wherein the discovery phase module, the data preparation phase module, the model planning phase module, the model building phase module, the results communication phase module, and the operationalizing phase module are configured to implement respective phases of a data analytics lifecycle and are implemented on the one or more processing elements associated with a computing system so as to automate the data analytics lifecycle; and

wherein the system is configured to;

provide to a user, via the graphical user interface and during multiple ones of the respective phases prior to the phase implemented by the operationalizing phase module, an inventory of one or more computing resources to implement the at least one selected model; and

permit a user, via the graphical user interface, to change, during one or more of the respective phases prior to the phase implemented by the operationalizing phase module, the at least one selected model based on the provided inventory;

wherein conditioning at least a portion of original data in the given data set to generate conditioned data comprises creating a separate analytics environment used to condition the portion of the original data, the separate analytics environment is created to have a capacity that is a selectable multiple of a capacity associated with the original data in the given data set, and one of one or more selectable multiples of the capacity are selected by the user between any two of the respective phases to dynamically provision and adjust the capacity of the separate analytics environment; and

wherein, in response to the user returning to at least one previous phase of the data analytics lifecycle and altering the previous phase, one or more subsequent phases of the data analytics lifecycle are automatically updated based on the user-altered previous phase.