AUTOMATIC ENUMERATION OF DATA ANALYSIS OPTIONS AND RAPID ANALYSIS OF STATISTICAL MODELS

• US 20160110362A1
• Filed: 10/20/2014
• Published: 04/21/2016
• Est. Priority Date: 10/20/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Khurana, Udayan, Parthasarathy, Srinivasan, Turaga, Deepak S., Pavuluri, Venkata N., Vu, Long H.
• Granted Patent: US 10,346,393 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/3323 using document space presentation or visualization, e.g. category, hierarchy or range presentation and selection

Abstract

Embodiments relate to analyzing dataset. A method of analyzing data is provided. The method obtains a description of a dataset. The method automatically generates a plurality of analysis options from the description of the dataset. The method generates a plurality of queries based on the analysis options. The method deploys the queries on the dataset to build a plurality of statistical models from the dataset.

Claims

1. A computer program product for analyzing data, the computer program product comprising:

a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a processing circuit to cause the processing circuit to perform a method comprising;

obtaining a description of a dataset;

automatically generating, by a computer, a plurality of analysis options from the description of the dataset;

generating a plurality of queries based on the analysis options; and

deploying the queries on the dataset to build a plurality of statistical models from the dataset.

2. The computer program product of claim 1, wherein the dataset is in a database format, and the description of a dataset includes a database schema.

3. The computer program product of claim 1, wherein the method further comprises allowing a user to remove one or more analysis options before generating queries from the analysis options.

4. The computer program product of claim 1, wherein the automatically generating the analysis options comprises applying a set of predetermined rules to the description of the dataset to generate one or more of data filtering conditions, database clauses, statistical functions and model-specific functions.

5. The computer program product of claim 4, wherein the automatically generating the analysis options further comprises combining the one or more of data filtering conditions, database clauses, statistical functions and model-specific functions.

6. The computer program product of claim 1, wherein the queries comprise database queries.

7. The computer program product of claim 1, wherein the method further comprises reusing the queries to analyze another dataset.

8. A computer system for analyzing statistical models, the system comprising:

a memory having computer readable instructions; and

a processor configured to execute the computer readable instructions, the instructions comprising;

obtaining a pool of statistical models to analyze;

ranking the statistical models in an order of decreasing interest to a user;

automatically grouping a highest ranked model and a set of other ranked models into a cluster based on similarities of the other ranked models to the highest ranked model; and

presenting the cluster of models to the user to interpret.

9. The system of claim 8, wherein the ranking comprises using at least one of skewness, entropy, and sparsity metrics.

10. The system of claim 9, wherein the instructions further comprise allowing the user to select one or more of the metrics.

11. The system of claim 8, wherein the instructions further comprise calculating the similarities between the highest ranked model and each of the other ranked models using at least one of Kolmogorov-Smirnov, Hamming, Cosine, Pearson and Jaccard metrics.

12. The system of claim 11, wherein the instructions further comprise allowing the user to select one or more of the metrics.

13. The system of claim 8, wherein the instructions further comprise:

removing the cluster of models from the pool of models;

automatically grouping a highest ranking model and a set of other ranked models from the remaining pool of models into another cluster; and

presenting the other cluster of models to the user.

14-20. -20. (canceled)