Systems, methods, and apparatuses for implementing data upload, processing, and predictive query API exposure

• US 9,690,815 B2
• Filed: 06/13/2016
• Issued: 06/27/2017
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Cronin, Beau David, Petschulat, Cap Christian, Jonas, Eric Michael
• Primary Examiner(s): Uddin, Md. I
• Application Number:
  US15/181,256
• Publication Number:
  US 20160292199A1
• Time in Patent Office: 379 Days
• Field of Search: 707722, 707736, 707737, 707739, 707741
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/2445 Data retrieval commands; View definitions
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24558 Binary matching operations
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2458 Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/18 : of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0201 : Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0202 : Market predictions or forecasting for commercial activities

Abstract

Disclosed herein are systems and methods for implementing data upload, processing, and predictive query API exposure including means for receiving a dataset in a tabular form, the dataset having a plurality of rows and a plurality of columns; processing the dataset to generate indices representing probabilistic relationships between the rows and the columns of the dataset; storing the indices in a database; exposing an Application Programming Interface (API) to query the indices in the database; receiving a request for a predictive query or a latent structure query against the indices in the database; querying the database for a prediction based on the request via the API; and returning the prediction responsive to the request. Other related embodiments are further disclosed.

Claims

1. A method comprising:

receiving a dataset in a tabular form, the dataset having a plurality of rows and a plurality of columns;

processing the dataset to generate indices representing probabilistic relationships between the rows and the columns of the dataset;

storing the indices in a database;

exposing an Application Programming Interface (API) to query the indices in the database;

receiving a request for a predictive query against the indices in the database;

querying the database for a predictive result based on the request via the API;

returning the predictive result responsive to the request, the predictive result being probabilistically related to rows or the columns of the dataset or both the rows and the columns of the dataset according to the generated indices representing the probabilistic relationships between the rows and the columns of the dataset; and

returning a confidence indicator with the predictive result, wherein the confidence indicator ranges from a minimum of 0 indicating a lowest possible confidence in the accuracy of the predictive result returned to a maximum of 1 indicating a highest possible confidence in the accuracy of the predictive result returned.

2. The method of claim 1, wherein processing the dataset comprises learning a joint probability distribution over the dataset to identify and describe the probabilistic relationships between elements of the dataset.

3. The method of claim 2, wherein the processing is triggered automatically responsive to receiving the dataset, and wherein learning the joint probability distribution is controlled by a default set of configuration parameters.

4. The method of claim 2, wherein learning the joint probability distribution is controlled by specified configuration parameters, the specified configuration parameters including one or more of:

a maximum period of time for processing the dataset;

a maximum number of iterations for processing the dataset;

a minimum number of iterations for processing the dataset;

a maximum amount of customer resources to be consumed by processing the dataset;

a maximum subscriber fee to be expended processing the dataset;

a minimum threshold predictive quality level to be attained by the processing of the dataset;

a minimum improvement to a confidence quality measure required for the processing to continue; and

a minimum or maximum number of the indices to be generated by the processing.

5. The method of claim 1, wherein:

processing the dataset to generate indices comprises iteratively learning joint probability distributions over the dataset to generate the indices; and

wherein the method further comprises;

periodically determining a confidence quality measure of the indices generated by the processing of the dataset; and

terminating processing of the dataset when the confidence quality measure attains a specified threshold.

6. The method of claim 5, further comprising:

receiving a predictive query or a latent structure query requesting a result from the indices generated by processing the dataset; and

executing the query against the generated indices prior to terminating processing of the dataset.

7. The method of claim 6, further comprising:

returning a predictive record set responsive to the predictive query or the latent structure query requesting the result; and

returning a notification with the result indicating processing of the dataset has not yet completed or a notification with the result indicating the confidence quality measure is below the specified threshold, or both.

8. The method of claim 5, wherein the confidence quality measure is determined by comparing a known result corresponding to observed and present values within the dataset with a predictive result obtained by querying the indices generated by the processing of the dataset.

9. The method of claim 5, wherein the confidence quality measure is determined by comparing ground truth data from the data set with one or more predictive results obtained by querying the indices generated by the processing of the dataset.

10. The method of claim 1, wherein processing the dataset comprises at least one of:

learning a Dirichlet Process Mixture Model (DPMM) of the dataset;

learning a cross categorization of the dataset;

learning an Indian buffet process model of the dataset; and

learning a mixture model or a mixture of finite mixtures model of the dataset.

11. Non-transitory computer readable storage media having instructions stored thereupon that, when executed by a system having at least a processor and a memory therein, the instructions cause the system to perform operations including:

receiving a dataset in a tabular form, the dataset having a plurality of rows and a plurality of columns;

processing the dataset to generate indices representing probabilistic relationships between the rows and the columns of the dataset;

storing the indices in a database;

exposing an Application Programming Interface (API) to query the indices in the database;

receiving a request for a predictive query against the indices in the database;

querying the database for a predictive result based on the request via the API;

returning the predictive result responsive to the request, the predictive result being probabilistically related to rows or the columns of the dataset or both the rows and the columns of the dataset according to the generated indices representing the probabilistic relationships between the rows and the columns of the dataset; and

returning a confidence indicator with the predictive result, wherein the confidence indicator ranges from a minimum of 0 indicating a lowest possible confidence in the accuracy of the predictive result returned to a maximum of 1 indicating a highest possible confidence in the accuracy of the predictive result returned.

12. The non-transitory computer readable storage media of claim 11:

wherein processing the dataset comprises learning a joint probability distribution over the dataset to identify and describe the probabilistic relationships between elements of the dataset.

13. The non-transitory computer readable storage media of claim 12, wherein the processing is triggered automatically responsive to receiving the dataset, and wherein learning the joint probability distribution is controlled by a default set of configuration parameters.

14. The non-transitory computer readable storage media of claim 12, wherein learning the joint probability distribution is controlled by specified configuration parameters, the specified configuration parameters including one or more of:

a maximum period of time for processing the dataset;

a maximum number of iterations for processing the dataset;

a minimum number of iterations for processing the dataset;

a maximum amount of customer resources to be consumed by processing the dataset;

a maximum subscriber fee to be expended processing the dataset;

a minimum threshold predictive quality level to be attained by the processing of the dataset;

a minimum improvement to a confidence quality measure required for the processing to continue; and

a minimum or maximum number of the indices to be generated by the processing.

15. The non-transitory computer readable storage media of claim 12, wherein:

processing the dataset to generate indices comprises iteratively learning joint probability distributions over the dataset to generate the indices; and

wherein the method further comprises;

periodically determining a confidence quality measure of the indices generated by the processing of the dataset;

terminating processing of the dataset when the confidence quality measure attains a specified threshold;

wherein the instructions when executed cause the system to perform operations further comprising;

receiving a predictive query or a latent structure query requesting a result from the indices generated by processing the dataset;

executing the query against the generated indices prior to terminating processing of the dataset

returning a predictive record set responsive to the predictive query or the latent structure query requesting the result; and

returning a notification with the result indicating processing of the dataset has not yet completed or a notification with the result indicating the confidence quality measure is below the specified threshold, or both.

16. The non-transitory computer readable storage media of claim 11, wherein processing the dataset comprises at least one of:

learning a Dirichlet Process Mixture Model (DPMM) of the dataset;

learning a cross categorization of the dataset;

learning an Indian buffet process model of the dataset; and

learning a mixture model or a mixture of finite mixtures model of the dataset.

17. A system to operate within a host organization, the system comprising:

a processor to execute instructions stored in memory of the system;

a request interface to receive as input a dataset in a tabular form, the dataset having a plurality of rows and a plurality of columns;

an analysis engine to process the dataset to generate indices representing probabilistic relationships between the rows and the columns of the dataset;

a predictive database system to store the generated indices;

an exposed Application Programming Interface (API) to query the indices stored in the predictive database system; and

the request interface to further receive a request for a predictive query against the indices stored in the predictive database system;

the API to query the database for a predictive result based on the request;

the request interface to further return the predictive result responsive to the request, the predictive result being probabilistically related to rows or the columns of the dataset or both the rows and the columns of the dataset according to the generated indices representing the probabilistic relationships between the rows and the columns of the dataset; and

the request interface to additionally return a confidence indicator with the predictive result, wherein the confidence indicator ranges from a minimum of 0 indicating a lowest possible confidence in the accuracy of the predictive result returned to a maximum of 1 indicating a highest possible confidence in the accuracy of the predictive result returned.

18. The system of claim 17, further comprising:

a web-server to implement the request interface;

wherein the web-server is to receive as additional input, a plurality of access requests from one or more client devices from among a plurality of customer organizations communicably interfaced with the host organization via a network;

a multi-tenant database system with predictive database functionality to implement the predictive database system; and

wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.

19. The system of claim 17, wherein the analysis engine to process the dataset comprises the analysis engine to learn a joint probability distribution over the dataset to identify and describe the probabilistic relationships between elements of the dataset.

20. The system of claim 19, wherein the analysis engine to process the dataset is triggered automatically responsive to receiving the dataset, and wherein the analysis engine to learn the joint probability distribution is controlled by a default set of configuration parameters.

21. The system of claim 19, wherein the analysis engine to learn the joint probability distribution is controlled by specified configuration parameters, the specified configuration parameters including one or more of:

a maximum period of time for processing the dataset;

a maximum number of iterations for processing the dataset;

a minimum number of iterations for processing the dataset;

a maximum amount of customer resources to be consumed by processing the dataset;

a maximum subscriber fee to be expended processing the dataset;

a minimum threshold predictive quality level to be attained by the processing of the dataset;

a minimum improvement to a confidence quality measure required for the processing to continue; and

a minimum or maximum number of the indices to be generated by the processing.

22. The system of claim 19:

wherein the analysis engine to process the dataset to generate the indices comprises the analysis engine to iteratively learn joint probability distributions over the dataset to generate the indices;

wherein the analysis engine is to further periodically determine a confidence quality measure of the indices generated by the processing of the dataset; and

wherein the analysis engine is to terminate processing of the dataset when the confidence quality measure attains a specified threshold.

23. The system of claim 22:

wherein the request interface to receive the request for the predictive query against the indices stored in the predictive database system comprises the request interface to receive a predictive query or a latent structure query requesting a result from the indices generated by processing the dataset; and

wherein the API to query the database for the predictive result based on the request comprises the API to execute the query against the generated indices prior to terminating processing of the dataset.

24. The system of claim 23:

wherein the request interface is to further return a predictive record set responsive to the predictive query or the latent structure query requesting the result; and

wherein the request interface is to additionally return a notification with the result indicating processing of the dataset has not yet completed or a notification with the result indicating the confidence quality measure is below the specified threshold, or both.

25. The system of claim 17, wherein the analysis engine to process the dataset comprises the analysis engine to learn at least one of:

a Dirichlet Process Mixture Model (DPMM) of the dataset;

a cross categorization of the dataset;

an Indian buffet process model of the dataset; and

a mixture model or a mixture of finite mixtures model of the dataset.