Systems, methods, and apparatuses for implementing predictive query interface as a cloud service

• US 9,240,016 B2
• Filed: 08/29/2013
• Issued: 01/19/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Cronin, Beau David, Petschulat, Cap Christian, Jonas, Eric Michael, Gasner, Max
• Primary Examiner(s): Uddin, Md. I
• Application Number:
  US14/014,258
• Publication Number:
  US 20140278754A1
• Time in Patent Office: 873 Days
• Field of Search: 707/722, 707/737, 707/739, 707/741
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/2445 Data retrieval commands; View definitions
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24558 Binary matching operations
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2458 Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/18 : of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0201 : Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0202 : Market predictions or forecasting for commercial activities

Abstract

Disclosed herein are systems and methods for implementing predictive query interface as a cloud service including means for exposing an interface to client devices operating remotely from a host organization, wherein the interface is accessible by the client devices via a public Internet; executing a predictive database at the host organization as an on-demand cloud based service for one or more subscribers; authenticating one of the client devices by verifying the client device is associated with one of the subscribers and based further on authentication credentials for the respective subscriber; receiving a request from the authenticated subscriber via the interface; executing a predictive query or a latent structure query against indices of the predictive database generated from a dataset of columns and rows on behalf of the authenticated subscriber, the indices representing probabilistic relationships between the rows and the columns of the dataset; and returning a predictive record set to the authenticated subscriber responsive to the request. Other related embodiments are further disclosed.

Claims

1. A method in a host organization, the method comprising:

exposing an interface to client devices operating remotely from the host organization, wherein the interface is accessible by the client devices via a public Internet;

executing a predictive database at the host organization as an on-demand cloud based service for one or more subscribers;

authenticating one of the client devices by verifying the client device is associated with one of the subscribers and based further on authentication credentials for the respective subscriber;

processing a dataset of columns and rows to generate indices on behalf of the authenticated subscriber, the indices representing probabilistic relationships between the rows and the columns of the dataset, wherein the processing comprises;

(i) processing the dataset by iteratively learning joint probability distributions over the dataset to generate the indices, (ii) periodically determining a predictive quality measure of the indices generated by the processing of the dataset, and (iii) terminating processing of the dataset when the predictive quality measure attains a specified threshold;

receiving a prediction request from the authenticated subscriber via the interface;

executing a query against the indices of the predictive database generated from the dataset;

returning a prediction result of the query to the authenticated subscriber responsive to the prediction request; and

returning a notification with the prediction result indicating processing of the stored dataset has not yet completed or returning a notification with the prediction result indicating the predictive quality measure is below the specified threshold, or returning both with the prediction result.

2. The method of claim 1, wherein executing the query comprises executing a Predictive Query Language (PreQL) query against the predictive database.

3. The method of claim 2, wherein executing the PreQL query comprises querying the predictive database by specifying at least one command selected from the group of PreQL commands comprising:

PREDICT, RELATED, SIMILAR, and GROUP.

4. The method of claim 2, wherein receiving the prediction request comprises receiving the PreQL query from the authenticated subscriber via a Predictive Query Language (PreQL) API exposed directly to the authenticated subscriber via the interface.

5. The method of claim 2, wherein receiving the prediction request comprises:

presenting a web form to the authenticated subscriber;

receiving inputs from the authenticated subscriber via the web form;

generating a PreQL query on behalf of the authenticated subscriber based on the inputs;

querying the predictive database via a Predictive Query Language (PreQL) API by specifying at least one command selected from the group of PreQL commands comprising;

PREDICT, RELATED, SIMILAR, and GROUP.

6. The method of claim 1:

wherein the authenticated subscriber accesses the on-demand cloud based service via a web-browser provided by a third party different than the host organization; and

wherein the authenticated subscriber submits the prediction request to the host organization and receives the prediction result from the host organization without installing any software from the host organization on the client device.

7. The method of claim 1, further comprising:

receiving the dataset from the authenticated subscriber prior to receiving the prediction request from the authenticated subscriber; and

processing the dataset on behalf of the authenticated subscriber to generate the indices, each of the indices representing probabilistic relationships between the rows and the columns of the dataset.

8. The method of claim 7, wherein receiving the dataset comprises at least one of:

receiving the dataset as a table having the columns and rows;

receiving a spreadsheet document and extracting the dataset from the spreadsheet document;

receiving the dataset as a binary file created by a database;

receiving one or more queries to a database and responsively receiving the dataset by executing the one or more queries against the database and capturing a record set returned by the one or more queries as the dataset;

receiving a name of a table in a database and retrieving the table from the database as the dataset;

receiving search parameters for a specified website and responsively querying the search parameters against the specified website and capturing search results as the dataset; and

receiving a link and authentication credentials for a remote repository and responsively authenticating with the remote repository and retrieving the dataset via the link.

9. The method of claim 7, wherein processing the dataset on behalf of the authenticated subscriber comprises learning a joint probability distribution over the dataset to identify and describe the probabilistic relationships between elements of the dataset.

10. The method of claim 9, wherein the processing is triggered automatically responsive to receiving the dataset, and wherein learning the joint probability distribution is controlled by specified configuration parameters, the specified configuration parameters including one or more of:

a maximum period of time for processing the dataset;

a maximum number of iterations for processing the dataset;

a minimum number of iterations for processing the dataset;

a maximum amount of customer resources to be consumed by processing the dataset;

a maximum subscriber fee to be expended processing the dataset;

a minimum threshold predictive quality level to be attained by the processing of the dataset;

a minimum improvement to a predictive quality measure required for the processing to continue; and

a minimum or maximum number of the indices to be generated by the processing.

11. The method of claim 7:

wherein the host organization comprises a plurality of application servers;

wherein the processing further comprises distributing the generation of the indices and storing of the indices amongst multiple of the application servers; and

wherein executing the query against indices of the predictive database comprises simultaneously querying the generated indices in parallel against the multiple of the application servers to which the indices were distributed and stored.

12. The method of claim 11:

wherein querying the generated indices in parallel yields different results from different versions of the indices at the multiple of the application servers to which the indices were distributed and stored;

wherein the method further comprises aggregating the different results via a join operation; and

returning one result set responsive to the executed query or one prediction responsive to the prediction request.

13. The method of claim 11:

wherein a greater quantity of the generated indices corresponds to an improved prediction accuracy; and

wherein a greater quantity of the application servers to which the indices are distributed and stored corresponds to an improved response time for executing the query.

14. The method of claim 1, wherein the predictive quality measure is determined by comparing a known result corresponding to known and non-null values within the dataset with a predictive result obtained by querying the indices generated by the processing of the dataset.

15. The method of claim 1, further comprising:

receiving a specified data source from the authenticated subscriber; and

periodically updating the indices based on the specified data source.

16. The method of claim 15, wherein periodically updating the indices comprises one of:

initiating a polling mechanism to check for changes at the specified data source and retrieving the changes when detected for use in updating the indices;

receiving push notifications from the specified data source indicating changes at the specified data source have occurred and accepting the changes for use in updating the indices; and

wherein the updating of the indices occurs without requiring an active authenticated session for the subscriber.

17. The method of claim 1, further comprising:

performing a join operation against two or more tables within the host organization and accessible to the authenticated subscriber;

capturing the output of the join operation as the dataset of rows and columns; and

processing the dataset to generate the indices representing the probabilistic relationships between the rows and the columns of the dataset.

18. The method of claim 17, wherein the authenticated subscriber specifies the two or more tables as input and wherein the host organization generates a query to perform the join operation and automatically initiates processing against the dataset on behalf of the authenticated subscriber.

19. The method of claim 1, further comprising:

generating the indices representing probabilistic relationships between the rows and the columns of the dataset by learning at least one of;

learning a Dirichlet Process Mixture Model (DPMM) of the dataset;

learning a cross categorization of the dataset;

learning an Indian buffet process model of the dataset; and

learning a mixture model or a mixture of finite mixtures model of the dataset.

20. The method of claim 1:

wherein each of the plurality of rows in the dataset corresponds to an entity;

wherein each of the plurality of columns corresponds to a characteristic for the entities; and

wherein a point of intersection between each respective row and each of the plurality of columns forms a cell to store a value at the point of intersection.

21. The method of claim 20:

wherein each entity represents a person, a place, or a thing; and

wherein each characteristic represents a characteristic, feature, aspect, quantity, range, identifier, mark, trait, or observable fact;

wherein each cell stores a data typed value at the point of intersection between each respective row and each of the plurality of columns, the value representing the characteristic for the entity'"'"'s row that intersects a column corresponding to the characteristic; and

wherein each value of every cell either null, different, or the same as any other value of any other cell.

22. A non-transitory computer readable storage medium having instructions stored thereon that, when executed by a processor in a host organization, the instructions cause the host organization to perform operations comprising:

exposing an interface to client devices operating remotely from the host organization, wherein the interface is accessible by the client devices via a public Internet;

executing a predictive database at the host organization as an on-demand cloud based service for one or more subscribers;

authenticating one of the client devices by verifying the client device is associated with one of the subscribers and based further on authentication credentials for the respective subscriber;

processing a dataset of columns and rows to generate indices on behalf of the authenticated subscriber, the indices representing probabilistic relationships between the rows and the columns of the dataset, wherein the processing comprises;

(i) processing the dataset by iteratively learning joint probability distributions over the dataset to generate the indices, (ii) periodically determining a predictive quality measure of the indices generated by the processing of the dataset, and (iii) terminating processing of the dataset when the predictive quality measure attains a specified threshold;

receiving a prediction request from the authenticated subscriber via the interface;

executing a query against the indices of the predictive database generated from the dataset;

returning a prediction result of the query to the authenticated subscriber responsive to the prediction request; and

returning a notification with the prediction result indicating processing of the stored dataset has not yet completed or returning a notification with the prediction result indicating the predictive quality measure is below the specified threshold, or returning both with the prediction result.

23. The non-transitory computer readable storage medium of claim 22:

wherein executing the query comprises executing a Predictive Query Language (PreQL) query against the predictive database; and

wherein executing the PreQL query comprises querying the predictive database by specifying at least one command selected from the group of PreQL commands comprising;

PREDICT, RELATED, SIMILAR, and GROUP.

24. The non-transitory computer readable storage medium of claim 22, wherein the instructions cause the host organization to perform operations further comprising:

receiving the dataset from the authenticated subscriber;

wherein receiving the dataset comprises at least one of;

(a) receiving the dataset as a table having the columns and rows, (b) receiving a spreadsheet document and extracting the dataset from the spreadsheet document, (c) receiving the dataset as a binary file created by a database, (d) receiving one or more queries to a database and responsively receiving the dataset by executing the one or more queries against the database and capturing a record set returned by the one or more queries as the dataset, (e) receiving a name of a table in a database and retrieving the table from the database as the dataset, (f) receiving search parameters for a specified website and responsively querying the search parameters against the specified website and capturing search results as the dataset, and (g) receiving a link and authentication credentials for a remote repository and responsively authenticating with the remote repository and retrieving the dataset via the link.

25. A system to operate within a host organization, the system comprising:

a processor to execute instructions stored in memory of the system;

a request interface exposed to client devices that operate remotely from the host organization, wherein the request interface is accessible by the client devices via a public Internet;

a predictive database to execute as an on-demand cloud based service for one or more subscribers;

an authenticator to verify client devices are associated with a subscriber and to further verify authentication credentials for the respective subscriber;

an analysis engine to process the dataset and generate the indices representing probabilistic relationships between the rows and the columns of the dataset, wherein the analysis engine to process the dataset comprises the analysis engine being adapted to;

(i) process the dataset by iteratively learning joint probability distributions over the dataset to generate the indices, (ii) periodically determine a predictive quality measure of the indices generated by the processing of the dataset, and (iii) terminate processing of the dataset when the predictive quality measure attains a specified threshold;

the request interface to receive as input, a prediction request from the subscriber;

one or more application servers to execute a query against indices of the predictive database generated from a dataset of columns and rows on behalf of the subscriber, the indices representing probabilistic relationships between the rows and the columns of the dataset;

the request interface to return a prediction result of the query to the subscriber responsive to the prediction request; and

the request interface to further return a notification with the prediction result that indicates processing of the stored dataset has not yet completed or to return a notification with the prediction result that indicates the predictive quality measure is below the specified threshold, or to return both with the prediction result.

26. The system of claim 25, further comprising:

a web-server to implement the request interface;

wherein the web-server is to receive as input, a plurality of access requests from one or more client devices from among a plurality of customer organizations communicably interfaced with the host organization via a network traversing at least a portion of the public Internet;

a multi-tenant database system with predictive database functionality to implement the predictive database; and

wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.

27. The system of claim 25, further comprising:

wherein the predictive database is to store the generated indices; and

a Predictive Query Language Application Programming Interface (PreQL API) exposed to the subscribers at the request interface, wherein the PreQL API is to accept PreQL queries having at least at least one command selected from the group of PreQL commands comprising;

PREDICT, RELATED, SIMILAR, and GROUP;

wherein the PreQL API executes the PreQL queries against the predictive database and returns predictive results.

28. The system of claim 25:

wherein the request interface is to further receive the dataset from the authenticated subscriber;

wherein the request interface to receive the dataset comprises at least one of;

(a) receiving the dataset as a table having the columns and rows, (b) receiving a spreadsheet document and extracting the dataset from the spreadsheet document, (c) receiving the dataset as a binary file created by a database, (d) receiving one or more queries to a database and responsively receiving the dataset by executing the one or more queries against the database and capturing a record set returned by the one or more queries as the dataset, (e) receiving a name of a table in a database and retrieving the table from the database as the dataset, (f) receiving search parameters for a specified website and responsively querying the search parameters against the specified website and capturing search results as the dataset, and (g) receiving a link and authentication credentials for a remote repository and responsively authenticating with the remote repository and retrieving the dataset via the link.