Systems, methods, and apparatuses for implementing a predict command with a predictive query interface

• US 9,342,836 B2
• Filed: 08/29/2013
• Issued: 05/17/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Cronin, Beau David, Petschulat, Cap Christian, Jonas, Eric Michael
• Primary Examiner(s): TRUONG, CAM Y T
• Application Number:
  US14/014,225
• Publication Number:
  US 20140280191A1
• Time in Patent Office: 992 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/2445 Data retrieval commands; View definitions
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24558 Binary matching operations
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2458 Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/18 : of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0201 : Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0202 : Market predictions or forecasting for commercial activities

Abstract

Disclosed herein are systems and methods for implementing a PREDICT command with a predictive query interface including means for generating indices from a dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the dataset; storing the indices within a database of a host organization; exposing the database of the host organization via a request interface; receiving, at the request interface, a query for the database specifying at least (i) a PREDICT command term, (ii) one or more specified columns to be predicted, and (iii) one or more column name=value pairs specifying column names to be fixed and the values by which to fix them; and querying the database using the PREDICT command term and passing the one or more specified columns to be predicted and the one or more column name=value pairs to generate a representation of a joint conditional distribution of the one or more specified columns to be predicted fixed according to the column name=value pairs using the indices stored in the database. Other related embodiments are further disclosed.

Claims

1. A computer implemented method in a host organization, the method comprising:

generating, by using a computer system, indices from a dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the dataset;

storing the indices within a database of the host organization;

exposing the database of the host organization via a request interface;

receiving, at the request interface, a query for the database specifying at least (i) a PREDICT command term, (ii) one or more specified columns to be predicted, and (iii) one or more column name=value pairs specifying column names to be fixed and corresponding values by which to fix the column names; and

querying the database using the PREDICT command term and passing the one or more specified columns to be predicted and the one or more column name=value pairs to generate a representation of a joint conditional distribution of the one or more specified columns to be predicted fixed according to the column name=value pairs using the indices stored in the database, wherein querying the database using the PREDICT command term comprises passing a JavaScript Object Notation (JSON) structured query to the database, the JSON structured query having a query syntax of;

the PREDICT command term as a required term;

required specification of the one or more specified columns to be predicted;

the required specification of the column names to be fixed and the values by which to fix the column names as the one or more column name=value pairs restricting output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified via the one or more column name=value pairs; and

an optional specification of one or more tables, datasets, data sources, and indices to be queried.

2. The method of claim 1, further comprising:

generating a predictive record set responsive to the querying;

wherein the predictive record set comprises a plurality of elements therein, each element of the elements specifying a value for each specified column of the one or more specified columns to be predicted; and

returning the predictive record set responsive to the query.

3. The method of claim 1, wherein exposing the database of the host organization comprises exposing a Predictive Query Language Application Programming Interface (PreQL API) directly to authenticated users, wherein the PreQL API is accessible to the authenticated users via a public Internet.

4. The method of claim 3, wherein querying the database using the PREDICT command term comprises passing a PreQL query to the database, the PreQL query having a query syntax of:

the PREDICT command term as a required term;

a required TARGET term specifying the one or more specified columns to be predicted;

a required WHERE term that specifies the column names to be fixed and the values by which to fix them as the one or more column name=value pairs, wherein the required WHERE term restricts output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified; and

an optional FROM term specifying one or more tables, datasets, data sources, and/or indices to be queried, when the optional FROM term is specified.

5. The method of claim 1:

wherein exposing the database of the host organization comprises exposing a web form directly to authenticated users, wherein the web form is accessible to the authenticated users via a public Internet;

wherein the host organization generates a predictive query for submission to the database based on input from the web form; and

wherein querying the database using the PREDICT command term comprises querying the database using the predictive query via a Predictive Query Language Application Programming Interface (PreQL API) within the host organization, the PreQL API being exposed indirectly to the authenticated users through the web form.

6. The method of claim 1, further comprising:

returning a predictive record set specifying a predicted value for each column of the columns originally in the dataset.

7. The method of claim 1, further comprising:

returning a synthetic data set responsive to the querying, wherein the synthetic data comprises synthetic rows having data therein which is consistent with the rows and the columns originally with the dataset according to the indices'"'"' probabilistic relationships between the rows and the columns but does not include any original record of the dataset.

8. The method of claim 7, wherein returning the synthetic dataset comprises at least one of:

anonymizing financial records from the dataset;

anonymizing medical records from the dataset; and

anonymizing Internet user records from the dataset.

9. The method of claim 1, further comprising:

returning distributions based on probabilistic relationships between the rows and the columns of the dataset using the indices; and

wherein the distributions returned comprise synthetic data from the indices which are mathematically derived from the columns and rows of the dataset but contain information about data that was not in any original record of the dataset and further in which the indices from which the distributions are derived are not constrained to the scope of the data of the original records of the dataset.

10. The method of claim 9, further comprising returning at least one of:

a confidence score for the distributions, wherein the confidence score ranges from 0 to 1 with 0 indicating no confidence in the predicted value and with 1 indicating a highest possible confidence in the predicted value; and

confidence intervals indicating a minimum and maximum value between which there is a certain confidence value lies.

11. The method of claim 9, wherein returning the distributions based on the probabilistic relationships, further comprises:

passing an optional record count term with the PREDICT command term when querying the database, the optional record count term specifying a quantity of records to be returned responsive to the querying; and

determining a required quantity of processing resources necessary to return the quantity of records specified by the record count.

12. The method of claim 9, wherein returning the distributions based on the probabilistic relationships, further comprises:

passing a minimum accuracy threshold with the PREDICT command term when querying the database; and

determining a required population of samples to be returned to satisfy the minimum accuracy threshold as a lower bound.

13. The method of claim 1:

wherein querying the database using the PREDICT command term comprises executing a Predictive Query Language (PreQL) structured query against the database for the PREDICT command term; and

wherein the method further comprises executing one or more additional PreQL structured queries against the database, each additional PreQL structured query of the one or more additional PreQL structured queries specifying at least one command selected from the group of PreQL commands comprising;

PREDICT, RELATED, SIMILAR, and GROUP.

14. The method of claim 1, further comprising:

receiving the dataset from an authenticated subscriber and subsequently receiving the query for the database from the authenticated subscriber; and

processing the dataset on behalf of the authenticated subscriber to generate the indices.

15. The method of claim 1:

wherein each row of the plurality of rows in the dataset corresponds to an entity;

wherein each column of the plurality of columns corresponds to a characteristic for the entities; and

wherein a point of intersection between each respective row and each column of the plurality of columns forms a cell to store a value at the point of intersection.

16. Non-transitory computer readable storage medium having instructions stored thereon that, when executed by a processor in a host organization, the instructions cause the host organization to perform operations comprising:

generating indices from a dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the dataset;

storing the indices within a database of the host organization;

exposing the database of the host organization via a request interface;

receiving, at the request interface, a query for the database specifying at least (i) a PREDICT command term, (ii) one or more specified columns to be predicted, and (iii) one or more column name=value pairs specifying column names to be fixed and corresponding values by which to fix the column names; and

querying the database using the PREDICT command term and passing the one or more specified columns to be predicted and the one or more column name=value pairs to generate a representation of a joint conditional distribution of the one or more specified columns to be predicted fixed according to the column name=value pairs using the indices stored in the database, wherein querying the database using the PREDICT command term comprises passing a JavaScript Object Notation (JSON) structured query to the database, the JSON structured query having a query syntax of;

the PREDICT command term as a required term;

required specification of the one or more specified columns to be predicted;

the required specification of the column names to be fixed and the values by which to fix the column names as the one or more column name=value pairs restricting output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified via the one or more column name=value pairs; and

an optional specification of one or more tables, datasets, data sources, and indices to be queried.

17. The non-transitory computer readable storage medium of claim 16, wherein exposing the database of the host organization comprises exposing a Predictive Query Language Application Programming Interface (PreQL API) directly to authenticated users, wherein the PreQL API is accessible to the authenticated users via a public Internet.

18. The non-transitory computer readable storage medium of claim 16, wherein querying the database using the PREDICT command term comprises passing a PreQL query to the database, the PreQL query having a query syntax of:

the PREDICT command term as a required term;

a required TARGET term specifying the one or more specified columns to be predicted;

a required WHERE term that specifies the column names to be fixed and the values by which to fix them as the one or more column name=value pairs, wherein the required WHERE term restricts output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix them as specified; and

an optional FROM term specifying one or more tables, datasets, data sources, and indices to be queried, when the optional FROM term is specified.

19. The non-transitory computer readable storage medium of claim 16, wherein querying the database using the PREDICT command term comprises passing a JavaScript Object Notation (JSON) structured query to the database, the JSON structured query having a query syntax of:

the PREDICT command term as a required term;

required specification of the one or more specified columns to be predicted;

required specification of the column names to be fixed and the values by which to fix them as the one or more column name=value pairs restricting output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified via the one or more column name=value pairs; and

an optional specification of one or more tables, datasets, data sources, and/or indices to be queried.

20. A system to operate within a host organization, the system comprising:

a processor to execute instructions stored in memory of the system;

an analysis engine to generate indices from a dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the dataset;

a predictive database to store the indices;

a request interface to expose the predictive database;

the request interface to receive a query for the database specifying at least (i) a PREDICT command term, (ii) one or more specified columns to be predicted, and (iii) one or more column name=value pairs specifying column names to be fixed and corresponding values by which to fix the column names; and

a query interface to query the predictive database using the PREDICT command term and passing the one or more specified columns to be predicted and the one or more column name=value pairs to generate a representation of a joint conditional distribution of the one or more specified columns to be predicted fixed according to the column name=value pairs using the indices stored in the database, wherein querying the database using the PREDICT command term comprises passing a JavaScript Object Notation (JSON) structured query to the database, the JSON structured query having a query syntax of;

the PREDICT command term as a required term;

required specification of the one or more specified columns to be predicted;

the required specification of the column names to be fixed and the values by which to fix the column names as the one or more column name=value pairs restricting output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified via the one or more column name=value pairs; and

an optional specification of one or more tables, datasets, data sources, and indices to be queried.

21. The system of claim 20, wherein the predictive database is to execute as an on-demand cloud based service at the host organization for one or more subscribers;

and

wherein the system further comprises an authenticator to verify client devices are associated with a subscriber and to further verify authentication credentials for the respective subscriber.

22. The system of claim 20, wherein the request interface exposes a Predictive Query Language Application Programming Interface (PreQL API) directly to authenticated users, wherein the PreQL API is accessible to the authenticated users via a public Internet.

23. The system of claim 20, wherein the query interface is to pass a Predictive Query Language (PreQL) structured query to the predictive database to execute the query, the PreQL structured query having a query syntax of:

the PREDICT command term as a required term;

a required TARGET term specifying the one or more specified columns to be predicted;

a required WHERE term that specifies the column names to be fixed and the values by which to fix the column names as the one or more column name=value pairs, wherein the required WHERE term restricts output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified; and

an optional FROM term specifying one or more tables, datasets, data sources, and/or indices to be queried, when the optional FROM term is specified.

24. The system of claim 20, further comprising:

a web-server to implement the request interface;

wherein the web-server is to receive as input, a plurality of access requests from one or more client devices from among a plurality of customer organizations communicably interfaced with the host organization via a network;

a multi-tenant database system with predictive database functionality to implement the predictive database; and

wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.

25. A computer-implemented method in a host organization, the method comprising:

generating, by using a computer, indices from a dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the dataset;

storing the indices within a database of the host organization;

exposing the database of the host organization via a request interface;

receiving, at the request interface, a query for the database specifying a PREDICT command term and with zero columns fixed so that no column identifiers (IDs) are passed with the PREDICT command term; and

querying the database using the PREDICT command term and the one or more specified columns to generate output, wherein the output comprises generated synthetic data having one or more entity rows with predicted values for every column of the dataset using the indices stored in the database, wherein querying the database using the PREDICT command term comprises passing a JavaScript Object Notation (JSON) structured query to the database, the JSON structured query having a query syntax of;

the PREDICT command term as a required term;

required specification of the one or more specified columns to be predicted;

the required specification of the column names to be fixed and corresponding values by which to fix the column names as the one or more column name=value pairs restricting output of the query to a predictive record set having returned elements that are probabilistically related to the one or more columns to be fixed and the corresponding values by which to fix the column names as specified via the one or more column name=value pairs; and

an optional specification of one or more tables, datasets, data sources, and indices to be queried.

26. The method of claim 25, further comprising:

returning the generated synthetic data having one or more entity rows with predicted values for every column of the dataset as a synthetic data set responsive to the querying, wherein the generated synthetic data comprises synthetic rows having data therein which is consistent with the rows and the columns originally with the dataset according to the indices'"'"' probabilistic relationships between the rows and the columns but does not include any original record of the dataset.

27. The method of claim 26, wherein returning the synthetic dataset comprises at least one of:

anonymizing financial records from the dataset;

anonymizing medical records from the dataset; and

anonymizing Internet user records from the dataset.