Systems, methods, and apparatuses for populating a table having null values using a predictive query interface

• US 9,235,846 B2
• Filed: 08/29/2013
• Issued: 01/12/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Petschulat, Cap Christian, Jonas, Eric Michael
• Primary Examiner(s): Uddin, Md. I
• Application Number:
  US14/014,264
• Publication Number:
  US 20140280066A1
• Time in Patent Office: 866 Days
• Field of Search: 707/722, 707/737, 707/739, 707/741
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/2445 Data retrieval commands; View definitions
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24558 Binary matching operations
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2458 Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/18 : of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/0201 : Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0202 : Market predictions or forecasting for commercial activities

Abstract

Disclosed herein are systems and methods for populating a table having null values using a predictive query interface including means for receiving a tabular dataset from a user as input, the tabular dataset having data values organized as columns and rows; identifying a plurality of null values within the tabular dataset, the null values being dispersed across multiple rows and multiple columns of the tabular dataset; generating indices from the tabular dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the tabular dataset; displaying the tabular dataset as output to the user, the displayed output including the data values depicted as known values and the null values depicted as unknown values; receiving input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values; querying the indices for the predicted values; and displaying the predicted values as updated output to the user. Other related embodiments are further disclosed.

Claims

1. A method in a host organization, the method comprising:

receiving a tabular dataset from a user as input, the tabular dataset having data values organized as columns and rows;

identifying a plurality of null values within the tabular dataset, the null values being dispersed across multiple rows and multiple columns of the tabular dataset;

generating indices from the tabular dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the tabular dataset;

displaying the tabular dataset as output to the user, the displayed output including the data values depicted as known values and the null values depicted as unknown values;

receiving input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values;

querying the indices for the predicted values;

receiving a confidence indicator for every one of the plurality of null values within the tabular dataset responsive to querying the indices for the predicted values, the confidence indicator based on a comparison of known results corresponding to known and non-null values within the dataset with the predicted values; and

displaying the predicted values as updated output to the user, wherein displaying the predicted values as updated output to the user comprises displaying selected ones of the predicted values that correspond to the confidence indicator being in excess of a default minimum confidence threshold or a user specified minimum confidence threshold when present.

2. The method of claim 1:

wherein generating indices from the tabular dataset of columns and rows further comprises storing the indices within a database of the host organization; and

wherein querying the indices for the predicted values comprises querying the database for the predicted values.

3. The method of claim 1:

wherein receiving input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values comprises receiving input from the user to populate all unknown values within the displayed tabular dataset with predicted values;

wherein querying the indices for the predicted values comprises querying the indices for a predicted value for every null value identified within the tabular dataset; and

wherein displaying the predicted values as updated output to the user comprises replacing all unknown values by displaying corresponding predicted values.

4. The method of claim 1:

wherein the plurality of null values within the tabular dataset are not restricted to any row or column of the tabular dataset; and

wherein displaying the predicted values as updated output to the user replaces the unknown values displayed with the tabular dataset without restriction to any row or column and without changing the indices upon which the predicted values are based.

5. The method of claim 1:

wherein querying the indices for the predicted values comprises querying the indices for each and every one of the identified plurality of null values within the tabular dataset;

wherein the method further comprises receiving the predicted values for each and every one of the identified plurality of null values within the tabular dataset responsive to the querying; and

wherein displaying the predicted values as updated output to the user comprises displaying the received predicted values.

6. The method of claim 1, wherein querying the indices for the predicted values comprises:

generating a Predictive Query Language (PreQL) query specifying a PREDICT command term for each and every one of the identified plurality of null values within the tabular dataset;

issuing each of the generated PreQL queries to a Predictive Query Language Application Programming Interface (PreQL API); and

receiving a predicted result for each and every one of the identified plurality of null values within the tabular dataset responsive to the issued PreQL queries.

7. The method of claim 1, wherein displaying the tabular dataset further comprises:

displaying the known values using black text within cells of a spreadsheet;

displaying the unknown values as blank cells within the spreadsheet; and

displaying the predicted values using colored or grayscale text within the cells of the spreadsheet.

8. The method of claim 1:

wherein displaying the predicted values as updated output to the user comprises displaying the updated output within a spreadsheet or table at a Graphical User Interface (GUI);

wherein the known values are displayed as populated cells within the spreadsheet or table at the GUI in a first type of text;

wherein predicted values are displayed as populated cells within the spreadsheet or table at the GUI in a second type of text discernable from the first type of text corresponding to the known values; and

wherein any remaining unknown values are displayed as empty cells within the spreadsheet or table at the GUI.

9. The method of claim 1:

wherein displaying the tabular dataset as output to the user and displaying the predicted values as updated output to the user comprises displaying the tabular dataset and the predicted values within a spreadsheet or table at a Graphical User Interface (GUI);

wherein the GUI further comprises a slider interface controllable by the user to specify an acceptable degree of uncertainty for the spreadsheet or table; and

wherein receiving input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values comprises receiving the acceptable degree of uncertainty as input from the user via the slider interface.

10. The method of claim 9, further comprising:

displaying a minimum fill percentage for the GUI, wherein the minimum fill percentage corresponds to a percentage of known values within the tabular dataset from a sum of all null values and all known values for the tabular dataset.

11. The method of claim 10, wherein the slider interface controllable by the user to specify the acceptable degree of uncertainty for the spreadsheet or table is restricted to a range encompassing the minimum fill percentage and a maximum degree of uncertainty necessary to completely populate the displayed tabular dataset.

12. The method of claim 11, further comprising:

populating the spreadsheet or table of the GUI to a 100% fill percentage responsive to input by the user specifying a maximum acceptable degree of uncertainty via the slider interface; and

populating all null values by degrading a required confidence threshold for each of the predicted values until a predicted value is available for every one of the plurality of null values within the tabular dataset.

13. The method of claim 1, further comprising:

displaying a user controllable minimum confidence threshold at a Graphical User Interface (GUI) displaying the tabular dataset as output to the user within a spreadsheet or table;

receiving the user specified minimum confidence threshold as input via the user controllable minimum confidence threshold; and

wherein displaying the predicted values as updated output to the user comprises displaying only the predicted values at the GUI having the corresponding confidence indicator equal to or greater than the user specified minimum confidence threshold.

14. The method of claim 1, further comprising:

displaying a user controllable minimum confidence threshold at a Graphical User Interface (GUI) displaying the tabular dataset as output to the user within a spreadsheet or table; and

displaying a maximum fill percentage for the GUI, wherein the maximum fill percentage corresponds to a sum of all known values and all null values returning a predicted value with the confidence indicator in excess of the user controllable minimum confidence threshold as a percentage of a sum of all null values and all known values.

15. The method of claim 1, further comprising:

receiving a distribution for every one of the plurality of null values within the tabular dataset responsive to querying the indices for the predicted values;

calculating a credible interval for each distribution received; and

wherein displaying the predicted values as updated output to the user comprises displaying selected ones of the predicted values that correspond to a calculated credible interval in excess of a minimum threshold.

16. The method of claim 1, wherein displaying the tabular dataset further comprises:

displaying the known values as a first text type within cells of a spreadsheet;

querying the indices for a predicted value corresponding to every one of the unknown values; and

wherein displaying the predicted values as updated output to the user comprises displaying the predicted values as a second text type within the cells of the spreadsheet, wherein the second text type has a displayed opacity in proportion to the confidence indicator for the predicted value displayed.

17. The method of claim 1:

wherein displaying the tabular dataset as output to the user comprises displaying the known values as black text within cells of a spreadsheet; and

wherein displaying the predicted values as updated output to the user comprises displaying the predicted values as grayscale text with the predicted values having a higher confidence indicator being displayed at darker grayscales than the predicted values having a lower confidence indicator.

18. The method of claim 1, further comprising:

displaying a prediction difficulty score for every column of the tabular dataset displayed as output to the user on a per-column basis, wherein the prediction difficulty score is calculated for each column of the tabular dataset by;

(i) identifying all unknown values within the column;

(ii) querying the indices for a predicted value corresponding to each of the unknown values identified within the column;

(iii) receiving the confidence indicator for each of the unknown values identified within the column; and

(iv) calculating the prediction difficulty score for the column based on the confidence indicators received for the unknown values identified within the column.

19. The method of claim 18, wherein the method further comprises:

displaying a maximum fill percentage for every column of the tabular dataset displayed as output to the user on a per-column basis, wherein the maximum fill percentage is based on a quantity of the unknown values identified within the respective column having confidence indicators exceeding a minimum confidence quality threshold.

20. A non-transitory computer readable storage medium having instructions stored thereon that, when executed by a processor in a host organization, the instructions cause the host organization to perform operations comprising:

receiving a tabular dataset from a user as input, the tabular dataset having data values organized as columns and rows;

identifying a plurality of null values within the tabular dataset, the null values being dispersed across multiple rows and multiple columns of the tabular dataset;

generating indices from the tabular dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the tabular dataset;

displaying the tabular dataset as output to the user, the displayed output including the data values depicted as known values and the null values depicted as unknown values;

receiving input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values;

querying the indices for the predicted values;

receiving a confidence indicator for every one of the plurality of null values within the tabular dataset responsive to querying the indices for the predicted values, the confidence indicator based on a comparison of known results corresponding to known and non-null values within the dataset with the predicted values; and

displaying the predicted values as updated output to the user, wherein displaying the predicted values as updated output to the user comprises displaying selected ones of the predicted values that correspond to the confidence indicator being in excess of a default minimum confidence threshold or a user specified minimum confidence threshold when present.

21. The non-transitory computer readable storage medium of claim 20:

wherein displaying the tabular dataset as output to the user and displaying the predicted values as updated output to the user comprises displaying the tabular dataset and the predicted values within a spreadsheet or table at a Graphical User Interface (GUI);

wherein the GUI further comprises a slider interface controllable by the user to specify a fill percentage for the spreadsheet or table; and

wherein receiving input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values comprises receiving the fill percentage as input from the user via the slider interface.

22. The non-transitory computer readable storage medium of claim 20, wherein querying the indices for the predicted values comprises:

generating a Predictive Query Language (PreQL) query specifying a PREDICT command term for each and every one of the identified plurality of null values within the tabular dataset;

issuing each of the generated PreQL queries to a Predictive Query Language Application Programming Interface (PreQL API); and

receiving a predicted result for each and every one of the identified plurality of null values within the tabular dataset responsive to the issued PreQL queries.

23. A system to operate within a host organization, the system comprising:

a processor to execute instructions stored in memory of the system;

a request interface to receive a tabular dataset from a user as input, the tabular dataset having data values organized as columns and rows;

an analysis engine to identify a plurality of null values within the tabular dataset, the null values being dispersed across multiple rows and multiple columns of the tabular dataset;

the analysis engine to further generate indices from the tabular dataset of columns and rows, the indices representing probabilistic relationships between the rows and the columns of the tabular dataset;

the request interface to return the tabular dataset as display output to the user, the display output including the data values depicted as known values and the null values depicted as unknown values;

the request interface to receive input from the user to populate at least a portion of the unknown values within the displayed tabular dataset with predicted values;

a query interface to query the indices for the predicted values;

the query interface to receive a confidence indicator for every one of the plurality of null values within the tabular dataset responsive to the query for the predicted values, the confidence indicator based on a comparison of known results corresponding to known and non-null values within the dataset with the predicted values; and

the request interface to further return the predicted values as updated display output to the user, wherein displaying the predicted values as updated output to the user comprises displaying selected ones of the predicted values that correspond to the confidence indicator being in excess of a default minimum confidence threshold or a user specified minimum confidence threshold when present.

24. The system of claim 23, further comprising:

a predictive database to store the indices; and

wherein the predictive database is to execute as an on-demand cloud based service at the host organization for one or more subscribers.

25. The system of claim 23, further comprising:

an authenticator to verify the user as a known subscriber and to further verify authentication credentials presented by the known subscriber.

26. The system of claim 23, further comprising:

a web-server to implement the request interface;

wherein the web-server is to receive as input, a plurality of access requests from one or more client devices from among a plurality of customer organizations communicably interfaced with the host organization via a network;

a multi-tenant database system with predictive database functionality to implement the predictive database; and

wherein each customer organization is an entity selected from the group consisting of;

a separate and distinct remote organization, an organizational group within the host organization, a business partner of the host organization, or a customer organization that subscribes to cloud computing services provided by the host organization.