User interface based variable machine modeling

• US 10,552,002 B1
• Filed: 07/20/2017
• Issued: 02/04/2020
• Est. Priority Date: 09/27/2016
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Maclean, Matthew, Duffield, Benjamin, Elliot, Mark
• Primary Examiner(s): Perveen, Rehana
• Assistant Examiner(s): Mikowski, Justin C
• Application Number:
  US15/655,408
• Time in Patent Office: 929 Days
• Field of Search: 703 2
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

In various example embodiments, a comparative modeling system is configured to receive selections of a data set, a transform scheme, and one or more machine-learning algorithms. In response to a selection of the one or more machine-learning algorithms, the comparative modeling system determines parameters within the one or more machine-learning algorithms. The comparative modeling system generates a plurality of models for the one or more machine-learning algorithms, determines comparison metric values for the plurality of models, and causes presentation of the comparison metric values for the plurality of models.

Claims

1. A method, comprising:

causing, by one or more processors of a machine, presentation of a graphical user interface having a set of selectable graphical interface elements including a first graphical interface element representing a set of data sets, a second graphical interface element representing a set of transform families, and a third graphical interface element representing a set of model families, each transform family comprising a particular transform scheme for transforming one or more values of a data set from one form to another form, each model family comprising a family identification and a set of code for a particular machine-learning algorithm that generates a particular machine-learning model for one or more values;

receiving, by the one or more processors of a machine, a selection of a particular data set through the graphical user interface, the particular data set including a set of values associated with a set of identifiers;

receiving, by the one or more processors of the machine, a selection of a transform scheme through the graphical user interface, the transform scheme configured to transform one or more values of the particular data set from a first form to a second form;

receiving, by the one or more processors of the machine, a selection of a first machine-learning algorithm and a second machine-learning algorithm through the graphical user interface, the first machine-learning algorithm configured to generate a first machine-learning model for the set of values and the second machine-learning algorithm configured to generate a second machine-learning model for the set of values;

in response to selection of the first machine-learning algorithm and the second machine-learning algorithm, determining a first iteration value for each given first parameter of two or more first parameters within the first machine-learning algorithm, and determining a second iteration value for each given second parameter of two or more second parameters within the second machine-learning algorithm;

iteratively executing, by the one or more processors of the machine, the first machine-learning algorithm, using the two or more first parameters, according to a first iteration order and the first iteration value to process the set of values and generate a plurality of first machine-learning models;

iteratively executing, by the one or more processors of the machine, the second machine-learning algorithm, using the two or more second parameters, according to a second iteration order and the second iteration value to process the set of values and generate a plurality of second machine-learning models;

determining, by the one or more processors of the machine, one or more comparison metric values for data output by each of the plurality of first machine-learning models and the plurality of second machine-learning models; and

causing presentation, by the one or more processors of the machine, of the comparison metric values for the data output by the plurality of first machine-learning models and the plurality of second machine-learning models, the presentation comprising a selectable user interface element configured to cause the presentation of a result of at least one of a first machine learning model or a second machine learning model, the result comprising at least one of the comparison metric values.

2. The method of claim 1, further comprising:

determining a first iteration order for the two or more first parameters within the first machine-learning algorithm and a second iteration order for the two or more second parameters within the second machine-learning algorithm.

3. The method of claim 1, wherein iteratively modeling the set of values according to the first machine-learning algorithm further comprises:

generating a first set of machine-learning models for the set of values, each machine-learning model of the first set of machine-learning models corresponding to a different first iteration value of each first parameter of the two or more first parameters within the first machine-learning algorithm.

4. The method of claim 1, wherein iteratively modeling the set of values according to the second machine-learning algorithm further comprises:

generating a second set of machine-learning models for the set of values, each machine-learning model of the second set of machine-learning models corresponding to a different second iteration value of each second parameter of the two or more second parameters within the second machine-learning algorithm.

5. The method of claim 1, wherein the set of model families comprises a predetermined set of model families, the method further comprising:

receiving an additional model family including another family identification and another set of code for an additional machine-learning algorithm for generating another model for the set of values;

incorporating the additional model family into the set of model families; and

generating a selectable graphical interface element for the additional model family within the third graphical interface element.

6. A computer implemented system, comprising:

one or more processors; and

a processor-readable storage device comprising processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

causing presentation of a graphical user interface having a set of selectable graphical interface elements including a first graphical interface element representing a set of data sets, a second graphical interface element representing a set of transform families, and a third graphical interface element representing a set of model families, each transform family comprising a particular transform scheme for transforming one or more values of a data set from one form to another form, each model family comprising a family identification and a set of code for a particular machine-learning algorithm that generates a particular machine-learning model for one or more values;

receiving a selection of a particular data set through the graphical user interface, the particular data set including a set of values associated with a set of identifiers;

receiving a selection of a transform scheme through the graphical user interface, the transform scheme configured to transform one or more values of the particular data set from a first form to a second form;

receiving a selection of a first machine-learning algorithm and a second machine-learning algorithm through the graphical user interface, the first machine-learning algorithm configured to generate a first machine-learning model for the set of values and the second machine-learning algorithm configured to generate a second machine-learning model for the set of values;

in response to selection of the first machine-learning algorithm and the second machine-learning algorithm, determining a first iteration value for each given first parameter of two or more first parameters within the first machine-learning algorithm, and determining a second iteration value for each given second parameter of two or more second parameters;

iteratively executing the first machine-learning algorithm, using the two or more first parameters, according to a first iteration order and the first iteration value to process the set of values and generate a plurality of first machine-learning models;

iteratively executing the second machine-learning algorithm, using the two or more second parameters, according to a second iteration order and the second iteration value to process the set of values and generate a plurality of second machine-learning models;

determining one or more comparison metric values for data output by each of the plurality of first machine-learning models and the plurality of second machine-learning models; and

causing presentation of the comparison metric values for the data output by the plurality of first machine-learning models and the plurality of second machine-learning models, the presentation comprising a selectable user interface element configured to cause the presentation of a result of at least one of a first machine learning model or a second machine learning model, the result comprising at least one of the comparison metric values.

7. The system of claim 6, wherein the operations further comprise:

determining a first iteration order for the two or more first parameters within the first machine-learning algorithm and a second iteration order for the two or more second parameters within the second machine-learning algorithm.

8. The system of claim 6, wherein iteratively modeling the set of values according to the first machine-learning algorithm further comprises:

generating a first set of machine-learning models for the set of values, each machine-learning model of the first set of machine-learning models corresponding to a different first iteration value of each first parameter of the two or more first parameters within the first machine-learning algorithm.

9. The system of claim 6, wherein iteratively modeling the set of values according to the second machine-learning algorithm further comprises:

generating a second set of machine-learning models for the set of values, each machine-learning model of the second set of machine-learning models corresponding to a different second iteration value of each second parameter of the two or more second parameters within the second machine-learning algorithm.

10. The system of claim 6, wherein the set of model families comprises a predetermined set of model families, the operations further comprising:

receiving an additional model family including another family identification and another set of code for a machine-learning algorithm for generating another model for the set of values;

incorporating the additional model family into the set of model families; and

generating a selectable graphical interface element for the additional model family within the third graphical interface element.

11. A processor-readable storage device comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

causing presentation of a graphical user interface having a set of selectable graphical interface elements including a first graphical interface element representing a set of data sets, a second graphical interface element representing a set of transform families, and a third graphical interface element representing a set of model families, each transform family comprising a particular transform scheme for transforming one or more values of a data set from one form to another form, each model family comprising a family identification and a set of code for a particular machine-learning algorithm that generates a particular machine-learning model for one or more values;

receiving a selection of a particular data set through the graphical user interface, the particular data set including a set of values associated with a set of identifiers;

receiving a selection of a transform scheme through the graphical user interface, the transform scheme configured to transform one or more values of the particular data set from a first form to a second form;

receiving selection of a first machine-learning algorithm and a second machine-learning algorithm through the graphical user interface, the first machine-learning algorithm configured to generate a first machine-learning model for the set of values and the second machine-learning algorithm configured to generate a second machine-learning model for the set of values;

in response to selection of the first machine-learning algorithm and the second machine-learning algorithm, determining a first iteration value for each given first parameter of two or more first parameters within the first machine-learning algorithm, and determining a second iteration value for each given second parameter of two or more second parameters within the second machine-learning algorithm;

iteratively executing the first machine-learning algorithm, using the two or more first parameters, according to a first iteration order and the first iteration value to process the set of values and generate a plurality of first machine-learning models;

iteratively executing the second machine-learning algorithm, using the two or more second parameters, according to a second iteration order and the second iteration value to process the set of values and generate a plurality of second machine-learning models;

determining one or more comparison metric values for data output by each of the plurality of first machine-learning models and the plurality of second machine-learning models; and

causing presentation of the comparison metric values for the data output by the plurality of first machine-learning models and the plurality of second machine-learning models, the presentation comprising a selectable user interface element configured to cause the presentation of a result of at least one of a first machine learning model or a second machine learning model, the result comprising at least one of the comparison metric values.

12. The processor-readable storage device of claim 11, wherein the operations further comprise:

determining a first iteration order for the two or more first parameters within the first machine-learning algorithm and a second iteration order for the two or more second parameters within the second machine-learning algorithm.

13. The processor-readable storage device of claim 11, wherein iteratively modeling the set of values according to the first machine-learning algorithm further comprises:

generating a first set of machine-learning models for the set of values, each machine-learning model of the first set of machine-learning models corresponding to a different first iteration value of each first parameter of the two or more first parameters within the first machine-learning algorithm.

14. The processor-readable storage device of claim 11, wherein iteratively modeling the set of values according to the second machine-learning algorithm further comprises:

generating a second set of machine-learning models for the set of values, each machine-learning model of the second set of machine-learning models corresponding to a different second iteration value of each second parameter of the two or more second parameters within the second machine-learning algorithm.