Feature processing recipes for machine learning

• US 9,886,670 B2
• Filed: 06/30/2014
• Issued: 02/06/2018
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Dirac, Leo Parker, Agarwal, Tarun
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Wu, Fuming
• Application Number:
  US14/319,880
• Publication Number:
  US 20150379423A1
• Time in Patent Office: 1,317 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A first representation of a feature processing recipe is received at a machine learning service. The recipe includes a section in which groups of variables on which common transformations are to be applied are defined, and a section in which a set of transformation operations are specified. The first representation of the recipe is validated based at least in part on a library of function definitions supported by the service, and an executable version of the recipe is generated. In response to a determination that the recipe is to be executed on a particular data set, a set of provider network resources is used to implement a transformation operation indicated in the recipe.

Claims

1. A system, comprising:

one or more computing devices configured to;

implement a network-based interface for a network-accessible machine learning service of a provider network, the network-based interface configured to receive text representations of different client recipes from a plurality of respective clients of the network-accessible machine learning service, wherein respective client recipes received via the network-based interface specify one or more transformations to be performed on respective data sets used for model training or prediction by the machine learning service, the respective text representations of client recipes comprising one or more of;

(a) a group definitions section indicating one or more groups of variables, wherein individual ones of the one or more groups comprise a plurality of variables on which at least one common transformation operation is to be applied, (b) an assignment section defining one or more intermediate variables, (c) a dependency section indicating respective references to one or more machine learning artifacts stored in a repository, or (d) an output section indicating one or more transformation operations to be applied to at least one entity indicated in the group definitions section, the assignment section, or the dependency section;

validate, in accordance with (a) a set of syntax rules defined by the machine learning service and (b) a set of library function definitions for transformation operation types supported by the machine learning service, a given text representation of a given client recipe received via the network-based interface;

generate, from the given text representation, an executable representation of the given client recipe;

store the executable representation in the repository;

determine that the given client recipe is to be applied to a particular data set;

verify that the particular data set meets a run-time acceptance criterion of the given client recipe; and

execute the executable representation of the given client recipe, using one or more selected provider network resources, to apply a particular transformation operation specified in the given client recipe to the particular data set.

2. The system as recited in claim 1, wherein the one or more computing devices are further configured to:

receive a request to apply the given client recipe to a different data set;

verify that the different data set meets the run-time acceptance criterion of the given client recipe; and

apply the particular transformation operation to the different data set.

3. The system as recited in claim 1, wherein the network-based interface is a programmatic interface, and the one or more computing devices are further configured to:

provide, to a client via the programmatic interface, an indication of a respective set of one or more recipes applicable to individual ones of a plurality of machine learning problem domains.

4. The system as recited in claim 1, wherein the given text representation comprises an indication of a particular machine learning model to be executed using a result of the particular transformation operation.

5. The system as recited in claim 1, wherein the one or more computing devices are further configured to:

determine, in response to an indication that automated parameter tuning is to be performed for the given client recipe, a plurality of parameter value options applicable to a different transformation operation of the one or more transformation operations;

generate, by the machine learning service, respective results of the different transformation operation using individual ones of the plurality of parameter value options; and

provide, by the machine learning service based on an analysis of the respective results, an indication of at least one candidate parameter value of the plurality of parameter value options that meets a parameter acceptance criterion.

6. A method, comprising:

performing, by one or more computing devices;

implementing a network-based interface for a network-accessible machine learning service of a provider network, the network-based interface configured to receive representations of different client recipes from a plurality of respective clients of the network-accessible machine learning service, wherein respective client recipes received via the network-based interface specify one or more transformations to be performed on respective data sets used for model training or prediction by the machine learning service, the respective representations of client recipes comprising one or more of;

(a) a group definitions section indicating one or more groups of variables, wherein individual ones of the one or more groups comprise a plurality of data set variables on which at least one common transformation operation is to be applied and (b) an output section indicating one or more transformation operations to be applied to at least one entity indicated in one or more of;

(i) the group definitions section or (ii) an input data set;

validating, in accordance with at least a set of library function definitions for transformation operation types supported by the machine learning service, a given representation of a given client recipe received via the network-based interface;

generating, from the given representation, an executable representation of the given client recipe;

determining that the given client recipe is to be applied to a particular data set;

verifying that the particular data set meets a run-time acceptance criterion; and

executing, using one or more selected provider network resources, the executable representation of the given client recipe to apply a particular transformation operation specified in the given client recipe to the particular data set.

7. The method as recited in claim 6, wherein the given representation is a text representation or a binary representation.

8. The method as recited in claim 6, wherein the given representation is generated by a client of the machine learning service using a tool obtained from the machine learning service.

9. The method as recited in claim 6, wherein a data type of at least one variable of an input data record of the particular data set comprises one or more of:

(a) text, (b) a numeric data type, (c) Boolean, (d) a binary data type, (d) a categorical data type, (e) an image processing data type, (f) an audio processing data type, (g) a bioinformatics data type, or (h) a structured data type.

10. The method as recited in claim 9, wherein the data type comprises a particular structured data type, further comprising performing, by the one or more computing devices:

selecting, based at least in part on the particular structured data type, a particular library function to be used for the particular transformation operation.

11. The method as recited in claim 6, wherein the given representation comprises an assignment section defining an intermediate variable in terms of one or more of:

(a) an input data set variable or (b) an entity defined in the group definitions section, wherein the intermediate variable is referenced in the output section.

12. The method as recited in claim 6, wherein the given representation comprises a dependency section indicating a reference to a particular artifact stored in a repository of the machine learning service, wherein the particular transformation operation consumes an output of the particular artifact as an input.

13. The method as recited in claim 12, wherein the particular artifact comprises one or more of:

(a) a machine learning model, (b) a different recipe, (c) a statistics set or (d) an alias that includes a reference to a machine learning model.

14. The method as recited in claim 6, wherein the particular transformation operation utilizes a user-defined function, further comprising performing, by the one or more computing devices:

receiving, at the machine learning service from a client prior to said receiving the given representation, an indication of a module implementing the user-defined function, wherein the module is in a text format or a binary format.

15. The method as recited in claim 6, further comprising performing, by the one or more computing devices:

validating the given representation in accordance with a set of syntax rules defined by the machine learning service.

16. The method as recited in claim 6, further comprising performing, by the one or more computing devices:

receiving a request to apply the given client recipe to a different data set;

verifying that the different data set meets the run-time acceptance criterion of the given client recipe; and

applying the particular transformation operation to the different data set.

17. The method as recited in claim 6, wherein the network-based interface comprises a programmatic interface, and further comprising performing, by the one or more computing devices:

providing, to a client via the programmatic interface, an indication of a respective set of one or more recipes applicable to individual ones of a plurality of machine learning problem domains.

18. The method as recited in claim 6, wherein the given representation comprises an indication of a particular machine learning model to be executed using a result of the particular transformation operation.

19. The method as recited in claim 6, further comprising performing, by the one or more computing devices:

determining, by the machine learning service in response to an indication that automated parameter tuning is to be performed for the given client recipe, a plurality of parameter value options applicable to a different transformation operation of the one or more transformation operations;

generating, by the machine learning service, respective results of the different transformation operation using individual ones of the plurality of parameter value options.

20. The method as recited in claim 19, further comprising performing, by the one or more computing devices:

selecting, by the machine learning service, a particular parameter value of the plurality of parameter value options as an acceptable value based at least in part on a particular result set corresponding to the particular parameter value.

21. The method as recited in claim 19, further comprising performing, by the one or more computing devices:

indicating, by the machine learning service to a client, at least a subset of the plurality of parameter value options as candidate values based on an analysis of the respective results; and

receiving, at the machine learning service from the client, an indication of a particular parameter value of the subset to be used for the different transformation operation.

22. The method as recited in claim 19, wherein the plurality of parameter value options comprise one or more of:

(a) respective lengths of n-grams to be derived from a language processing data set, (b) respective quantile bin boundaries for a particular variable, (c) image processing parameter values, (d) a number of clusters into which a data set is to be classified, (e) values for a cluster boundary threshold, or (f) dimensionality values for a vector representation of a text document.

23. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors:

implement a network-based interface for a network-accessible machine learning service of a provider network, the network-based interface configured to receive representations of different client recipes from a plurality of respective clients of the network-accessible machine learning service, wherein respective client recipes received via the network-based interface specify one or more transformations to be performed on respective data sets used for model training or prediction by the machine learning service, the respective representations of client recipes comprising one or more of;

(a) a group definitions section indicating one or more groups of variables, wherein individual ones of the one or more groups comprise a plurality of data set variables on which at least one common transformation operation is to be applied, or (b) an output section indicating one or more transformation operations to be applied to at least one entity indicated in one or more of (i) the group definitions section or (ii) an input data set of the recipe;

validate, in accordance with at least a set of library function definitions for transformation operation types supported by the machine learning service, a given representation of a given client recipe received via the network-based interface;

generate an executable representation of the given client recipe; and

in response to a determination that the given client recipe is to be applied to a particular data set, execute the executable representation of the given client recipe using one or more selected provider network resources to apply a particular transformation operation of the given client recipe to the particular data set.

24. The non-transitory computer-accessible storage medium as recited in claim 23, wherein the given representation comprises an assignment section defining an intermediate variable in terms of one or more of:

(a) an input data set variable or (b) an entity defined in the group definitions section, wherein the intermediate variable is referenced in the output section.

25. The non-transitory computer-accessible storage medium as recited in claim 23, wherein the given representation comprises a dependency section indicating a reference to a particular artifact stored in a repository of the machine learning service, wherein the particular transformation operation consumes an output of the particular artifact as an input.

26. The non-transitory computer-accessible storage medium as recited in claim 23, wherein the particular artifact comprises one or more of:

(a) a machine learning model, (b) a different recipe, (c) an alias or (d) a set of statistics.

27. The non-transitory computer-accessible storage medium as recited in claim 23, wherein the set of library function definitions comprise one or more of:

(a) a quantile bin function, (b) a Cartesian product function, (c) a bi-gram function, (d) an n-gram function, (e) an orthogonal sparse bigram function, (f) a calendar function, (g) an image processing function, (h) an audio processing function, (i) a bio-informatics processing function, or (j) a natural language processing function.