Model vector generation for machine learning algorithms

• US 10,068,186 B2
• Filed: 03/20/2015
• Issued: 09/04/2018
• Est. Priority Date: 03/20/2015
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Shi, Xingtian, Li, Wen-Syan
• Primary Examiner(s): Chen, Alan
• Application Number:
  US14/663,701
• Publication Number:
  US 20160275413A1
• Time in Patent Office: 1,264 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/126 Evolutionary algorithms, e.g. genetic algorithms or genetic programming

Abstract

Techniques are described for forming a machine learning model vector, or just model vector, that represents a weighted combination of machine learning models, each associated with a corresponding feature set and parameterized by corresponding model parameters. A model vector generator generates such a model vector for executing automated machine learning with respect to historical data, including generating the model vector through an iterative selection of values for a feature vector, a weighted model vector, and a parameter vector that comprise the model vector. Accordingly, the various benefits of known and future machine learning algorithms are provided in a fast, effective, and efficient manner, which is highly adaptable to many different types of use cases.

Claims

1. A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed, are configured to cause at least one processor to:

determine a model vector <

d_t^h, w_t^h, δ

_t^h>

, in which d_t^h represents a feature vector including t feature subsets of a feature set, w_t^h represents a weighted model vector including t weighted automated learning models, and δ

_t^h represents t parameter sets parameterizing the w_t^h weighted automated learning models;

adjust weights of w_t^h to obtain an updated w_t^h, w_t^h+1, based on performance evaluations of the _t weighted automated learning models, and based on w_t^h;

search a feature solution space to obtain t updated feature subsets of the feature set, to thereby obtain an updated d_t^h, d_t^h+1,

search a parameter solution space to obtain t updated parameter sets, to thereby obtain an updated δ

_t^h, δ

_t^h+1;

determine an optimized model vector (d_t^h+1, w_t^h+1, w_t^h+1);

receive a forecast request for a forecast related to the feature set; and

provide the forecast, using the optimized model vector (d_t^h+1, w_t^h+1, w_t^h+1).

2. The computer program product of claim 1, wherein the feature set is selected from features of historical data against which the model vector will be trained and/or applied.

3. The computer program product of claim 1, wherein the automated learning models are obtained from a model pool including a plurality of automated learning models and associated machine learning algorithms.

4. The computer program product of claim 3, wherein the model pool includes both supervised and unsupervised learning models.

5. The computer program product of claim 1, wherein the instructions, when executed, are configured to cause the at least one processor to:

adjust weights of w_t^h to obtain the updated w_t^h, w_t^h+1, including evaluating weights of w_t^h by applying each of the weighted learning models using historical data, from which the feature set was obtained, as inputs to obtain corresponding outputs; and

comparing the outputs against the historical data to determine an extent to which the weighted learning models were successful.

6. The computer program product of claim 5, wherein the instructions, when executed, are configured to cause the at least one processor to:

adjust weights of w_t^h to obtain the updated w_t^h, w_t^h+1, including raising or lowering each weight in w_t^h+1, based on the compared outputs.

7. The computer program product of claim 1, wherein the instructions, when executed, are configured to cause the at least one processor to:

search the feature solution space including executing a meta-heuristic search of potential feature subsets for each feature vector.

8. The computer program product of claim 7, wherein the meta-heuristic search includes execution of a genetic algorithm in which potential feature subsets are represented as chromosomes, evaluated, and a subset of the chromosomes are selected and combined to obtain new potential feature sets.

9. The computer program product of claim 1, wherein the instructions, when executed, are configured to cause the at least one processor to:

search the parameter solution space including performing cross validation of potential parameter values using historical data against which the model vector will be trained and/or applied.

10. The computer program product of claim 1, wherein the instructions, when executed, are configured to cause the at least one processor to:

iteratively adjust the weights, search the feature solution space, and search the parameters solution space, until an optimized model vector (d_t^h+1, w_t^h+1, δ

_t^h+1) is selected after the reaching of a stop condition for the iterations.

11. The computer program product of claim 1, wherein the instructions, when executed, are configured to cause the at least one processor to:

search the feature solution space and the parameter solution space synchronously for current weight vector w_t^h.

12. A computer-implemented method for executing instructions stored on a non-transitory computer readable storage medium, the method comprising:

determining a model vector <

d_t^h, w_t^h, δ

_t^h>

, in which d_t^h represents a feature vector including t feature subsets of a feature set, w_t^h represents a weighted model vector including t weighted automated learning models, and δ

_t^h represents t parameter sets parameterizing the w_t^h weighted automated learning models;

adjusting weights of w_t^h to obtain an updated w_t^h, w_t^h+1, based on performance evaluations of the _t weighted automated learning models, and based on w_t^h;

searching a feature solution space to obtain t updated feature subsets of the feature set, to thereby obtain an updated d_t^h, d_t^h+1;

searching a parameter solution space to obtain t updated parameter sets, to thereby obtain an updated δ

_t^h, δ

_t^h+1;

determining an optimized model vector (d_t^h+1, w_t^h+1, δ

_t^h+1);

receiving a forecast request for a forecast related to the feature set; and

providing the forecast, using the optimized model vector (d_t^h+1, w_t^h+1, w_t^h+1).

13. The method of claim 12, wherein the adjusting the weights of w_t^h to obtain the updated w_t^h, w_t^h+1 includes

evaluating weights of w_t^h by applying each of the weighted learning models using historical data, from which the feature set was obtained, as inputs to obtain corresponding outputs;

and

comparing the outputs against the historical data to determine an extent to which the weighted learning models were successful.

14. The method of claim 12, further comprising:

iteratively adjusting the weights, searching the feature solution space, and searching the parameters solution space, until an optimized model vector (d_t^h+1, w_t^h+1, δ

_t^h+1) is selected after the reaching of a stop condition for the iterations.

15. The method of claim 12, wherein the feature solution space and the parameter solution space are searched synchronously for current weight vector w_t^h.

16. A system comprising:

at least one processor; and

instructions recorded on a non-transitory computer-readable medium, and executable by the at least one processor, the system including

a model vector generator configured to generate a model vector for executing automated machine learning with respect to historical data, the model vector generator configured to generate the model vector through an iterative selection of values for a feature vector, a weighted model vector, and a parameter vector that comprise the model vector, the model vector generator including

a weight selector configured to obtain the weighted model vector including adjusting each weight associated with a corresponding automated machine learning model of a plurality of automated machine learning models obtained from a model pool, wherein a magnitude of each weight is adjusted up or down as needed during each iteration to reflect a relative contribution of each weighted learning model to an accurate operation of the weighted model vector;

a feature selector configured to obtain the feature vector including executing a meta-heuristic search of a feature solution space of a feature set of features of the historic data, the feature vector including a plurality of feature subsets, each corresponding to a weighted learning model of the weighted model vector; and

a parameter selector configured to obtain the parameter vector including searching a parameter solution space of potential parameters for each learning model of the weighted model vector,

wherein each iteration of the iterative selection of values for the feature vector, the weighted model vector, and the parameter vector includes the adjusting of each weight, the searching of the feature solution space, and the searching of the parameter solution space; and

further wherein

the system is configured to receive a forecast request related to the feature set, and provide the forecast using the model vector.

17. The system of claim 16, wherein the weight selector includes a model evaluator configured to:

execute the adjusting of the weights next-iteration weights, including evaluating the weights by applying each of the weighted learning models using the historical data to obtain inputs to calculate corresponding outputs; and

compare the outputs against the historical data to determine an extent to which the weighted learning models were successful.

18. The system of claim 17, wherein the adjusting of the weights includes raising or lowering each weight based on the compared outputs.

19. The system of claim 16 wherein feature selector executes the meta-heuristic search including executing of a genetic algorithm in which potential feature subsets are represented as chromosomes, evaluated, and a subset of the chromosomes are selected and combined to obtain new potential feature sets.

20. The system of claim 16, wherein the parameter selector is configured to search the parameter solution space including performing cross validation of potential parameter values using historical data against which the model vector will be trained and/or applied.