Classification system training

• US 10,049,302 B1
• Filed: 03/05/2018
• Issued: 08/14/2018
• Est. Priority Date: 07/17/2017
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Liu, Jun, Xu, Yan, Griffin, Joshua David, Chari, Manoj Keshavmurthi
• Primary Examiner(s): Smith, Paulinho E
• Application Number:
  US15/911,882
• Time in Patent Office: 162 Days
• Field of Search: None
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2193 based on specific statistical tests
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 5/045 Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
• CPC Class Codes: G06N 7/00 Computing arrangements based on specific mathematical models

Abstract

A computing device trains models for streaming classification. A baseline penalty value is computed that is inversely proportional to a square of a maximum explanatory variable value. A set of penalty values is computed based on the baseline penalty value. For each penalty value of the set of penalty values, a classification type model is trained using the respective penalty value and the observation vectors to compute parameters that define a trained model, the classification type model is validated using the respective penalty value and the observation vectors to compute a validation criterion value that quantifies a validation error, and the validation criterion value, the respective penalty value, and the parameters that define a trained model are stored to the computer-readable medium. The classification type model is trained to predict the response variable value of each observation vector based on the respective explanatory variable value of each observation vector.

Claims

1. A non-transitory computer-readable medium having stored thereon computer-readable instructions that when executed by a computing device cause the computing device to:

compute a baseline penalty value using a plurality of observation vectors, wherein each observation vector of the plurality of observation vectors includes an explanatory variable value and a response variable value, wherein the baseline penalty value is inversely proportional to a square of a maximum explanatory variable value;

compute a set of penalty values based on the computed baseline penalty value;

for each penalty value of the set of penalty values,

train a classification type model using the respective penalty value and the plurality of observation vectors to compute parameters that define a trained model, wherein the classification type model is trained to predict the response variable value of each observation vector based on the respective explanatory variable value of each observation vector;

validate the trained classification type model using the respective penalty value and the plurality of observation vectors to compute a validation criterion value for the trained classification type model that quantifies a validation error; and

store the computed validation criterion value, the respective penalty value, and the computed parameters that define a trained model to the computer-readable medium;

determine a best classification model based on the stored, computed validation criterion value of each trained classification type model; and

output the respective penalty value and the computed parameters associated with the determined best classification model for predicting a new response variable value from a new observation vector.

2. The non-transitory computer-readable medium of claim 1, wherein the baseline penalty value is computed using

3. The non-transitory computer-readable medium of claim 1, wherein the classification type model is selected from the group consisting of a support vector machine model algorithm, and a logistic regression model algorithm.

4. The non-transitory computer-readable medium of claim 1, wherein the validation criterion value is one or more of an accumulative loss function value, an accumulated misclassification error value, and an F-score value.

5. The non-transitory computer-readable medium of claim 1, wherein each observation vector of the plurality of observation vectors includes a plurality of explanatory variable values, wherein the explanatory variable value is one of the plurality of explanatory variable values.

6. The non-transitory computer-readable medium of claim 1, wherein the set of penalty values is computed using C_B*[ρ

<sup>−

t</sup>, ρ

<sup>−

t+t</sup>, . . . , ρ

⁰, . . . , ρ

<sup>t−

1</sup>, ρ

^t], where t is a search size parameter, and ρ

is a log base parameter.

7. The non-transitory computer-readable medium of claim 1, wherein the computer-readable instructions further cause the computing device to:

read a second explanatory variable value from an input dataset;

compute the new response variable value using the output respective penalty value, the output, computed parameters, and the read second explanatory variable value; and

output the computed new response variable value.

8. The non-transitory computer-readable medium of claim 1, wherein, after outputting the respective penalty value and the computed parameters, the computer-readable instructions further cause the computing device to:

for each penalty value of the set of penalty values,

train the classification type model using the respective penalty value and a second plurality of observation vectors to compute second parameters, wherein each observation vector of the second plurality of observation vectors includes a second explanatory variable value and a second response variable value, wherein the classification type model is trained to predict the second response variable value of each observation vector based on the respective second explanatory variable value of each observation vector;

validate the trained classification type model using the respective penalty value and the second plurality of observation vectors to compute a second validation criterion value; and

store the computed second validation criterion value, the respective penalty value, and the computed second parameters to the computer-readable medium;

determine a new best classification model based on the stored, computed second validation criterion value of each classification type model trained using the second plurality of observation vectors; and

output the respective penalty value and the computed second parameters associated with the determined, new best classification model for predicting a second new response variable value from a second new observation vector.

9. The non-transitory computer-readable medium of claim 8, wherein the computer-readable instructions further cause the computing device to:

read a third explanatory variable value from an input dataset;

compute the new response variable value using the output respective penalty value, the output, computed second parameters, and the read third explanatory variable value; and

output the computed new response variable value.

10. The non-transitory computer-readable medium of claim 8, wherein each observation vector of the plurality of observation vectors is received in an event stream.

11. The non-transitory computer-readable medium of claim 10, wherein each observation vector of the second plurality of observation vectors is received in the event stream.

12. The non-transitory computer-readable medium of claim 11, wherein the computer-readable instructions further cause the computing device to:

receive a third explanatory variable value from an event publishing device;

compute the new response variable value using the output respective penalty value, the output, computed second parameters, and the received third explanatory variable value; and

send the computed new response variable value to an event subscribing device.

13. The non-transitory computer-readable medium of claim 1, wherein each observation vector of the plurality of observation vectors is received in an event stream.

14. The non-transitory computer-readable medium of claim 1, wherein the output, computed parameters are Lagrange constants computed by solving for an optimal value of an objective function defined based on the classification type model.

15. A computing device comprising:

a processor; and

a non-transitory computer-readable medium operably coupled to the processor, the computer-readable medium having computer-readable instructions stored thereon that, when executed by the processor, cause the computing device to

compute a baseline penalty value using a plurality of observation vectors, wherein each observation vector of the plurality of observation vectors includes an explanatory variable value and a response variable value, wherein the baseline penalty value is inversely proportional to a square of a maximum explanatory variable value;

compute a set of penalty values based on the computed baseline penalty value;

for each penalty value of the set of penalty values,

train a classification type model using the respective penalty value and the plurality of observation vectors to compute parameters that define a trained model, wherein the classification type model is trained to predict the response variable value of each observation vector based on the respective explanatory variable value of each observation vector;

validate the trained classification type model using the respective penalty value and the plurality of observation vectors to compute a validation criterion value for the trained classification type model that quantifies a validation error; and

store the computed validation criterion value, the respective penalty value, and the computed parameters that define a trained model to the computer-readable medium;

determine a best classification model based on the stored, computed validation criterion value of each trained classification type model; and

output the respective penalty value and the computed parameters associated with the determined best classification model for predicting a new response variable value from a new observation vector.

16. The computing device of claim 15, wherein the baseline penalty value is computed using

17. A method of providing training of classification models, the method comprising:

computing, by a computing device, a baseline penalty value using a plurality of observation vectors, wherein each observation vector of the plurality of observation vectors includes an explanatory variable value and a response variable value, wherein the baseline penalty value is inversely proportional to a square of a maximum explanatory variable value;

computing, by the computing device, a set of penalty values based on the computed baseline penalty value;

for each penalty value of the set of penalty values,

training, by the computing device, a classification type model using the respective penalty value and the plurality of observation vectors to compute parameters that define a trained model, wherein the classification type model is trained to predict the response variable value of each observation vector based on the respective explanatory variable value of each observation vector;

validating, by the computing device, the trained classification type model using the respective penalty value and the plurality of observation vectors to compute a validation criterion value for the trained classification type model that quantifies a validation error; and

storing, by the computing device, the computed validation criterion value, the respective penalty value, and the computed parameters that define a trained model to the computer-readable medium;

determining, by the computing device, a best classification model based on the stored, computed validation criterion value of each trained classification type model; and

outputting, by the computing device, the respective penalty value and the computed parameters associated with the determined best classification model for predicting a new response variable value from a new observation vector.

18. The method of claim 17, wherein the baseline penalty value is computed using

19. The method of claim 17, wherein the classification type model is selected from the group consisting of a support vector machine model algorithm, and a logistic regression model algorithm.

20. The method of claim 17, wherein the validation criterion value is one or more of an accumulative loss function value, an accumulated misclassification error value, and an F-score value.

21. The method of claim 17, wherein each observation vector of the plurality of observation vectors includes a plurality of explanatory variable values, wherein the explanatory variable value is one of the plurality of explanatory variable values.

22. The method of claim 17, wherein the set of penalty values is computed using C_B*[ρ

<sup>−

t</sup>, ρ

<sup>−

t+1</sup>, . . . , ρ

⁰, . . . , ρ

<sup>t−

1</sup>, ρ

^t], where t is a search size parameter, and ρ

is a log base parameter.

23. The method of claim 17, further comprising:

reading, by the computing device, a second explanatory variable value from an input dataset;

computing, by the computing device, the new response variable value using the output respective penalty value, the output, computed parameters, and the read second explanatory variable value; and

outputting, by the computing device, the computed new response variable value.

24. The method of claim 17, further comprising after outputting the respective penalty value and the computed parameters:

for each penalty value of the set of penalty values,

training, by the computing device, the classification type model using the respective penalty value and a second plurality of observation vectors to compute second parameters, wherein each observation vector of the second plurality of observation vectors includes a second explanatory variable value and a second response variable value, wherein the classification type model is trained to predict the second response variable value of each observation vector based on the respective second explanatory variable value of each observation vector;

validating, by the computing device, the trained classification type model using the respective penalty value and the second plurality of observation vectors to compute a second validation criterion value; and

storing, by the computing device, the computed second validation criterion value, the respective penalty value, and the computed second parameters to the computer-readable medium;

determining, by the computing device, a new best classification model based on the stored, computed second validation criterion value of each classification type model trained using the second plurality of observation vectors; and

outputting, by the computing device, the respective penalty value and the computed second parameters associated with the determined, new best classification model for predicting a second new response variable value from a second new observation vector.

25. The method of claim 24, further comprising:

reading, by the computing device, a third explanatory variable value from an input dataset;

computing, by the computing device, the new response variable value using the output respective penalty value, the output, computed second parameters, and the read third explanatory variable value; and

outputting, by the computing device, the computed new response variable value.

26. The method of claim 24, wherein each observation vector of the plurality of observation vectors is received in an event stream.

27. The method of claim 26, wherein each observation vector of the second plurality of observation vectors is received in the event stream.

28. The method of claim 27, further comprising:

receiving, by the computing device, a third explanatory variable value from an event publishing device;

computing, by the computing device, the new response variable value using the output respective penalty value, the output, computed second parameters, and the received third explanatory variable value; and

sending, by the computing device, the computed new response variable value to an event subscribing device.

29. The method of claim 17, wherein each observation vector of the plurality of observation vectors is received in an event stream.

30. The method of claim 17, wherein the output, computed parameters are Lagrange constants computed by solving for an optimal value of an objective function defined based on the classification type model.