Template regularization for generalization of learning systems
First Claim
1. A computer-implemented method of training a machine learning model on labeled examples, wherein the machine learning model is configured to receive an example having a plurality of features and to generate a predicted output for the received example, the method comprising:
- obtaining data defining a plurality of templates, wherein each template corresponds to one or more categories of features;
assigning a respective regularization penalty to each of the plurality of templates; and
training the machine learning model on the labeled examples, comprising, for each labeled example and for each of the plurality of templates;
determining, using the machine learning model, a respective weight for the template based on the features of the labeled example that belong to the one or more categories that correspond to the template, andmodifying the respective weight for the template by applying the respective regularization penalty for the template to the respective weight for the template determined by the machine learning model,wherein, during the training, a template having a lower regularization penalty is emphasized over a template having a higher regularization penalty.
2 Assignments
0 Petitions
Accused Products
Abstract
Systems and techniques are disclosed for training a machine learning model based on one or more regularization penalties associated with one or more features. A template having a lower regularization penalty may be given preference over a template having a higher regularization penalty. A regularization penalty may be determined based on domain knowledge. A restrictive regularization penalty may be assigned to a template based on determining that a template occurrence is below a stability threshold and may be modified if the template occurrence meets or exceeds the stability threshold.
-
Citations
20 Claims
-
1. A computer-implemented method of training a machine learning model on labeled examples, wherein the machine learning model is configured to receive an example having a plurality of features and to generate a predicted output for the received example, the method comprising:
-
obtaining data defining a plurality of templates, wherein each template corresponds to one or more categories of features; assigning a respective regularization penalty to each of the plurality of templates; and training the machine learning model on the labeled examples, comprising, for each labeled example and for each of the plurality of templates; determining, using the machine learning model, a respective weight for the template based on the features of the labeled example that belong to the one or more categories that correspond to the template, and modifying the respective weight for the template by applying the respective regularization penalty for the template to the respective weight for the template determined by the machine learning model, wherein, during the training, a template having a lower regularization penalty is emphasized over a template having a higher regularization penalty. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A system comprising:
-
one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising; obtaining data defining a plurality of templates, wherein each template corresponds to one or more categories of features; assigning a respective regularization penalty to each of the plurality of templates; and training the machine learning model on the labeled examples, comprising, for each labeled example and for each of the plurality of templates; determining, using the machine learning model, a respective weight for the template based on the features of the labeled example that belong to the one or more categories that correspond to the template, and modifying the respective weight for the template by applying the respective regularization penalty for the template to the respective weight for the template determined by the machine learning model, wherein, during the training, a template having a lower regularization penalty is emphasized over a template having a higher regularization penalty. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
-
17. A non-transitory computer readable medium encoded with a computer program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
-
obtaining data defining a plurality of templates, wherein each template corresponds to one or more categories of features; assigning a respective regularization penalty to each of the plurality of templates; and training the machine learning model on the labeled examples, comprising, for each labeled example and for each of the plurality of templates; determining, using the machine learning model, a respective weight for the template based on the features of the labeled example that belong to the one or more categories that correspond to the template, and modifying the respective weight for the template by applying the respective regularization penalty for the template to the respective weight for the template determined by the machine learning model, wherein, during the training, a template having a lower regularization penalty is emphasized over a template having a higher regularization penalty. - View Dependent Claims (18, 19, 20)
-
Specification