SHARP DISCREPANCY LEARNING
First Claim
Patent Images
1. A method comprising:
- providing training data to a neural network that includes an output layer and one or more hidden layers, each of the hidden layers comprising multiple nodes and corresponding parameters;
calculating a gradient for the neural network by applying a sharp discrepancy output layer objective function to the output layer, wherein the sharp discrepancy output layer objective function is dependent on the training data and parameters;
training the neural network using the gradient to determine a probability that data received by the neural network has features similar to key features of one or more keywords or key phrases, wherein training the neural network using the gradient comprises using the gradient to update the parameters.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network. One of the methods includes training a neural network using sharp discrepancy learning by providing training data to the neural network, calculating a gradient using a sharp discrepancy output layer objective function to classify the neural network parameters for correct and incorrect network model states, and training the neural network using the gradient to determine a probability that data received by the neural network has features similar to key features of one or more keywords or key phrases.
-
Citations
20 Claims
-
1. A method comprising:
-
providing training data to a neural network that includes an output layer and one or more hidden layers, each of the hidden layers comprising multiple nodes and corresponding parameters; calculating a gradient for the neural network by applying a sharp discrepancy output layer objective function to the output layer, wherein the sharp discrepancy output layer objective function is dependent on the training data and parameters; training the neural network using the gradient to determine a probability that data received by the neural network has features similar to key features of one or more keywords or key phrases, wherein training the neural network using the gradient comprises using the gradient to update the parameters. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising; providing training data to a neural network that includes an output layer and one or more hidden layers, each of the hidden layers comprising multiple nodes and corresponding parameters; calculating a gradient for the neural network by applying a sharp discrepancy output layer objective function to the output layer, wherein the sharp discrepancy output layer objective function is dependent on the training data and parameters; training the neural network using the gradient to determine a probability that data received by the neural network has features similar to key features of one or more keywords or key phrases, wherein training the neural network using the gradient comprises using the gradient to update the parameters. - View Dependent Claims (19)
-
20. A computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
-
providing training data to a neural network that includes an output layer and one or more hidden layers, each of the hidden layers comprising multiple nodes and corresponding parameters; calculating a gradient for the neural network by applying a sharp discrepancy output layer objective function to the output layer, wherein the sharp discrepancy output layer objective function is dependent on the training data and parameters; training the neural network using the gradient to determine a probability that data received by the neural network has features similar to key features of one or more keywords or key phrases, wherein training the neural network using the gradient comprises using the gradient to update the parameters.
-
Specification