Augmented generalized deep learning with special vocabulary
First Claim
Patent Images
1. A method for customizing a neural network trained on a general dataset to a custom dataset, the method comprising:
- providing a trained speech recognition neural network, the trained speech recognition neural network including a plurality of layers each having a plurality of nodes, the trained speech recognition neural network including an output layer with nodes corresponding to words of a vocabulary, the nodes of the output layer outputting values, wherein the values output by the nodes in the output layer correspond to a probability of the corresponding word in the vocabulary being a correct transcription of an input;
for a plurality of words in the vocabulary, determining a frequency of occurrence of the word in a general training set and a frequency of occurrence of the word in a custom dataset;
during inference using the trained speech recognition neural network, for each word in the plurality of words, adjusting the value output by the output node for the word based on the frequency of occurrence of the word in the custom dataset and the frequency of occurrence of the word in the general training set to obtain a custom model probability; and
generating a transcription of a spoken input based on the custom model probability.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and methods are disclosed for customizing a neural network for a custom dataset, when the neural network has been trained on data from a general dataset. The neural network may comprise an output layer including one or more nodes corresponding to candidate outputs. The values of the nodes in the output layer may correspond to a probability that the candidate output is the correct output for an input. The values of the nodes in the output layer may be adjusted for higher performance when the neural network is used to process data from a custom dataset.
-
Citations
20 Claims
-
1. A method for customizing a neural network trained on a general dataset to a custom dataset, the method comprising:
-
providing a trained speech recognition neural network, the trained speech recognition neural network including a plurality of layers each having a plurality of nodes, the trained speech recognition neural network including an output layer with nodes corresponding to words of a vocabulary, the nodes of the output layer outputting values, wherein the values output by the nodes in the output layer correspond to a probability of the corresponding word in the vocabulary being a correct transcription of an input; for a plurality of words in the vocabulary, determining a frequency of occurrence of the word in a general training set and a frequency of occurrence of the word in a custom dataset; during inference using the trained speech recognition neural network, for each word in the plurality of words, adjusting the value output by the output node for the word based on the frequency of occurrence of the word in the custom dataset and the frequency of occurrence of the word in the general training set to obtain a custom model probability; and generating a transcription of a spoken input based on the custom model probability. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A non-transitory computer-readable medium comprising instructions for:
-
providing a trained speech recognition neural network, the trained speech recognition neural network including a plurality of layers each having a plurality of nodes, the trained speech recognition neural network including an output layer with nodes corresponding to words of a vocabulary, the nodes of the output layer outputting values, wherein the values output by the nodes in the output layer correspond to a probability of the corresponding word in the vocabulary being a correct transcription of an input; for a plurality of words in the vocabulary, determining a frequency of occurrence of the word in a general training set and a frequency of occurrence of the word in a custom dataset; during inference using the trained speech recognition neural network, for each word in the plurality of words, adjusting the value output by the output node for the word based on the frequency of occurrence of the word in the custom dataset and the frequency of occurrence of the word in the general training set to obtain a custom model probability; and generating a transcription of a spoken input based on the custom model probability. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable medium comprising instructions for:
-
providing a trained speech recognition neural network, the trained speech recognition neural network including a plurality of layers each having a plurality of nodes, the trained speech recognition neural network including an output layer with nodes corresponding to words of a vocabulary, the nodes of the output layer outputting values, wherein the values output by the nodes in the output layer correspond to a probability of the corresponding word in the vocabulary being a correct transcription of an input; during inference using the trained speech recognition neural network, adjusting the values output by a plurality of nodes in the output layer based on a frequency of occurrence of the corresponding word in a general training set and a frequency of occurrence of the corresponding word in a custom dataset to obtain a custom model probability; and generating a transcription of a spoken input based on the custom model probability. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification