NEURAL NETWORK TRAINING APPARATUS AND METHOD, AND SPEECH RECOGNITION APPARATUS AND METHOD

• US 20170133006A1
• Filed: 11/04/2016
• Published: 05/11/2017
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: CHOI, Hee Youl, LEE, Ho Shik
• Granted Patent: US 10,529,317 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A neural network training apparatus includes a primary trainer configured to perform a primary training of a neural network model based on clean training data and target data corresponding to the clean training data; and a secondary trainer configured to perform a secondary training of the neural network model on which the primary training has been performed based on noisy training data and an output probability distribution of an output class for the clean training data calculated during the primary training of the neural network model.

Claims

1. A neural network training apparatus comprising:

a primary trainer configured to perform a primary training of a neural network model based on clean training data and target data corresponding to the clean training data; and

a secondary trainer configured to perform a secondary training of the neural network model on which the primary training has been performed based on noisy training data and a probability distribution of an output class for the clean training data calculated during the primary training of the neural network model.

2. The neural network training apparatus of claim 1, wherein the secondary trainer is further configured to perform the secondary training of the neural network model on which the primary training has been performed using the noisy training data as an input and the probability distribution of the output class for the clean training data calculated during the primary training of the neural network model as a target.

3. The neural network training apparatus of claim 1, wherein the noisy training data comprises data in which the clean training data is distorted or the clean training data and training noise data are mixed.

4. The neural network training apparatus of claim 1, wherein the neural network model is a neural network-based acoustic model.

5. The neural network training apparatus of claim 1, wherein the primary trainer is further configured to perform the primary training using a first objective function that performs training of the neural network model to obtain the target data from the clean training data.

6. The neural network training apparatus of claim 1, wherein the secondary trainer is further configured to perform the secondary training using a second objective function that is a combination of the probability distribution of the output class for the clean training data calculated during the primary training of the neural network model and an activation function of an output layer of the neural network model.

7. The neural network training apparatus of claim 1, wherein the secondary trainer is further configured to perform the secondary training using a second objective function that is a weighted sum of an objective function that performs training of the neural network model to obtain the target data from the clean training data and an objective function that is a combination of the probability distribution of the output class for the clean training data calculated during the primary training of the neural network model and an activation function of an output layer of the neural network model.

8. A neural network training method comprising:

performing a primary training of a neural network model based on clean training data and target data corresponding to the clean training data; and

performing a secondary training of the neural network model on which the primary training has been performed based on noisy training data and a probability distribution of an output class for the clean training data calculated during the primary training of the neural network model.

9. The neural network training method of claim 8, wherein the secondary training comprises performing the secondary training of the neural network model on which the primary training has been performed using the noisy training data as an input and the probability distribution of the output class for the clean training data calculated during the primary training of the neural network model as a target.

10. The neural network training method of claim 8, wherein the noisy training data comprises data in which the clean training data is distorted or the clean training data and training noise data are mixed.

11. The neural network training method of claim 8, wherein the neural network model is a neural network-based acoustic model.

12. The neural network training method of claim 8, wherein the performing of the primary training comprises performing the primary training using a first objective function that performs training of the neural network model to obtain the target data from the clean training data.

13. The neural network training method of claim 8, wherein the performing of the secondary training comprises performing the secondary training using a second objective function that is a combination of the probability distribution of the output class for the clean training data calculated during the primary training of the neural network model and an activation function of an output layer of the neural network model.

14. The neural network training method of claim 8, wherein the performing of the secondary training comprises performing the secondary training using a second objective function that is a weighted sum of an objective function that performs training of the neural network model to obtain the target data from the clean training data and an objective function that is a combination of the probability distribution of the output class for the clean training data calculated during the primary training of the neural network model and an activation function of an output layer of the neural network model.

15. A computer-readable storage medium storing instructions, that when executed by a processor, cause the processor to perform the method of claim 8.

16. A speech recognition apparatus comprising:

a feature extractor configured to extract a feature of noisy speech data; and

a phoneme probability calculator configured to calculate a probability of a phoneme corresponding to the extracted feature using an acoustic model;

wherein the acoustic model is an acoustic model on which a primary training has been performed based on speech training data and a phoneme sequence corresponding to the speech training data, and a secondary training has been performed based on noisy speech training data and a probability distribution of an output class for the speech training data calculated during the primary training of the acoustic model.

17. The speech recognition apparatus of claim 16, wherein the primary training of the acoustic model is performed using the speech training data as an input and the phoneme sequence corresponding to the speech training data as a target.

18. The speech recognition apparatus of claim 16, wherein the secondary training of the acoustic model is performed using the noisy speech training data as an input and the probability distribution of the output class for the speech training data calculated during the primary training of the acoustic model as a target.

19. The speech recognition apparatus of claim 16, wherein the noisy speech training data comprises data in which the speech training data is distorted or the speech training data and training noise data are mixed.

20. The speech recognition apparatus of claim 16, wherein the acoustic model is a neural network-based acoustic model.

21. The speech recognition apparatus of claim 16, wherein the primary training of the acoustic model is performed using a first objective function that performs training of the acoustic model to obtain a phoneme from the speech training data.

22. The speech recognition apparatus of claim 16, wherein the secondary training of the acoustic model is performed using a second objective function that is a combination of the probability distribution of the output class for the speech training data calculated during the primary training of the acoustic model and an activation function of an output layer of the acoustic model.

23. The speech recognition apparatus of claim 16, wherein the secondary training of the acoustic model is performed using a second objective function that is a weighted sum of an objective function that performs training of the acoustic model to obtain a phoneme from the speech training data and an objective function that is a combination of the probability distribution of the output class for the speech training data calculated during the primary training of the acoustic model and an activation function of an output layer of the acoustic model.

24. A neural network training apparatus comprising:

a primary trainer configured to perform a primary training of a neural network model based on clean training data and hard target data; and

a secondary trainer configured to perform a secondary training of the neural network model on which the primary training has been performed based on noisy training data and soft target data obtained during the primary training of the neural network model.

25. The neural network training apparatus of claim 24, wherein the noisy training data is obtained by distorting the clean training data or mixing the clean training data with noise.

26. The neural network training apparatus of claim 24, wherein the soft target data is a probability distribution of an output class for the clean training data calculated during the primary training of the neural network model.

27. The neural network training apparatus of claim 24, wherein the secondary trainer is further configured to perform the secondary training based on the noisy training data, the soft target data, and an activation function of an output layer of the neural network model.