Conservative training method for adapting a neural network of an automatic speech recognition device

• US 8,126,710 B2
• Filed: 06/01/2005
• Issued: 02/28/2012
• Est. Priority Date: 06/01/2005
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Loquendo SpA (Microsoft Corporation)
• Inventors: Gemello, Roberto, Mana, Franco
• Primary Examiner(s): Smits, Talivaldis I
• Assistant Examiner(s): Pullias, Jesse
• Application Number:
  US11/921,303
• Publication Number:
  US 20090216528A1
• Time in Patent Office: 2,463 Days
• Field of Search: 704/232, 704/255, 704/256.5, 704/E15.008, 704/9, 704/17, 704/32, 704/36, 704/202
• US Class Current: 704/232
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A method of adapting a neural network of an automatic speech recognition device, includes the steps of: providing a neural network including an input stage, an intermediate stage and an output stage, the output stage outputting phoneme probabilities; providing a linear stage in the neural network; and training the linear stage by means of an adaptation set; wherein the step of providing the linear stage includes the step of providing the linear stage after the intermediate stage.

Claims

1. A method of adapting a neural network of an automatic speech recognition device, comprising the steps of:

providing a neural network comprising an input stage for storing at least one voice signal sample, an intermediate stage and an output stage, said output stage outputting phoneme probabilities;

providing a linear stage in said neural network; and

training said linear stage by means of an adaptation set,

wherein the step of providing said linear stage comprises the step of providing said linear stage after said intermediate stage;

wherein the step of training said linear stage comprises training said linear stage so that the phoneme probability of a phoneme belonging to an absent class is equal to the phoneme probability of said phoneme calculated by said neural network before the step of providing a linear stage; and

wherein the step of training said linear stage further comprises training said linear stage so that the phoneme probability of the phoneme corresponding to a voice signal sample of said adaptation set is calculated by subtracting the phoneme probabilities of all the phonemes belonging to said absent class from 1.

2. The method according to claim 1, wherein the step of training said linear stage comprises training said linear stage so that the phoneme probability of the remaining phonemes is set equal to zero.

3. The method according to claim 1, wherein the step of providing said linear stage comprises the step of providing said linear stage between said intermediate stage and said output stage.

4. The method according to claim 1, wherein the step of providing said neural network comprises the step of providing a neural network comprising two intermediate stages and wherein the step of providing said linear stage comprises providing said linear stage between said two intermediate stages.

5. The method according to claim 1, wherein the step of training said linear stage comprises the step of training said linear stage by means of an error back-propagation algorithm.

6. The method according to claim 1, further comprising a step of providing an equivalent stage obtained by combining said linear stage and either the following intermediate stage or the output stage.

14. A non-transitory computer readable medium having a program recorded thereon, said computer readable medium comprising a computer program code portion for performing all the steps of claim 1, when said computer program code portion is executed by a computer.

7. A neural network comprising:

a computer;

an input stage for storing at least one voice signal sample;

an intermediate stage;

an output stage; and

a linear stage which is to be trained by means of an adaptation set,

wherein said output stage is configured to output phoneme probabilities;

wherein said linear stage is provided after said intermediate stage and is configured to be trained so that the phoneme probability of a phoneme belonging to an absent class is equal to the phoneme probability of said phoneme calculated by said neural network before the step of providing a linear stage; and

wherein said linear stage is configured to be trained so that the phoneme probability of the phoneme corresponding to a voice signal sample of said adaptation set is calculated by subtracting the phoneme probabilities of all the phonemes belonging to said absent class from 1.

8. The neural network according to claim 7, wherein said linear stage is configured to be trained so that the phoneme probability of the remaining phonemes is set equal to zero.

9. The neural network according to claim 7, wherein said linear stage is provided between said intermediate stage and said output stage.

10. The neural network according to claim 7, wherein the neural network comprises two intermediate stages and said linear stage is provided between said two intermediate stages.

11. The neural network according to claim 7, wherein said linear stage is configured to be trained by means of an error back-propagation algorithm.

12. The neural network according to claim 7, wherein the neural network comprises an equivalent stage obtained by combining said linear stage and either the following intermediate stage or the output stage.

13. An automatic speech recognition device comprising a pattern matching block comprising a neural network according to claim 7.

15. A method of adapting a multi-layer neural network of an automatic speech recognition device, comprising the steps of:

providing a neural network comprising an input stage for storing at least one voice signal sample, an intermediate stage having input connections associated to a first weight matrix and an output stage having input connections associated to a second weight matrix, said output stage outputting phoneme probabilities;

providing a linear stage in said neural network after said intermediate stage, said linear stage having a same number of nodes as said intermediate stage; and

training said linear stage by means of an adaptation set, said first weight matrix and said second weight matrix being kept fixed during said training,

wherein the step of training said linear stage comprises training said linear stage so that the phoneme probability of a phoneme belonging to an absent class is equal to the phoneme probability of said phoneme calculated by said neural network before the step of providing a linear stage; and

wherein the step of training said linear stage comprises training said linear stage so that the phoneme probability of the phoneme corresponding to a voice signal sample of said adaptation set is calculated by subtracting the phoneme probabilities of all the phonemes belonging to said absent class from 1.

16. A multi-layer neural network computation module, comprising:

a computer;

an input stage for storing at least one voice signal sample;

an intermediate stage having input connections associated to a first weight matrix;

an output stage having input connections associated to a second weight matrix; and

a linear stage configured to be trained by means of an adaptation set,

wherein said first weight matrix and said second weight matrix are kept fixed while said linear stage is trained;

wherein said output stage is configured to output phoneme probabilities;

wherein said linear stage is provided after said intermediate stage, said linear stage having a same number of nodes as said intermediate stage;

wherein said linear stage is provided after said intermediate stage and is configured to be trained so that the phoneme probability of a phoneme belonging to an absent class is equal to the phoneme probability of said phoneme calculated by said neural network before the step of providing a linear stage; and

wherein said linear stage is configured to be trained so that the phoneme probability of the phoneme corresponding to a voice signal sample of said adaptation set is calculated by subtracting the phoneme probabilities of all the phonemes belonging to said absent class from 1.