Low latency real-time vocal tract length normalization

• US 8,909,527 B2
• Filed: 06/24/2009
• Issued: 12/09/2014
• Est. Priority Date: 01/12/2005
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Ljolje, Andrej, Saraclar, Murat, Goffin, Vincent
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US12/490,634
• Publication Number:
  US 20090259465A1
• Time in Patent Office: 1,994 Days
• Field of Search: 704/200, 704211-218, 704/220, 704/231, 704/236, 704241-245, 704246-250, 704270-271, 704E17001-E17016, 704E11001-E11007, 379 8801- 8828, 379156-166, 379/207.01
• US Class Current: 704/241
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/12 using dynamic programming techniques, e.g. dynamic time warping [DTW]
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 17/08 Use of distortion metrics or a particular distance between probe pattern and reference templates

Abstract

A method and system for training an automatic speech recognition system are provided. The method includes separating training data into speaker specific segments, and for each speaker specific segment, performing the following acts: generating spectral data, selecting a first warping factor and warping the spectral data, and comparing the warped spectral data with a speech model. The method also includes iteratively performing the steps of selecting another warping factor and generating another warped spectral data, comparing the other warped spectral data with the speech model, and if the other warping factor produces a closer match to the speech model, saving the other warping factor as the best warping factor for the speaker specific segment. The system includes modules configured to control a processor in the system to perform the steps of the method.

Claims

1. A method comprising:

separating training data into speaker specific segments;

performing, for a speaker specific segment of the speaker specific segments;

generating spectral data representative of the speaker specific segment, the spectral data comprising a plurality of warping factors;

selecting a first warping factor as a best warping factor from the plurality of warping factors based on a determination made during speech recognition of the speaker specific segment, and generating a warped spectral data representation of the spectral data using the first warping factor;

comparing the warped spectral data representation to a vocal tract length normalized acoustic model;

iteratively carrying out, until a comparison indicates a warping factor difference below 0.02, the acts of;

selecting an other warping factor and generating an other warped spectral data representation;

comparing the other warped spectral data representation to the vocal tract length normalized acoustic model, to yield the comparison; and

when the other warping factor produces a closer match to the vocal tract length normalized acoustic model, saving the other warping factor as a best warping factor for the speaker specific segment;

training a new acoustic model using a warped spectral data representation of all the training data that is generated using the best warping factor for each of the speaker specific segments;

selecting the new acoustic model as the vocal tract length normalized acoustic model; and

repeating the steps of performing and selecting until the best warping factor for each of the speaker specific segments is stable.

2. The method of claim 1, wherein the plurality of warping factors have a range from about 0.8 to about 1.2.

3. The method of claim 2, wherein the range comprises increments of about 0.02 between each warping factor.

4. The method of claim 1, wherein the end condition comprises a predetermined amount of total speech having been used to select the best warping factor.

5. The method of claim 1, wherein the end condition comprises a difference between a latest warping factor and a preceding warping factor being smaller than a predetermined amount.

6. The method of claim 1, wherein the spectral data is a short-term magnitude spectrum of the speaker specific segment.

7. The method of claim 1, wherein the spectral data comprises a spectral axis modified by the warping factor.

8. The method of claim 7, wherein the new acoustic model comprises a vocal tract length normalized acoustic model based on the spectral axis modified by the warping factor.

9. The computer-implemented method of claim 1, wherein the vocal tract length normalized acoustic model is initially a generic acoustic model.

10. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

separating training data into speaker specific segments;

performing, for a speaker specific segment of the speaker specific segments;

generating spectral data representative of the speaker specific segment, the spectral data comprising a plurality of warping factors;

selecting a first warping factor as a best warping factor from the plurality of warping factors based on a determination made during speech recognition of the speaker specific segment, and generating a warped spectral data representation of the spectral data using the first warping factor;

comparing the warped spectral data representation to a vocal tract length normalized acoustic model;

iteratively carrying out, until a comparison indicates a warping factor difference below 0.02, the acts of;

selecting an other warping factor and generating an other warped spectral data representation;

comparing the other warped spectral data representation to the vocal tract length normalized acoustic model, to yield the comparison; and

when the other warping factor produces a closer match to the vocal tract length normalized acoustic model, saving the other warping factor as a best warping factor for the speaker specific segment; and

training a new acoustic model using a warped spectral data representation of all the training data that is generated using the best warping factor for each of the speaker specific segments;

selecting the new acoustic model as the vocal tract length normalized acoustic model; and

repeating the steps of performing and selecting until the best warping factor for each of the speaker specific segments is stable.

11. The system of claim 10, wherein each the plurality of warping factors have a range from about 0.8 to about 1.2.

12. The system of claim 11, wherein the range comprises increments of about 0.02 between each warping factor.

13. The system of claim 10, the computer-readable storage medium having additional instructions stored which result in the operations further comprising rescoring the lattices based on the selected ones of the plurality of warped spectral axes.

14. The system of claim 10, the computer-readable storage medium having additional instructions stored which result in the operations further comprising determining when the selected ones of the plurality of warped spectral axes are stable.

15. The system of claim 10, the computer-readable storage medium having additional instructions stored which result in the operations further comprising iteratively performing:

selecting an other warping factor and generating an other warped spectral data representation based on the respective warped spectral axes;

comparing the warped spectral data representation to the vocal tract length normalized acoustic model for a given iteration; and

when the other warping factor produces a closer match to the vocal tract length normalized acoustic model, saving the other warping factor as a best warping factor for the respective speaker specific segment.

16. The system of claim 15, wherein the end condition comprises a predetermined amount of total speech having been used to select the best warping factor.

17. The system of claim 15, wherein the end condition comprises a difference between a latest warping factor and a preceding warping factor being smaller than a predetermined amount.

18. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

separating training data into speaker specific segments;

performing, for a speaker specific segment of the speaker specific segments;

generating spectral data representative of the speaker specific segment, the spectral data comprising a plurality of warping factors;

selecting a first warping factor as a best warping factor from the plurality of warping factors based on a determination made during speech recognition of the speaker specific segment, and generating a warped spectral data representation of the spectral data using the first warping factor;

comparing the warped spectral data representation to a vocal tract length normalized acoustic model;

iteratively carrying out, until a comparison indicates a warping factor difference below 0.02, the acts of;

selecting an other warping factor and generating an other warped spectral data representation;

comparing the other warped spectral data representation to the vocal tract length normalized acoustic model, to yield the comparison; and

when the other warping factor produces a closer match to the vocal tract length normalized acoustic model, saving the other warping factor as a best warping factor for the speaker specific segment; and

training a new acoustic model using a warped spectral data representation of all the training data that is generated using the best warping factor for each of the speaker specific segments;

selecting the new acoustic model as the vocal tract length normalized acoustic model; and

repeating the steps of performing and selecting until the best warping factor for each of the speaker specific segments is stable.