Automatically retraining a speech recognition system

• US 6,789,062 B1
• Filed: 02/25/2000
• Issued: 09/07/2004
• Est. Priority Date: 02/25/2000
• Status: Expired due to Term

• Current Assignee: SpeechWorks International, Inc. (Microsoft Corporation)
• Original Assignee: SpeechWorks International, Inc. (Microsoft Corporation)
• Inventors: Phillips, Michael S., Barnard, Etienne, Govindarajan, Krishna K., Fanty, Mark
• Primary Examiner(s): McFadden, Susan
• Application Number:
  US09/512,785
• Time in Patent Office: 1,656 Days
• Field of Search: 704/231, 704/236, 704/247, 704/251, 704/250, 704/252, 704/255, 704/240, 704/256, 704/257, 704/270, 704/243
• US Class Current: 704/250
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

A telephone-based interactive speech recognition system is retrained using variable weighting and incremental retraining. Variable weighting involves changing the relative influence of particular measurement data to be reflected in a statistical model. Statistical model data is determined based upon an initial set of measurement data determined from an initial set of speech utterances. When new statistical model data is to be generated to reflect new measurement data determined from new speech utterances, a weighting factor is applied to the new measurement data to generate weighted new measurement data. The new statistical model data is then determined based upon the initial set of measurement data and the weighted new measurement data. Incremental retraining involves generating new statistical model data using prior statistical model data to reduce the amount of prior measurement data that must be maintained and processed. When prior statistical model data needs to be updated to reflect characteristics and attributes of new speech utterances, statistical model data is generated for the new speech utterances. Then the prior statistical model data and the statistical model data for the new measurement data are processed to generate the new statistical model data.

Claims

1. A method for automatically updating a statistical model for a speech recognition system comprising:

accepting data representing a plurality of individual prior speech utterances on which the statistical model is based;

accepting data representing one or more individual new speech utterances; and

generating an updated statistical model for use by the speech recognition system to recognize speech utterances, including relatively weighting the data representing the plurality of individual prior speech utterances and the data representing the one or more individual new speech utterances according to a weighting factor.

2. A method as recited in claim 1, further comprising determining the weighting factor according to a desired relative influence on the updated statistical model of the plurality of individual prior speech utterances and the one or more individual new utterances.

3. A method as recited in claim 2, wherein determining the weighting factor includes determining the weighting factor based upon an amount of the data representing the plurality of individual prior speech utterances.

4. A method as recited in claim 2, wherein determining the weighting factor includes determining the weighting factor based upon a ratio of an amount of the data representing the plurality of individual prior speech utterances to an amount of the data representing the one or more individual new speech utterances.

5. A method as recited in claim 2, wherein determining the weighting factor includes determining a confidence that each of at least some of the one or the individual new speech utterances was correctly recognized.

6. A method as recited in claim 2, wherein determining the weighting factor includes determining whether a recognition result of each of at least some of the one or more individual new speech utterances was confirmed by a user.

7. A method as recited in claim 2, wherein determining the weighting factor includes using a subject matter of each of at least some of the one or more individual new speech utterances.

8. The method as recited in claim 2, wherein determining the weighting factor includes determining the weighting factor so that the desired relative influence of the one or more individual new utterances is greater than the influence of the plurality of prior speech utterances.

9. The method as recited in claim 2, wherein determining the weighting factor includes determining the weighting factor according to a topic of utterances in the one or more individual new speech utterances.

10. A method as recited in claim 1, wherein determining the weighting factor includes determining a confidence that each of at least some of the one or more individual new speech utterances was correctly recognized.

11. A method as recited in claim 1, wherein determining the weighting factor includes determining whether a recognition result of each of at least some of the one or more individual new speech utterances was confirmed by a user.

12. The method as recited in claim 1, wherein generating the updated statistical model includes weighting the data representing the plurality of prior speech utterances to produce prior weighted data, and combining the prior weighted data with the data representing the one or more individual new utterances.

13. The method of claim 1, further comprising using the updated statistical model to recognize speech utterances.

14. A computer-readable medium carrying one or more sequences of one or more instructions for automatically retraining a telephone-based interactive speech recognition system, the one or more sequences of one or more instructions including instructions which, when executed by one or more processors, cause the one or more processors to perform steps comprising:

selecting a portion of prior measurement data for a prior set of speech utterances based upon a set of measurement data selection criteria;

retrieving the portion of prior measurement data;

determining new measurement data for a new set of speech utterances;

applying a weighting factor to the new measurement data to generate weighted new measurement data; and

generating new statistical model data for use by the telephone-based interactive speech recognition system, including using the portion of prior measurement data and the weighted new measurement data.

15. A computer-readable medium as recited in claim 14, further comprising instructions which, when executed by the one or more processors, cause the one or more processors to determine the weighting factor based upon one or more weighting criteria.

16. A computer-readable medium as recited in claim 15, wherein

the one or more weighting criteria includes an amount of data contained in the portion of prior measurement data, and

the step of determining the weighting factor based upon the one or more weighting criteria includes determining the weighting factor based upon an amount of data contained in the portion of prior measurement data.

17. A computer-readable medium as recited in claim 15, wherein

the one or more weighting criteria includes a ratio of an amount of data contained in the portion of prior measurement data to an amount of data contained in the new measurement data, and

the step of determining the weighting factor upon the one or more weighting criteria includes determining the weighting factor based upon a ratio of an amount of data contained in the portion of prior measurement data to an amount of data contained in the new measurement data.

18. A computer-readable medium as recited in claim 17, wherein

the set of measurement data selection criteria includes a confidence that a particular utterance was correctly recognized, and

the step of selecting a portion of prior measurement data for a prior set of speech utterances based upon the set of measurement data selection criteria includes selecting a portion of prior measurement data for a prior set of speech utterances based upon the confidence that the particular utterance was correctly recognized.

19. A computer-readable medium as recited in claim 17, wherein

the set of measurement data selection criteria includes whether a recognition result of a particular utterance was confirmed by a user, and

the step of selecting a portion of prior measurement data for a prior set of speech utterances based upon the set of measurement data selection criteria includes selecting a portion of prior measurement data for a prior set of speech utterances based upon whether a recognition result of a particular utterance was confirmed by a user.

20. A computer-readable medium as recited in claim 15, wherein

the one or more weighting criteria includes the subject matter of the new set of speech utterances,

the step of determining the weighting factor based upon the one or more weighting criteria includes determining the weighting factor based upon the subject matter of the new set of speech utterances.

21. A method as recited in claim 14, wherein

the set of measurement data selection criteria includes a confidence that a particular utterance was correctly recognized, and

the step of selecting a portion of prior measurement data for a prior set of speech utterances based upon the set of measurement data selection criteria includes selecting a portion of prior measurement data for a prior set of speech utterances based upon the confidence that the particular utterance was correctly recognized.

22. A computer-readable medium as recited in claim 14, wherein

the set of measurement data selection criteria includes whether a recognition result of a particular utterance was confirmed by a user, and

the step of selecting a portion of prior measurement data for a prior set of speech utterances based upon the set of measurement data selection criteria includes selecting a portion of prior measurement data for a prior set of speech utterances based upon whether a recognition result of a particular utterance was confirmed by a user.

23. A telephone-based interactive speech recognition system comprising:

a storage medium; and

a retraining mechanism communicatively coupled to the storage medium, wherein the retraining mechanism is configured to

select a portion of prior measurement data for a prior set of speech utterances based upon a set of measurement data selection criteria,

retrieve the portion of prior measurement data from the storage medium,

determine new measurement data for a new set of speech utterances,

apply a weighting factor to the new measurement data to generate weighted new measurement data, and

generate new statistical model data for use by the telephone-based interactive speech recognition system, including using the portion of prior measurement data and the weighted new measurement data.

24. A telephone-based interactive speech recognition system as recited in claim 23, wherein the retraining mechanism is further configured to determine the weighting factor based upon on or more weighting criteria.

25. A telephone-based interactive speech recognition system as recited in claim 24, wherein

the one or more weighting criteria includes an amount of data contained in the portion of prior measurement data, and

the retraining mechanism is further configured to determine the weighting factor based upon the one or more weighting criteria includes determining the weighting factor based upon an amount of data contained in the portion of prior measurement data.

26. A telephone-based interactive speech recognition system as recited in claim 24, wherein

the one or more weighting criteria includes a ratio of an amount of data contained in the portion of prior measurement data to an amount of data contained in the new measurement data, and

the step of determining the weighting factor based upon the one or more weighting criteria includes determining the weighting factor based upon a ratio of an amount of data contained in the portion of prior measurement data to an amount of data contained in the new measurement data.

27. A telephone-based interactive speech recognition system as recited in claim 24, wherein

the one or weighting criteria includes the subject matter of the new set of speech

the step of determining the weighting factor based upon the one or more weighting criteria includes determining the weighting factor based upon the subject matter of the new set of speech utterances.

28. A telephone-based interactive speech recognition system as recited in claim 23, wherein

the set of measurement data selection criteria includes a confidence that a particular utterance was correctly recognized, and

the step of selecting a portion of prior measurement data for a prior set of speech utterances based upon the set of measurement data selection criteria includes selecting a portion of prior measurement data for a prior set of speech utterances based upon the confidence that the particular utterance was correctly recognized.

29. A telephone-based interactive speech recognition system as recited in claim 23, wherein

the set to measurement data selection criteria includes whether a recognition result of a particular utterance was confirmed by a user, and

the step of selecting a portion of prior measurement data for a prior set of speech utterances based upon the set of measurement data selection criteria includes selecting a portion of prior measurement data for a prior set of speech utterances based upon whether a recognition result of a particular utterance was confirmed by a user.

30. A method for automatically updating a statistical model for a speech recognition system comprising:

selecting a portion of prior measurement data for a prior set of speech utterances based upon a set of measurement data selection criteria;

retrieving the portion of prior measurement data;

determining new measurement data for a new set of speech utterances;

applying a weighting factor to the new measurement data to generate weighted new measurement data;

generating new statistical model data using the portion of prior measurement data and the weighted new measurement data; and

using the new statistical model data to recognize speech utterances.

31. The method of claim 30 in which the measurement data selection criteria includes whether a recognition result of a particular utterance was confirmed by a user.

32. The method of claim 30 in which the measurement data selection criteria includes a confidence that a particular utterance was correctly recognized.