Speech recognition with shadow modeling

• US 20040148169A1
• Filed: 01/23/2003
• Published: 07/29/2004
• Est. Priority Date: 01/23/2003
• Status: Abandoned Application

• Current Assignee: Aurilab LLC
• Original Assignee: Aurilab LLC
• Inventors: Baker, James K.
• Application Number:
  US10/348,967
• Publication Number:
  US 20040148169A1
• Time in Patent Office: Days
• US Class Current: 704/254
• CPC Class Codes: G10L 15/065 Adaptation

Abstract

A speech recognition method, system and program product for the context of an existing model for a speech element, the method comprising in one embodiment: detecting an unusual instance of the speech; creating a new model to recognize the unusual instance of the speech element; computing a score for both the existing model by itself and the new model on new speech data; determining a comparative accuracy parameter for each of the models; and selecting to keep the existing model, or to keep the new model, or to keep both the existing model and the new model based on the comparative accuracy parameters of the respective models.

Claims

1. A speech recognition method in the context of an existing model for a speech element, comprising:

detecting an unusual instance of the speech element;

creating a new model to recognize the unusual instance of the speech element;

computing a score for both the existing model by itself and the new model on new speech data;

determining a comparative accuracy parameter for each of the models; and

selecting to keep the existing model, or to keep the new model, or to keep both the existing model and the new model based on the comparative accuracy parameters of the respective models.

2. The method as defined in claim 1, wherein the step of determining an accuracy parameter for each model comprises:

determining if the speech element is present in the new speech data; and

determining the comparative accuracy parameter for one of the models based on whether the score for that model was higher or lower than the other of the models and based on whether the speech element was present in the new speech data.

3. The method as defined in claim 1, further comprising selecting a hypothesis as a recognized hypothesis.

4. The method as defined in claim 3, wherein the recognized hypothesis is displayed in order to receive explicit or implicit correction input.

5. The method as defined in claim 3, wherein the selecting a hypothesis step comprises, if one hypothesis ranks best when ranked using the score from one of the models of a given speech element and hypothesizes an instance of the given speech element, and a different hypothesis ranks best when ranked using the scores from the other model of the given speech element and does not hypothesize an instance of the given speech element, then the portion of the time that the models are used to determine the selection of the hypothesis as the recognized hypothesis, is determined substantially randomly.

6. The method as defined in claim 1, further comprising

ranking a hypothesis among a list of hypotheses based at least in part on the score computed for the existing model;

ranking the hypothesis among a list of hypotheses based at least in part on the score computed for the hybrid model;

determining if the speech element represented by the hypothesis is present in the new speech data; and

determining the comparative accuracy parameter for each of the existing model and the hybrid model based on whether the score for that model was higher or lower than the other of the models and based on whether the speech element represented by the hypothesis was present in the new speech data.

7. The method as defined in claim 6, wherein if there is a correction or a confirmation, the rewards and penalties are made larger for a model that ranked its hypothesis higher in the list of hypotheses as compared to the rewards and penalties for a model that ranked its hypothesis lower in the list of hypotheses.

8. The method as defined in claim 1, further comprising training the new model.

9. The method as defined in claim 1, further comprising training the new model against previous instances of training data for the speech element being modeled.

10. The method as defined in claim 1, further comprising unsupervised training of the new model against instances of the speech element that have been recognized and not corrected.

11. The method as defined in claim 1, wherein the creating a new model step comprises determining a mean for the new model based on a data value in the unusual instance, and using a variance from the existing model as the variance for the new model.

12. The method as defined in claim 11, further comprising:

time aligning the unusual instance with the existing model;

creating a network with a state per frame; and

for each frame using the variance from the existing model time aligned with frame and using the acoustic parameters from frame as the mean.

13. The method as defined in claim 1, wherein the comparative accuracy parameter is determined at least in part by a rate of correction by a user.

14. The method as defined in claim 1, wherein the comparative accuracy parameter is determined at least in part by a rate of correction determined automatically by the use of extra knowledge.

15. A speech recognition method in the context of an existing model for a speech element, comprising:

detecting an unusual instance of the speech;

creating a new model to recognize the unusual instance of the speech element;

creating a hybrid model that includes the new and the existing models;

computing a score for at least the existing model by itself and the hybrid model on new speech data;

determining a comparative accuracy parameter for at least each of the existing model and the hybrid model; and

selecting to keep the existing model, or to keep the hybrid model, or to keep both the existing model and the hybrid model based on the comparative accuracy parameters of the respective models.

16. The method as defined in claim 15, wherein the hybrid model comprises modeling the speech element as being generated by a stochastic process that is a mixture distribution of the existing model and the new model.

17. The method as defined in claim 16, wherein the mixture distribution is determined by matching the hybrid model to existing training data.

18. The method as defined in claim 15, wherein a score is calculated for the new model, a comparative accuracy parameter is determined for the new model, and wherein the selecting step may include selecting the new model.

19. The method as defined in claim 15, further comprising

ranking a hypothesis within a list of hypotheses based at least in part on the score computed for the existing model;

ranking the hypothesis within a list of hypotheses based at least in part on the score computed for the hybrid model; and

determining if the speech element represented by the hypothesis is present in the new speech data; and

determining the comparative accuracy parameter for each of the existing model and the hybrid model based on whether the score for that model was higher or lower than the other of the models and based on whether the speech element represented by the hypothesis was present in the new speech data.

20. The method as defined in claim 15, further comprising selecting a hypothesis as a recognized hypothesis.

21. The method as defined in claim 20, wherein the recognized hypothesis is displayed in order to receive explicit or implicit correction input.

22. The method as defined in claim 20, wherein the selecting a hypothesis step comprises, if one hypothesis ranks best when ranked using the score from one of the models of a given speech element and hypothesizes an instance of the given speech element, and a different hypothesis ranks best when ranked using the scores from the other model of the given speech element and does not hypothesize an instance of the given speech element, then the portion of the time that the models are used to determine the selection of the hypothesis as the recognized hypothesis, is determined substantially randomly.

23. The method as defined in claim 20, wherein if there is a correction or a confirmation, the rewards and penalties are made larger for a model that ranked its hypothesis higher in the list of hypotheses as compared to the rewards and penalties for a model that ranked its hypothesis lower in the list of hypotheses.

24. The method as defined in claim 15, further comprising training the hybrid model.

25. The method as defined in claim 15, further comprising training the hybrid model against previous instances of training data for the speech element being modeled.

26. The method as defined in claim 15, further comprising unsupervised training of the hybrid model against instances of the speech element that have been recognized and not corrected.

27. The method as defined in claim 15, wherein the creating a new model step comprises determining a mean for the new model based on a data value in the unusual instance, and using a variance from the existing model as the variance for the new model.

28. The method as defined in claim 27, further comprising:

time aligning the unusual instance with the existing model;

creating a network with a state per frame; and

for each frame using the variance from the existing model time aligned with frame and using the acoustic parameters from frame as the mean.

29. The method as defined in claim 15, wherein the comparative accuracy parameter is determined at least in part by a rate of correction by a user.

30. The method as defined in claim 15, wherein the comparative accuracy parameter is determined at least in part by a rate of correction determined automatically by the use of extra knowledge.

31. A program product for speech recognition in the context of an existing model for a speech element, comprising machine-readable program code for causing, when executed, a machine to perform the following method steps:

detecting an unusual instance of the speech element;

creating a new model to recognize the unusual instance of the speech element;

computing a score for both the existing model by itself and the new model on new speech data;

determining a comparative accuracy parameter for each of the models; and

selecting to keep the existing model, or to keep the new model, or to keep both the existing model and the new model based on the comparative accuracy parameters of the respective models.

32. The program product as defined in claim 31, wherein the step of determining an accuracy parameter for each model comprises:

determining if the speech element is present in the new speech data; and

determining the comparative accuracy parameter for one of the models based on whether the score for that model was higher or lower than the other of the models and based on whether the speech element was present in the new speech data.

33. The program product as defined in claim 31, further comprising program code for selecting a hypothesis as a recognized hypothesis.

34. The program product as defined in claim 33, wherein the recognized hypothesis is displayed in order to receive explicit or implicit correction input.

35. The program product as defined in claim 33, wherein the selecting a hypothesis step comprises, if one hypothesis ranks best when ranked using the score from one of the models of a given speech element and hypothesizes an instance of the given speech element, and a different hypothesis ranks best when ranked using the scores from the other model of the given speech element and does not hypothesize an instance of the given speech element, then the portion of the time that the models are used to determine the selection of the hypothesis as the recognized hypothesis, is determined substantially randomly.

36. The program product as defined in claim 31, further comprising program code for

ranking a hypothesis among a list of hypotheses based at least in part on the score computed for the existing model;

ranking the hypothesis among a list of hypotheses based at least in part on the score computed for the hybrid model; and

determining if the speech element represented by the hypothesis is present in the new speech data; and

determining the comparative accuracy parameter for each of the existing model and the hybrid model based on whether the score for that model was higher or lower than the other of the models and based on whether the speech element represented by the hypothesis was present in the new speech data.

37. The program product as defined in claim 36, wherein if there is a correction or a confirmation, the rewards and penalties are made larger for a model that ranked its hypothesis higher in the list of hypotheses as compared to the rewards and penalties for a model that ranked its hypothesis lower in the list of hypotheses.

38. The program product as defined in claim 31, further comprising program code for training the new model.

39. The program product as defined in claim 31, further comprising program code for training the new model against previous instances of training data for the speech element being modeled.

40. The program product as defined in claim 31, further comprising program code for unsupervised training of the new model against instances of the speech element that have been recognized and not corrected.

41. The program product as defined in claim 31, wherein the creating a new model step comprises determining a mean for the new model based on a data value in the unusual instance, and using a variance from the existing model as the variance for the new model.

42. The program product as defined in claim 31, further comprising program code for:

time aligning the unusual instance with the existing model;

creating a network with a state per frame; and

for each frame using the variance from the existing model time aligned with frame and using the acoustic parameters from frame as the mean.

43. The program product as defined in claim 31, wherein the comparative accuracy parameter is determined at least in part by a rate of correction by a user.

44. The program product as defined in claim 31, wherein the comparative accuracy parameter is determined at least in part by a rate of correction determined automatically by the use of extra knowledge.

45. A program product for speech recognition in the context of an existing model for a speech element, comprising machine-readable program code for causing, when executed, a machine to perform the following method steps:

detecting an unusual instance of the speech;

creating a new model to recognize the unusual instance of the speech element;

creating a hybrid model that includes the new and the existing models;

computing a score for at least the existing model by itself and the hybrid model on new speech data;

determining a comparative accuracy parameter for at least each of the existing model and the hybrid model; and

selecting to keep the existing model, or to keep the hybrid model, or to keep both the existing model and the hybrid model based on the comparative accuracy parameters of the respective models.

46. The program product as defined in claim 45, wherein the hybrid model comprises modeling the speech element as being generated by a stochastic process that is a mixture distribution of the existing model and the new model.

47. The program product as defined in claim 46, wherein the mixture distribution is determined by matching the hybrid model to existing training data.

48. The program product as defined in claim 45, wherein a score is calculated for the new model, a comparative accuracy parameter is determined for the new model, and wherein the selecting step may include selecting the new model.

49. The program payment as defined in claim 45, further comprising program code for

ranking a hypothesis among a list of hypotheses based at least in part on the score computed for the existing model;

ranking the hypothesis among a list of hypotheses based at least in part on the score computed for the hybrid model; and

determining if the speech element represented by the hypothesis is present in the new speech data; and

determining the comparative accuracy parameter for each of the existing model and the hybrid model based on whether the score for that model was higher or lower than the other of the models and based on whether the speech element represented by the hypothesis was present in the new speech data.

50. The program product as defined in claim 45, further comprising program code for selecting a hypothesis as a recognized hypothesis.

51. The program product as defined in claim 50, wherein the recognized hypothesis is displayed in order to receive explicit or implicit correction input.

52. The program product as defined in claim 50, wherein the selecting a hypothesis step comprises, if one hypothesis ranks best when ranked using the score from one of the models of a given speech element and hypothesizes an instance of the given speech element, and a different hypothesis ranks best when ranked using the scores from the other model of the given speech element and does not hypothesize an instance of the given speech element, then the portion of the time that the models are used to determine the selection of the hypothesis as the recognized hypothesis, is determined substantially randomly.

53. The program product as defined in claim 50, wherein if there is a correction or a confirmation, the rewards and penalties are made larger for a model that ranked its hypothesis higher in the of hypotheses as compared to the rewards and penalties for a model that ranked its hypothesis lower in the of hypotheses.

54. The program product as defined in claim 45, further comprising program code for training the hybrid model.

55. The program product as defined in claim 45, further comprising program code for training the hybrid model against previous instances of training data for the speech element being modeled.

56. The program product as defined in claim 45, further comprising program code for unsupervised training of the hybrid model against instances of the speech element that have been recognized and not corrected.

57. The program product as defined in claim 45, wherein the creating a new model step comprises determining a mean for the new model based on a data value in the unusual instance, and using a variance from the existing model as the variance for the new model.

58. The program product as defined in claim 57, further comprising program code for:

time aligning the unusual instance with the existing model;

creating a network with a state per frame; and

for each frame using the variance from the existing model time aligned with frame and using the acoustic parameters from frame as the mean.

59. The program product as defined in claim 45, wherein the comparative accuracy parameter is determined at least in part by a rate of correction by a user.

60. The program product as defined in claim 45, wherein the comparative accuracy parameter is determined at least in part by a rate of correction determined automatically by the use of extra knowledge.

61. A system for speech recognition in the context of an existing model for a speech element, comprising:

a component for detecting an unusual instance of the speech;

a component for creating a new model to recognize the unusual instance of the speech element;

a component for computing a score for both the existing model by itself and the new model on new speech data;

a component for determining a comparative accuracy parameter for each of the models; and

a component for selecting to keep the existing model, or to keep the new model, or to keep both the existing model and the new model based on the comparative accuracy parameters of the respective models.

62. A system for speech recognition in the context of an existing model for a speech element, comprising:

a component for detecting an unusual instance of the speech;

a component for creating a new model to recognize the unusual instance of the speech element;

a component for creating a hybrid model that includes the new and the existing models;

a component for computing a score for at least the existing model by itself and the hybrid model on new speech data;

a component for determining a comparative accuracy parameter for at least each of the existing model and the hybrid model; and

a component for selecting to keep the existing model, or to keep the hybrid model, or to keep both the existing model and the hybrid model based on the comparative accuracy parameters of the respective models.