METHODS AND APPARATUSES FOR AUTOMATIC SPEECH RECOGNITION

• US 20110004475A1
• Filed: 07/02/2009
• Published: 01/06/2011
• Est. Priority Date: 07/02/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bellegarda, Jerome R.
• Granted Patent: US 9,431,006 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

Exemplary embodiments of methods and apparatuses for automatic speech recognition are described. First model parameters associated with a first representation of an input signal are generated. The first representation of the input signal is a discrete parameter representation. Second model parameters associated with a second representation of the input signal are generated. The second representation of the input signal includes a continuous parameter representation of residuals of the input signal. The first representation of the input signal includes discrete parameters representing first portions of the input signal. The second representation includes discrete parameters representing second portions of the input signal that are smaller than the first portions. Third model parameters are generated to couple the first representation of the input signal with the second representation of the input signal. The first representation and the second representation of the input signal are mapped into a vector space.

Claims

1. A machine implemented method, comprising:

generating first model parameters associated with a first representation of an input signal, the first representation being a discrete parameter representation;

generating second model parameters associated with a second representation of the input signal, the second representation includes a continuous parameter representation of residuals of the input signal; and

generating third model parameters to couple the first representation with the second representation.

2. A machine implemented method as in claim 1, wherein the generating the third model parameters includes

mapping the first representation and the second representation into a vector space.

3. A machine implemented method as in claim 1, wherein the first representation includes discrete parameters representing first portions of the input signal, and the second representation includes discrete parameters representing second portions of the input signal that have a finer granularity than the first portions.

4. A machine implemented method as in claim 1, wherein the generating the first model parameters includes

training a first distortion model based on first training data.

5. A machine implemented method as in claim 1, wherein the generating the second model parameters includes

training a second distortion model based on second training data.

6. A machine implemented method as in claim 1, wherein the generating the third model parameters includes

training a latent perception model based on third training data.

7. A machine implemented method as in claim 1, wherein the first model parameters are generated based on a phoneme sequence associated with a word sequence.

8. A machine implemented method as in claim 1, wherein the second model parameters are generated based on a cluster label sequence associated with a phoneme sequence.

9. A machine implemented method as in claim 1, further comprising

producing an acoustic evidence based on the second model parameters.

10. A machine implemented method as in claim 1, further comprising

receiving the input signal;

sampling the input signal into the second portions; and

determining a recovered first parameter sequence based on the second portions and the second model parameters;

determining a recovered second parameter sequence based on the recovered first parameter sequence and the first model parameters; and

determining a recovered word sequence based on the recovered second parameter sequence.

11. A machine implemented method to perform speech recognition, comprising:

receiving first portions of an acoustic signal;

determining a likelihood of a recovered first parameter sequence representing the first portions of the acoustic signal; and

determining a likelihood of a recovered second parameter sequence associated with the recovered first parameter sequence, wherein the second parameter sequence represents second portions of the acoustic signal having a coarser granularity than the first portions.

12. A machine implemented method as in claim 11, further comprising

determining a joint likelihood of the recovered first parameter sequence and the recovered second parameter sequence.

13. A machine implemented method as in claim 11, further comprising

determining a likelihood of a recovered word sequence based on the recovered second parameter sequence;

and

outputting the recovered word sequence.

14. A machine implemented method as in claim 11, wherein the first portions of the acoustic signal are associated with frames, and the second portions are associated with phonemes.

15. A machine implemented method as in claim 11, wherein the recovered second parameter sequence includes a discrete parameter representation of the acoustic signal;

and the recovered first parameter sequence includes a continuous parameter representation of residuals of the acoustic signal.

16. A machine implemented method as in claim 11, further comprising

representing the first portions of the acoustic signal by cluster labels, wherein a cluster label is associated with a set of the first portions;

computing residuals of the first portions based on the cluster labels; and

representing the residuals of the first portions by one or more continuous parameters.

17. A machine implemented method as in claim 11, further comprising

determining a likelihood of a continuous parameter representation of the acoustic signal based on the recovered first parameter sequence.

18. A machine implemented method as in claim 11, wherein the likelihood of the recovered first parameter sequence is determined based on a first distortion model.

19. A machine implemented method as in claim 11, wherein the likelihood of the recovered

second parameter sequence is determined based on a second distortion model.

20. A machine implemented method as in claim 11, wherein the determining the likelihood of

the recovered first parameter sequence includes

matching the recovered first parameter sequence with a first parameter sequence derived from training data;

and

selecting the recovered first parameter sequence based on the matching.

21. A machine implemented method as in claim 11, wherein the determining the likelihood of

the recovered second parameter sequence includes

mapping the recovered first parameter sequence to the recovered second parameter sequence.

22. A machine-readable storage medium storing executable program instructions which when executed by a data processing system causes the system to perform operations, comprising:

generating first model parameters associated with a first representation of an input signal, the first representation being a discrete parameter representation;

generating second model parameters associated with a second representation of the input signal, the second representation includes a continuous parameter representation of residuals of the input signal; and

generating third model parameters to couple the first representation with the second representation.

23. A machine-readable medium as in claim 22, wherein the generating the third model parameters includes

mapping the first representation and the second representation into a vector space.

24. A machine-readable medium as in claim 22, wherein the first representation includes discrete parameters representing first portions of the input signal, and the second representation includes discrete parameters representing second portions of the input signal that have a finer granularity than the first portions.

25. A machine-readable medium as in claim 22, wherein the generating the first model parameters includes training a first distortion model based on first training data.

26. A machine-readable medium as in claim 22, wherein the generating the second model parameters includes training a second distortion model based on second training data.

27. A machine-readable medium as in claim 22, wherein the generating the third model parameters includes training a latent perception model based on third training data.

28. A machine-readable medium as in claim 22, wherein the first model parameters are generated based on a phoneme sequence associated with a word sequence.

29. A machine-readable medium as in claim 22, wherein the second model parameters are generated based on a cluster label sequence associated with a phoneme sequence.

30. A machine-readable medium as in claim 22, further comprising instructions that cause the system to perform operations comprising

producing an acoustic evidence based on the second model parameters.

31. A machine-readable medium as in claim 22, further comprising instructions that cause the system to perform operations comprising

receiving the input signal;

sampling the input signal into the second portions; and

determining a recovered first parameter sequence based on the second portions and the second model parameters;

determining a recovered second parameter sequence based on the recovered first parameter sequence and the first model parameters; and

determining a recovered word sequence based on the recovered second parameter sequence.

32. A machine-readable medium storing executable program instructions which when executed by a data processing system causes the system to perform operations to recognize speech, comprising:

receiving first portions of an acoustic signal;

determining a likelihood of a recovered first parameter sequence representing the first portions of the acoustic signal; and

determining a likelihood of a recovered second parameter sequence associated with the recovered first parameter sequence, wherein the second parameter sequence represents second portions of the acoustic signal that have a coarser granularity than the first portions.

33. A machine-readable medium as in claim 32, further comprising instructions that cause the system to perform operations comprising

determining a joint likelihood of the recovered first parameter sequence and the recovered second parameter sequence.

34. A machine-readable medium as in claim 32, further comprising instructions that cause the system to perform operations comprising

determining a likelihood of a recovered word sequence based on the recovered second parameter sequence;

and

outputting the recovered word sequence.

35. A machine-readable medium as in claim 32, wherein the first portions of the acoustic signal are associated with frames, and the second portions are associated with phonemes.

36. A machine-readable medium as in claim 32, wherein the recovered second parameter sequence includes a discrete parameter representation of the acoustic signal;

and the recovered first parameter sequence includes a continuous parameter representation of residuals of the acoustic signal.

37. A machine-readable medium as in claim 32, further comprising instructions that cause the system to perform operations comprising

representing the first portions of the acoustic signal by cluster labels, wherein a cluster label is associated with a set of the first portions;

computing residuals of the first portions based on the cluster labels; and

representing the residuals of the first portions by one or more continuous parameters.

38. A machine-readable medium as in claim 32, further comprising instructions that cause the system to perform operations comprising

determining a likelihood of a continuous parameter representation of the acoustic signal based on the recovered first parameter sequence.

39. A machine-readable medium as in claim 32, wherein the likelihood of the recovered first parameter sequence is determined based on a first distortion model.

40. A machine-readable medium as in claim 32, wherein the likelihood of the recovered second parameter sequence is determined based on a second distortion model.

41. A machine-readable medium as in claim 32, wherein the determining the likelihood of the recovered first parameter sequence includes

matching the recovered first parameter sequence with a first parameter sequence derived from training data;

and

selecting the recovered first parameter sequence based on the matching.

42. A machine-readable medium as in claim 32, wherein the determining the likelihood of the recovered second parameter sequence includes

mapping the recovered first parameter sequence to the recovered second parameter sequence.

43. A data processing system, comprising:

a memory; and

a processor coupled to the memory, wherein the processor is configured to generate first model parameters associated with a first representation of an input signal, the first representation being a discrete parameter representation;

to generate second model parameters associated with a second representation of the input signal, the second representation includes a continuous parameter representation of residuals of the input signal; and

to generate third model parameters to couple the first representation with the second representation.

44. A data processing system as in claim 43, wherein the generating the third model parameters includes

mapping the first representation and the second representation into a vector space.

45. A data processing system as in claim 43, wherein the first representation includes discrete parameters representing first portions of the input signal, and the second representation includes discrete parameters representing second portions of the input signal that have a finer granularity than the first portions.

46. A data processing system as in claim 43, wherein the generating the first model parameters includes

training a first distortion model based on first training data.

47. A data processing system as in claim 43, wherein the generating the second model parameters includes

training a second distortion model based on second training data.

48. A data processing system as in claim 43, wherein the generating the third model parameters includes

training a latent perception model based on third training data.

49. A data processing system as in claim 43, wherein the first model parameters are generated based on a phoneme sequence associated with a word sequence.

50. A data processing system as in claim 43, wherein the second model parameters are generated based on a cluster label sequence associated with a phoneme sequence.

51. A data processing system as in claim 43, wherein the processor is further configured to producing an acoustic evidence based on the second model parameters.

52. A data processing system as in claim 43, wherein the processor is further configured to receive the input signal,

to sample the input signal into the second portions, and

to determine a recovered first parameter sequence based on the second portions and the second model parameters,

to determine a recovered second parameter sequence based on the recovered first parameter sequence and the first model parameters, and

to determine a recovered word sequence based on the recovered second parameter sequence,

53. A data processing system to perform speech recognition, comprising:

a memory; and

a processor coupled to the memory, the processor is configured to receive first portions of an acoustic signal;

to determine a likelihood of a recovered first parameter sequence representing the first portions of the acoustic signal; and

to determine a likelihood of a recovered second parameter sequence associated with the recovered first parameter sequence, wherein the second parameter sequence represents second portions of the acoustic signal that have a coarser granularity than the first portions.

54. A data processing system as in claim 53, wherein the processor is further configured to determine a joint likelihood of the recovered first parameter sequence and the recovered second parameter sequence.

55. A data processing system as in claim 53, wherein the processor is further configured to determine a likelihood of a recovered word sequence based on the recovered second parameter sequence;

and to output the recovered word sequence.

56. A data processing system as in claim 53, wherein the first portions of the acoustic signal are associated with frames, and the second portions are associated with phonemes.

57. A data processing system as in claim 53, wherein the recovered second parameter sequence includes a discrete parameter representation of the acoustic signal;

and the recovered first parameter sequence includes a continuous parameter representation of residuals of the acoustic signal.

58. A data processing system as in claim 53, wherein the processor is further configured to represent the first portions of the acoustic signal by cluster labels, wherein a cluster label is associated with a set of the first portions;

to compute residuals of the first portions based on the cluster labels; and

to represent the residuals of the first portions by one or more continuous parameters.

59. A data processing system as in claim 53, wherein the processor is further configured to determine a likelihood of a continuous parameter representation of the acoustic signal based on the recovered first parameter sequence.

60. A data processing system as in claim 53, wherein the likelihood of the recovered first parameter sequence is determined based on a first distortion model.

61. A data processing system as in claim 53, wherein the likelihood of the recovered second parameter sequence is determined based on a second distortion model.

62. A data processing system as in claim 53, wherein the determining the likelihood of the recovered first parameter sequence includes

matching the recovered first parameter sequence with an a first parameter sequence derived from training data;

and

selecting the recovered first parameter sequence based on the matching.

63. A data processing system as in claim 53, wherein the determining the likelihood of the recovered second parameter sequence includes

mapping the recovered first parameter sequence to the recovered second parameter sequence.

64. A data processing system, comprising:

means for generating first model parameters associated with a first representation of an input signal, the first representation being a discrete parameter representation;

means for generating second model parameters associated with a second representation of the input signal, the second representation includes a continuous parameter representation of residuals of the input signal; and

means for generating third model parameters to couple the first representation with the second representation.

65. A data processing system to perform speech recognition, comprising:

means for receiving first portions of an acoustic signal;

means for determining a likelihood of a recovered first parameter sequence representing the first portions of the acoustic signal; and

means for determining a likelihood of a recovered second parameter sequence associated with the recovered first parameter sequence, wherein the second parameter sequence represents second portions of the acoustic signal that have a coarser granularity than the first portions.

66. A machine readable storage medium containing executable instructions which when executed cause a data processing system to perform a speech recognition method, the method comprising:

receiving an acoustic signal;

extracting features from a digitized representation of the acoustic signal;

comparing at least some of the features to a first component of an acoustic model, the first component having a discrete parameter representation;

comparing at least some of the features to a second component of the acoustic model, the second component having a continuous parameter representation which models residuals of speech signals;

determining a recognized word from the comparing of at least some of the features to the first and the second components, wherein the discrete parameter representation and the continuous parameter representation are both used to map the features to at least one cluster label which is used to determine at least one phoneme.

67. The medium as in claim 66 wherein the discrete parameter representation and the continuous parameter representation are separately and independently trained and are coupled after training.