System and method of performing user-specific automatic speech recognition

• US 8,145,481 B2
• Filed: 09/09/2008
• Issued: 03/27/2012
• Est. Priority Date: 03/20/2001
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Gajic, Bojana, Narayanan, Shrikanth Sambasivan, Parthasarathy, Sarangarajan, Rose, Richard Cameron, Rosenberg, Aaron Edward
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US12/207,175
• Publication Number:
  US 20090006088A1
• Time in Patent Office: 1,295 Days
• Field of Search: 704/231, 704/233, 704/243, 704/244
• US Class Current: 704/231
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence

Abstract

Speech recognition models are dynamically re-configurable based on user information, application information, background information such as background noise and transducer information such as transducer response characteristics to provide users with alternate input modes to keyboard text entry. Word recognition lattices are generated for each data field of an application and dynamically concatenated into a single word recognition lattice. A language model is applied to the concatenated word recognition lattice to determine the relationships between the word recognition lattices and repeated until the generated word recognition lattices are acceptable or differ from a predetermined value only by a threshold amount. These techniques of dynamic re-configurable speech recognition provide for deployment of speech recognition on small devices such as mobile phones and personal digital assistants as well environments such as office, home or vehicle while maintaining the accuracy of the speech recognition.

Claims

1. A method of performing speech processing, the method comprising:

repeatedly determining parameters of a background model based on sampled information collected at a periodic time interval during a received voice request from a speaker;

determining parameters of a transducer model;

adapting a speech recognition model based on user-specific transformations corresponding to a determined identity of the speaker and on at least one of the background model and the transducer model;

receiving a data field selection by the speaker; and

applying, via a processor, one of a plurality of language models to the received voice request for speech recognition based on the data field, wherein the speech recognition uses a different language model for each data field selected by the speaker.

2. The method of claim 1, further comprising:

determining the identity of the speaker based, at least in part, on a user identifier.

7. The method of claim 1, further comprising:

translating the voice request into an HTTP protocol request.

8. The method of claim 7, further comprising:

forwarding information from a database based on the HTTP protocol request to a dialog server; and

generating from a dialog server a response to the voice request.

9. The method of claim 8, wherein the response is one of a voice response, a text response, a tactile response and/or a Braille response.

3. A method of performing speech processing, the method comprising:

repeatedly determining parameters of a background model based on sampled information collected at a period time interval during a received voice request from a speaker;

determining parameters of a transducer model;

adapting a speech recognition model based on user-specific transformations corresponding to a determined identity of the speaker and on at least one of the background model and the transducer model;

re-scoring, via a processor, automatic speech recognition using the speech recognition model by;

generating word lattices representative of speech utterances in the received voice request;

concatenating the word lattices into a single concatenated lattice; and

applying at least one language model to the single concatenated lattice in order to determine word lattice inter-relationships;

determining information in the received voice request based on the re-scored results of the speech recognition model; and

adjusting the periodic time interval based, at least in part, on determined changes in the sampled information.

4. The method of claim 3, further comprising:

generating a confidence score to determine whether the generated word lattices are acceptable.

5. The method of claim 4, wherein:

the parameters of the background model are determined based on a first sample period;

the parameters of the transducer model are determined based on a second sample period; and

the confidence score is compared to a predetermined value in order to determine whether to perform the automatic speech recognition process again.

6. The method of claim 4, further comprising:

saving at least one of the parameters of the background model and the parameters of the transducer model.

10. A system for performing speech processing, the system comprising:

a processor;

a first module configured to control the processor to determine parameters of a background model repeatedly based on sampled information collected at a periodic time interval during a received voice request;

a second module configured to control the processor to determine parameters of a transducer model;

a third module configured to control the processor to adapt a speech recognition model based on user-specific transformations corresponding to a determined identity of the speaker and on at least one of the background model and the transducer model;

a fourth module configured to control the processor to receive a data field selection by the speaker; and

a fifth module configured to control the processor to apply one of a plurality of language models to the received voice request for speech recognition based on the data field, wherein the speech recognition uses a different language model for each data field selected by the speaker.

11. The system of claim 10, further comprising:

a sixth module configured to control the processor to determine an identity of the speaker based, at least in part, on a user identifier.

16. The system of claim 10, further comprising:

a sixth module configured to control the processor to translate the voice request into an HTTP protocol request.

17. The system of claim 10, further comprising:

a sixth module configured to control the processor to forward information from a database based on the HTTP protocol request to a dialog server; and

a seventh module configured to control the processor to generate from a dialog server a response to the voice request.

18. The system of claim 17, wherein the response is one of a voice response, a text response, a tactile response and/or a Braille response.

12. A system for performing speech processing, the system comprising:

a processor;

a first module configured to control the processor to determine parameters of a background model repeatedly based on sampled information collected at a periodic time interval during a received voice request;

a second module configured to control the processor to determine parameters of a transducer model;

a third module configured to control the processor to adapt a speech recognition model based on user-specific transformations corresponding to a determined identity of the speaker and on at least one of the background model and the transducer model;

a fourth module configured to control the processor to re-score automatic speech recognition using the speech recognition model, the fourth module further configured to control the processor to;

generate word lattices representative of speech utterances in the received voice request,

concatenate the word lattices into a single concatenated lattice, and

apply at least one language model to the single concatenated lattice in order to determine word lattice inter-relationships;

a fifth module configured to control the processor to determine information in the received voice request based on the re-scored results of the speech recognition model; and

a sixth module configured to control the processor to adjust the periodic time interval based, at least in part, on determined changes in the sampled information.

13. The system of claim 12, further comprising:

a seventh module configured to control the processor to generate a confidence score to determine whether the generated word lattices are acceptable.

14. The system of claim 13, wherein:

the parameters of the background model are determined based on a first sample period;

the parameters of the transducer model are determined based on a second sample period; and

the confidence score is compared to a predetermined value in order to determine whether to perform the automatic speech recognition process again.

15. The system of claim 13, further comprising:

a eighth module configured to control the processor to save at least one of the parameters of the background model and the parameters of the transducer model.

19. A non-transitory computer readable medium storing a computer program having instructions for controlling a processor of a computer device to perform speech recognition, the steps comprising:

determining an identity of the speaker based, at least in part, on a user identifier;

determining parameters of a background model based on sampled information collected at a periodic time interval during a received voice request;

determining parameters of a transducer model; and

adapting a speech recognition model based on user-specific transformations corresponding to the determined identity of the speaker and on at least one of the background model and the transducer model;

receiving a data field selection by the speaker; and

applying one of a plurality of language models to the received voice request for speech recognition based on the data field, wherein the speech recognition uses a different language model for each data field selected by the speaker.

20. The non-transitory computer readable medium of claim 19, the steps further comprising:

translating the voice request into an HTTP protocol request.

21. The non-transitory computer readable medium of claim 19, wherein the steps further comprise:

forwarding information from a database based on the HTTP protocol request to a dialog server; and

generating a response to the voice request.

22. The non-transitory computer readable medium of claim 21, wherein the response is one of a voice response, a text response, a tactile response and/or a Braille response.