System and method for mobile automatic speech recognition

• US 7,386,443 B1
• Filed: 01/09/2004
• Issued: 06/10/2008
• Est. Priority Date: 01/09/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Parthasarathy, Sarangarajan, Rose, Richard Cameron
• Primary Examiner(s): CHAWAN, VIJAY B
• Application Number:
  US10/754,927
• Time in Patent Office: 1,614 Days
• Field of Search: 704270-275, 704/207, 704/205, 704/233, 704/201, 704/246, 455/563, 455/418
• US Class Current: 704/201
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 21/007 characterised by the process used

Abstract

A system and method of updating automatic speech recognition parameters on a mobile device are disclosed. The method comprises storing user account-specific adaptation data associated with ASR on a computing device associated with a wireless network, generating new ASR adaptation parameters based on transmitted information from the mobile device when a communication channel between the computing device and the mobile device becomes available and transmitting the new ASR adaptation data to the mobile device when a communication channel between the computing device and the mobile device becomes available. The new ASR adaptation data on the mobile device more accurately recognizes user utterances.

Claims

1. A computing device within a network that adapts parameters of an automatic speech recognition (ASR) system embedded on a mobile device, the computing device comprising:

means for receiving user account-specific adaptation data associated with ASR from a mobile device;

means for generating new ASR adaptation parameters using the user account-specific adaptation data transmitted from the mobile device; and

means for transmitting the new ASR adaptation parameters to the mobile device when a communication channel with the mobile device becomes available, wherein the new ASR adaptation parameters provide improved ASR for the embedded ASR system.

2. The computing device of claim 1, wherein the means for generating new ASR adaptation parameters further comprises network-based adaptation algorithms that estimate the adaptation parameters within the adaptation data based on speech utterances and ASR results obtained from the ASR system on the mobile device.

3. The computing device of claim 2, wherein the means for generating new ASR adaptation parameters further comprises means for generating the ASR adaptation parameters after determining that a threshold amount of transmitted information has been received from the mobile device.

4. The computing device of claim 1, further comprising:

means for synchronizing network-based account-specific adaptation data with the transmitted user account-specific adaptation data received from the mobile device, wherein prior to generating new ASR adaptation parameters on the computing device, the means for synchronizing synchronizes the adaptation data.

5. The computing device of claim 1, wherein the means for receiving user account-specific adaptation data associated with ASR from a mobile device further comprises means for receiving multi-modal input as part of the user account-specific adaptation data.

6. The computing device of claim 5, wherein the multi-modal input is utilized to generate the new ASR adaptation parameters.

7. A mobile device that communicates with a network via a wireless link, the mobile device comprising:

means for performing natural language automatic speech recognition (ASR);

means for receiving and storing user data associated with communication between a user and the mobile device;

means for transmitting the user data over a wireless link to a computing device associated with the wireless network, the computing device synchronizing the transmitted user data with stored user account-specific adaptation data and generating new ASR adaptation parameters based on the user data; and

means for receiving the new ASR adaptation parameters from the computing device when a communications channel becomes available between the computing device and the mobile device, wherein the new ASR adaptation parameters improve ASR performance.

8. The mobile device of claim 7, wherein the computing device revises the user account-specific adaptation data using network-based adaptation algorithms that estimate the adaptation parameters within the adaptation data based on the transmitted user data.

9. The mobile device of claim 8, wherein the computing device revises the adaptation data after determining that a sufficient amount of transmitted user data has been received from the mobile device.

10. The mobile device of claim 7, wherein the computing device, prior to revising the user account-specific adaptation data, synchronizes the adaptation data.

11. The mobile device of claim 7, wherein the received user data further comprises data associated with multi-modal communication between the user and the mobile device.

12. The mobile device of claim 11, wherein the multi-modal communication includes at least user utterances and user stylus input or keyboard input.

13. A method of generating new automatic speech recognition (ASR) parameters on a mobile device, the mobile device having an embedded ASR system, the method comprising:

storing a user account-specific adaptation data associated with ASR on a computing device associated with a wireless network that at least intermittently communicates with a mobile device;

generating new ASR adaptation parameters based on transmitted information from the mobile device; and

transmitting the new ASR adaptation parameters to the mobile device when a communication channel between the computing device and the mobile device becomes available, wherein the embedded ASR system more accurately recognized user utterances.

14. The method of claim 13, wherein generating new ASR adaptation parameters further comprises utilizing network-based adaptation algorithms that estimate the ASR adaptation parameters within the adaptation data based on speech utterances and ASR results obtained from the ASR system on the mobile device.

15. The method of claim 14, wherein generating new ASR adaptation parameters further comprises revising the adaptation data after determining that a sufficient amount of uploaded information has been received from the mobile device.

16. The method of claim 13, further comprising:

synchronizing the network-based account-specific adaptation data with the transmitted adaptation data received from the mobile device, wherein prior to generating new ASR adaptation parameters on the computing device, the synchronizing step synchronizes the adaptation data.

17. The method of claim 13, wherein storing a user account-specific adaptation data associated with ASR further comprises storing user adaptation data associated with multi-modal communication between the user and the mobile device.

18. The method of claim 17, wherein the multi-modal communication comprises at least user speech utterances and stylus or keyboard input.

19. A method of generating new automatic speech (ASR) parameters associated with an embedded ASR system on a mobile device, the method comprising:

performing automatic speech recognition (ASR) on the mobile device;

receiving and storing ASR data associated with speech communication between a user and the mobile device;

transmitting the ASR data over a wireless link to a computing device associated with the wireless network, the computing device synchronizing the transmitted ASR data with stored user account-specific adaptation data and generating new ASR adaptation parameters based on the ASR data; and

receiving the new ASR adaptation parameters from the computing device when a communications channel becomes available between the computing device and the mobile device, wherein the new ASR adaptation parameters are used for improving ASR performance on the mobile device.

20. The method of claim 19, wherein the computing device revises the user account-specific adaptation data using network-based adaptation algorithms that estimate the ASR adaptation parameters within the adaptation data based on the transmitted ASR data.

21. The method of claim 20, wherein the computing device revises the adaptation data after determining that a sufficient amount of transmitted ASR parameters has been received from the mobile device.

22. The method of claim 19, wherein the computing device, prior to revising the user account-specific adaptation data, synchronizes the adaptation data.

23. The method of claim 19, wherein the ASR parameters are associated with a statistical analysis of raw speech data.