Distributed speech recognition for mobile communication devices

• US 20030182113A1
• Filed: 03/24/2003
• Published: 09/25/2003
• Est. Priority Date: 11/22/1999
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Huang, Xuedong
• Application Number:
  US10/395,609
• Publication Number:
  US 20030182113A1
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

A method of performing speech recognition, and a mobile computing device implementing the same, are disclosed. The method includes receiving audible speech at a microphone of the mobile computing device. The audible speech is converted into speech signals at the mobile computing device. Also at the mobile computing device, preliminary and secondary speech recognition functions are performed on the speech signals to obtain requests for results from modules. Then, the requests for results are transmitted from the mobile computing device to a second computing device located remotely from the mobile computing device to obtain the results which are then transmitted back to the mobile computing device for completion of the speech recognition process.

Claims

1. A method of performing speech recognition, the method comprising:

receiving audible speech at a microphone of a mobile computing device;

converting the audible speech into speech signals using the mobile computing device;

performing preliminary speech recognition functions on the speech signals using the mobile computing device to obtain intermediate speech recognition results;

performing secondary speech recognition functions on the speech signals using the mobile computing device to obtain requests for results;

transmitting at least a portion of the requests for results to a second computing device located remotely from the mobile device in order to access at least one module located on the second computing device to obtain the requested results; and

receiving the requested results, from the second computing device, at the mobile computing device to provide output text representative of the audible speech.

2. The method of claim 1, and further comprising:

receiving the at least a portion of the requests for results at the second computing device;

accessing the at least one module on the second computing device to get the requested results; and

transmitting the requested results to the mobile device.

3. The method of claim 2 further comprising:

transmitting a portion of the requests for results to an acoustic model located on the mobile computing device.

4. The method of claim 3 wherein transmitting the at least a portion of the requests for results to the second computing device further comprises transmitting the at least a portion of the requests for results to a language model located on the remote computer the method further comprising:

transmitting a portion of the requests for results to a language model located on the mobile computing device.

5. The method of claim 4 further comprising:

updating the language model on the mobile computing device with information contained in the language model on the second computing device.

6. The method of claim 2, wherein accessing the at least one module on the second computing device further comprises accessing acoustic model information stored in a memory of the second computing device to provide the output text on the mobile computing device representative of the audible speech as a function of the intermediate speech recognition results and of the acoustic model information.

7. The method of claim 2, wherein accessing the at least one module on the second computing device further comprises accessing language model information stored in a memory of the second computing device to provide the output text on the mobile computing device representative of the audible speech as a function of the intermediate speech recognition results and of the language model information.

8. The method of claim 1, wherein converting the audible speech into speech signals at the mobile computing device further comprises:

converting the audible speech signals into analog signals; and

digitizing the analog signals to obtain the speech signals.

9. The method of claim 1, wherein performing the preliminary speech recognition functions on the speech signals to obtain the intermediate speech recognition results further comprises performing feature extraction functions on the speech signals to obtain the intermediate speech recognition results indicative of features of the speech signals.

10. The method of claim 9, wherein performing preliminary speech recognition functions on the speech signals further comprises determining Mel-Frequency Cepstrum Coefficients from the speech signals, wherein performing secondary speech recognition functions further comprises determining the requests for results based upon the Mel-Frequency Cepstrum Coefficients, and wherein transmitting at least a portion of the requests for results further comprises transmitting the at least a portion of the requests for results based on the Mel-Frequency Cepstrum Coefficients from the mobile computing device to the second computing device.

11. The method of claim 9, wherein performing preliminary speech recognition functions on the speech signals further comprises determining vector quantized indices from the speech signals, wherein performing secondary speech recognition functions further comprises determining the requests for results based upon the vector furnished indicates, and wherein transmitting requests further comprises transmitting requests based upon the vector quantized indices from the mobile computing device to the second computing device.

12. The method of claim 9, wherein performing preliminary speech recognition functions on the speech signals further comprises determining Hidden Markov Modeling (HMM) scores from the speech signals, wherein performing secondary speech recognition functions further comprises determining the requests for results based upon the vector furnished indicates, and wherein transmitting requests further comprises transmitting requests based upon the HMM scores from the mobile computing device to the second computing device.

13. The method of claim 9, wherein performing preliminary speech recognition functions on the speech signals further comprises determining Hidden Markov Modeling (HMM) state output probability density functions from the speech signals, wherein performing secondary speech recognition functions further comprises determining the requests for results based upon the vector furnished indicates, and wherein transmitting requests further comprises transmitting requests based upon the HMM state output probability density functions from the mobile computing device to the second computing device.

14. The method of claim 9, wherein performing preliminary speech recognition functions on the speech signals further comprises determining Cepstral coefficients from the speech signals, wherein performing secondary speech recognition functions further comprises determining the requests for results based upon the vector furnished indicates, and wherein transmitting requests further comprises transmitting requests based upon the Cepstral coefficients from the mobile computing device to the second computing device.

15. The method of claim 9, wherein performing preliminary speech recognition functions on the speech signals further comprises determining feature vectors from the speech signals, wherein performing secondary speech recognition functions further comprises determining the requests for results based upon the vector furnished indicates, and wherein transmitting the intermediate speech recognition results from the mobile device to the second computing device further comprises transmitting the feature vectors from the mobile computing device to the second computing device.

16. The method of claim 1, wherein transmitting the at least a portion of the requests for results further comprises transmitting the at least a portion of the requests for results from the mobile computing device to the second computing device over a wireless communications network.

17. The method of claim 1, wherein transmitting the at least a portion of the requests for results further comprises transmitting the at least a portion of the requests for results from the mobile computing device to the second computing device over a communications network having a bandwidth which is less than a bandwidth of the microphone of the mobile computing device.

18. The method of claim 1, and further comprising providing the output text, at the mobile computing device, as a function of the received requested results.

19. A computer-readable medium having mobile computer-executable instructions for performing the steps of:

implementing preliminary and secondary speech recognition functions on speech signals, corresponding to audible speech from a user of a mobile computer having a microphone, to obtain requests for results;

sending at least a portion of the requests for results to a transmitter of the mobile computer to transmit the at least a portion of the requests for results from the mobile computer to a second computer located remotely from the mobile computer; and

receiving the results from the second computer to finish the speech recognition functions on the mobile device.

20. The computer readable medium of claim 19, wherein the computer-executable instructions for performing the step of implementing the preliminary and secondary speech recognition functions on the speech signals further includes computer-executable instructions for performing feature extraction functions on the speech signals to obtain intermediate speech recognition results indicative of features of the speech signals, and wherein the computer executable instructions further comprise using the intermediate speech recognition results to obtain the requests for results based upon the feature extraction functions.

21. The method of claim 20, wherein the computer-executable instructions for performing the feature extraction functions on the speech signals further includes computer-executable instructions for determining Mel-Frequency Cepstrum Coefficients from the speech signals, and wherein the computer-executable instructions for sending the at least a portion of the requests for results to the transmitter of the mobile computer further includes computer-executable instructions for sending the at least a portion of the requests for results based on the Mel-Frequency Cepstrum Coefficients.

22. The method of claim 20, wherein the computer-executable instructions for performing the feature extraction functions on the speech signals further includes computer-executable instructions for determining vector quantized indices from the speech signals, and wherein the computer-executable instructions for sending the at least a portion of the requests for results to the transmitter of the mobile computer further includes computer-executable instructions for sending the at least a portion of the requests for results based on the vector quantized indices.

23. The method of claim 20, wherein the computer-executable instructions for performing the feature extraction functions on the speech signals further includes computer-executable instructions for determining Hidden Markov Modeling (HMM) scores from the speech signals, and wherein the computer-executable instructions for sending the at least a portion of the requests for results to the transmitter of the mobile computer further includes computer-executable instructions for sending the at least a portion of the requests for results based on the HMM scores.

24. The method of claim 20, wherein the computer-executable instructions for performing the feature extraction functions on the speech signals further includes computer-executable instructions for determining Hidden Markov Modeling (HMM) state output probability density functions from the speech signals, and wherein the computer-executable instructions for sending the at least a portion of the requests for results to the transmitter of the mobile computer further includes computer-executable instructions for sending the at least a portion of the requests for results based on the HMM state output probability density functions.

25. The method of claim 20, wherein the computer-executable instructions for performing the feature extraction functions on the speech signals further includes computer-executable instructions for determining Cepstral coefficients from the speech signals, and wherein the computer-executable instructions for sending the at least a portion of the requests for results to the transmitter of the mobile computer further includes computer-executable instructions for sending the at least a portion of the requests for results based on the Cepstral coefficients.

26. The method of claim 20, wherein the computer-executable instructions for performing the feature extraction functions on the speech signals further includes computer-executable instructions for determining feature vectors from the speech signals, and wherein the computer-executable instructions for sending the at least a portion of the requests for results to the transmitter of the mobile computer further includes computer-executable instructions for sending the at least a portion of the requests for results based on feature vectors.

27. A mobile computer comprising:

a microphone adapted to convert audible speech into analog signals;

an analog-to-digital converter coupled to the microphone and adapted to digitize the audible speech to provide speech signals;

a feature extraction module adapted to perform preliminary speech recognition functions on the speech signals to provide intermediate speech recognition results;

a speech recognition module configured to perform secondary speech recognition results to obtain requests for results; and

a transceiver coupled to the speech recognition module and adapted to transmit at least a portion of the requests for results from the mobile computer to a second computer located remotely from the mobile computer, and to receive the requested results from the second computer.

28. The mobile computer of claim 27, wherein the feature extraction module is adapted to determine from the speech signals Mel-Frequency Cepstrum Coefficients and to provide the Mel-Frequency Cepstrum Coefficients as the intermediate speech recognition results.

29. The mobile computer of claim 27, wherein the feature extraction module is adapted to determine from the speech signals vector quantized indices and to provide the vector quantized indices as the intermediate speech recognition results.

30. The mobile computer of claim 27, wherein the feature extraction module is adapted to determine from the speech signals Hidden Markov Modeling (HMM) scores and to provide the HMM scores as the intermediate speech recognition results.

31. The mobile computer of claim 27, wherein the feature extraction module is adapted to determine from the speech signals Hidden Markov Modeling (HMM) state output probability density functions and to provide the HMM state output probability density functions as the intermediate speech recognition results.

32. The mobile computer of claim 27, wherein the feature extraction module is adapted to determine from the speech signals Cepstral coefficients and to provide the Cepstral coefficients as the intermediate speech recognition results.

33. The mobile computer of claim 27, wherein the feature extraction module is adapted to determine from the speech signals feature vectors and to provide the feature vectors as the intermediate speech recognition results.

34. The mobile computer of claim 27 further comprising:

an acoustic model configured to provide acoustic model results to the speech recognition module in response to the requests for results.

35. The mobile computer of claim 27 further comprising:

a language model configured to provide results to the speech recognition module in response to the request for results.

36. The mobile computer of claim 33 further comprising:

a language model configured to provide results to the speech recognition module in response to the request for results;

wherein the language model is configured to be updated from a remote language model.

37. The mobile computer of claim 34 further comprising:

a language model configured to provide results to the speech recognition module in response to the request for results;

wherein the language model is configured to be updated from a remote language model.