Speech recognition text entry for software applications

• US 8,996,379 B2
• Filed: 10/01/2007
• Issued: 03/31/2015
• Est. Priority Date: 03/07/2007
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Vlingo Corporation (Microsoft Corporation)
• Inventors: Cerra, Joseph P., Kishchenko, Roman V., Nguyen, John N., Phillips, Michael S., Shu, Han
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US11/865,694
• Publication Number:
  US 20080221879A1
• Time in Patent Office: 2,738 Days
• Field of Search: 704/270, 704/275
• US Class Current: 704/270
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications

Abstract

In embodiments of the present invention improved capabilities are described for a mobile environment speech processing facility. The present invention may provide for the entering of text into a software application resident on a mobile communication facility, where recorded speech may be presented by the user using the mobile communications facility'"'"'s resident capture facility. Transmission of the recording may be provided through a wireless communication facility to a speech recognition facility, and may be accompanied by information related to the software application. Results may be generated utilizing the speech recognition facility that may be independent of structured grammar, and may be based at least in part on the information relating to the software application and the recording. The results may then be transmitted to the mobile communications facility, where they may be loaded into the software application. In addition, the speech recognition facility may be adapted based on usage.

Claims

1. A method of entering text into a software application resident on a mobile communication facility comprising:

recording speech presented by a user using a mobile communication facility resident capture facility;

transmitting the recording through a wireless communication facility to a speech recognition facility, wherein an automated speech recognition router is configured to provide a connection between the wireless communication facility, the speech recognition facility, and a user database and wherein the automated speech recognition router is further configured to select the speech recognition facility from one or more available speech recognition facilities, wherein the automated speech recognition router is configured to stream one or more waveform packets to an automated speech recognition server and to cache the speech presented by the user and session information at the automated speech recognition router;

transmitting information relating to the software application to the speech recognition facility;

selecting at least one statistical language model from a plurality of language models;

generating results utilizing the speech recognition facility using the at least one statistical language model based at least in part on the information relating to the software application and the recording;

transmitting the results to the mobile communications facility;

loading the results into the software application; and

adapting the speech recognition facility based on usage.

2. The method of claim 1, wherein adapting the speech recognition facility based on usage includes at least one of adapting an acoustic model, adapting a set of pronunciation, adapting a vocabulary, and adapting a language model.

3. The method of claim 1, wherein, adapting the speech recognition facility includes adapting recognition models based on usage data.

4. The method of claim 1, wherein the at least one statistical language model is a large vocabulary statistical language model.

5. The method of claim 1, further comprising combining words recognized from more than one statistical language model to generate the results.

6. The method of claim 1, further comprising a subsequent step of generating results in the speech recognition facility based on results obtained in a previous step of generating results.

7. The method of claim 3, wherein adapting recognition models makes use of human transcriptions of speech of the user.

8. The method of claim 1, wherein the plurality of language models is an initial set of language models selected based on the information relating to the software application and the recording.

9. The method of claim 3, wherein adapting recognition models is specific to the user or groups of users.

10. The method of claim 3, wherein adapting recognition models is specific to the software application or groups of software applications.

11. The method of claim 1, further comprising using manual input of words representing the user'"'"'s speech to improve the statistical language model.

12. The method of claim 1, wherein the information relating to the software application includes at least one of an identity of the application, an identity of a text box within the application, contextual information within the application, an identity of the mobile communication facility, and an identity of the user.

13. The method of claim 1, further comprising deciding whether the selected at least one statistical language model provides insufficient recognition output and selecting at least one other language model based on the speech recognized by the selected at least one statistical language model.

14. The method of claim 1, wherein the speech recognition facility includes at least one of an acoustic model, a set of pronunciations, and a vocabulary.

15. The method of claim 1, wherein the at least one statistical language model is selected based on the information relating to the software application and the recording.

16. The method of claim 1, further comprising selecting a different at least one statistical language model based on the results generated.

17. The method of claim 1, further comprising displaying a correction interface for the user to correct or change the results.

18. The method of claim 1, further comprising running a plurality of language models in multiple passes in the speech recognition facility.

19. The method of claim 8, wherein the initial set of language models includes a general language model for a search, a general language model for date and time entry, and a general language model for digit string entry.

20. The method of claim 8, wherein the initial set of language models is tailored to a group to which the user belongs.

21. The method of claim 18, wherein the outputs of the multiple passes in the speech recognition facility are combined into a single result by a merging of results from the multiple passes.

22. The method of claim 21, wherein the merging of results is at a word level.

23. The method of claim 21, wherein the merging of results is done at a phrase level.

24. A method of entering text into a software application resident on a mobile communication facility comprising:

recording speech presented by a user using a mobile communication facility resident capture facility;

transmitting the recording through a wireless communication facility to a speech recognition facility, wherein an automated speech recognition router is configured to provide a connection between the wireless communication facility, the speech recognition facility, and a user database and wherein the automated speech recognition router is further configured to select the speech recognition facility from one or more available speech recognition facilities, wherein the automated speech recognition router is configured to stream one or more waveform packets to an automated speech recognition server and to cache the speech presented by the user and session information at the automated speech recognition router;

transmitting information relating to the software application to the speech recognition facility;

selecting an initial set of a plurality of language models for processing the recorded speech;

generating results utilizing the speech recognition facility independent of a structured grammar based at least in part on the information relating to the software application and the recording;

transmitting the results to the mobile communications facility; and

loading the results into the software application.

25. The method of claim 24, wherein at least one of the initial set of the plurality of language models is a statistical language model.

26. The method of claim 24, further comprising deciding whether the selected initial set provides insufficient recognition output and selecting at least one other language model based on the speech recognized by the selected initial set.

27. The method of claim 24, further comprising adapting the speech recognition facility based on usage.