Using results of unstructured language model based speech recognition to control a system-level function of a mobile communications facility

• US 8,838,457 B2
• Filed: 08/01/2008
• Issued: 09/16/2014
• Est. Priority Date: 03/07/2007
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Vlingo Corporation (Microsoft Corporation)
• Inventors: Cerra, Joseph P., Nguyen, John N., Phillips, Michael S., Shu, Han
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US12/184,342
• Publication Number:
  US 20090030696A1
• Time in Patent Office: 2,237 Days
• Field of Search: 704/275, 704/270, 704/270.1, 704/231, 704/235, 704/236, 704/243, 704/251
• US Class Current: 704/275
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A user may control a mobile communication facility through recognized speech provided to the mobile communication facility. Speech that is recorded by a user using a mobile communication facility resident capture facility. A speech recognition facility generates results of the recorded speech using an unstructured language model based at least in part on information relating to the recording. A function of the operating system of the mobile communication facility is controlled based on the results.

Claims

1. A method of allowing a user to control a mobile communication facility comprising:

receiving speech and information currently displayed in a mobile communication facility from a user using a mobile communication facility resident capture facility, wherein the speech presented by the user includes a command and a subject and wherein the speech and information was transmitted from the mobile communication facility to a speech recognition facility;

utilizing, by the speech recognition facility, (i) contextual information not provided in the speech and (ii) at least one statistical language model to recognize the command and a subject from the speech presented by the user, wherein the contextual information includes usage history of the mobile communication facility, information from a user'"'"'s favorites list, information about the user'"'"'s address book or contact list, email content, or information currently displayed in by the mobile communication facility;

determining, by the speech recognition facility, at least one application to invoke on the mobile communication facility to perform an operation on the mobile communication facility based on the contextual information, the command, and the subject of the speech, wherein the operation includes an action defined by the command using parameters based on the subject; and

causing the mobile communication facility to automatically perform the operation on the mobile communication facility using the determined at least one application.

2. The method of claim 1 further comprising deciding whether the at least one statistical language model provides insufficient recognition output and selecting at least one other language model apart from the set of language models based on speech recognized by the selected at least one statistical language model.

3. The method of claim 1 further comprising using user feedback to adapt at least one of an acoustic model and a pronunciation model of the speech recognition facility.

4. The method of claim 1 further comprising adapting a language model used by the speech recognition facility for tagging.

5. The method of claim 1 wherein the command is “

navigate to”

.

6. The method of claim 1 wherein the mobile communication facility presents a series of screens allowing the user to input or select a destination and to display points of interest.

7. The method of claim 1 wherein the mobile communication facility presents a series of screens allowing the user to plot a route to a desired destination.

8. The method of claim 1 wherein the at least one of the plurality of functions of the mobile communication facility includes a navigation function including a playback mode.

9. The method of claim 1 wherein the at least one application includes a search engine application and the subject includes a search query.

10. The method of claim 1 wherein the subject includes a geographical landmark.

11. The method of claim 10 wherein the operation includes displaying turn-by-turn navigation within a navigation application.

12. The method of claim 1 wherein the at least one application includes a music application and the subject includes a media file.

13. The method of claim 12 wherein the subject includes a name of content in the media file.

14. The method of claim 13 wherein the mobile communication facility presents a series of screens allowing a user to select a desired song from a desired artist.

15. A method of allowing a user to control a mobile communication facility compromising:

providing an input facility of a mobile communication facility, the input facility allowing a user to begin to record speech on the mobile communication facility;

upon user interaction with the input facility, recording speech presented by a user using a mobile communication facility resident capture facility, wherein the speech includes a command and a subject;

transmitting speech and information currently displayed in the mobile communication facility from the mobile communication facility resident capture facility, wherein the speech presented by the user includes the command and the subject;

transmitting contextual information not provided in the speech, wherein the contextual information includes usage history of the mobile communication facility, information from a user'"'"'s favorites list, information about the user'"'"'s address book or contact list, email content, or information currently displayed in by the mobile communication facility;

receiving generated results utilizing a remote speech recognition facility using the at least one language model based at least in part on content of the recorded speech and the contextual information relating to the recording;

receiving from the remote speech recognition facility a determination of at least one application to invoke on the mobile communication facility to perform an operation on the mobile communication facility based on the contextual information, the command, and the subject of the speech, wherein the operation includes an action defined by the command using parameters based on the subject; and

automatically performing the action on the mobile communication facility using the determined at least one application.

16. The method of claim 15, further comprising deciding whether the at least one language model and the initial set of language models provides insufficient recognition output and selecting at least one other language model apart from the initial set of language models based on the speech recognized in a first pass by the selected at least one language model and in a second pass using a different language model from the initial set of language models.

17. The method of claim 15, further comprising generating results using a different language model, in which the speech is determined in a plurality of generating results.

18. The method of claim 15 wherein the command is “

navigate to”

.

19. The method of claim 15 wherein the mobile communication facility presents a series of screens allowing the user to input or select a destination and to display points of interest.

20. The method of claim 15 wherein the mobile communication facility presents a series of screens allowing the user to plot a route to a desired destination.

21. The method of claim 15 wherein the at least one of the plurality of functions of the mobile communication facility includes a navigation function including a playback mode.

22. The method of claim 15 wherein the at least one application includes a search engine application and the subject includes a search query.

23. The method of claim 15 wherein the subject includes a geographical landmark.

24. The method of claim 23 wherein the operation includes displaying turn-by-turn navigation within a navigation application.

25. The method of claim 15 wherein the at least one application includes a music application and the subject includes a media file.

26. The method of claim 25 wherein the subject includes a name of content in the media file.

27. The method of claim 26 wherein the mobile communication facility presents a series of screens allowing a user to select a desired song from a desired artist.

28. A system of allowing a user to control a mobile communication facility comprising:

a mobile communication facility resident capture facility for recording speech presented by a user, wherein the speech includes a command and a subject, wherein the mobile communication facility is configured to transmit speech and information currently displayed in the mobile communication facility from the mobile communication facility resident capture facility, wherein the speech presented by the user includes a command and a subject;

a remote speech recognition facility for generating results using at least one language model from a first set of language models that includes at least one statistical model based at least in part on content of the recorded speech and contextual information relating to the recording, wherein the speech recognition facility is configured to utilize (i) contextual information not provided in the speech and (ii) at least one statistical language model to recognize the command and the subject from the speech presented by the user, wherein the contextual information includes usage history of the mobile communication facility, information from a user'"'"'s favorites list, information about the user'"'"'s address book or contact list, email content, or information currently displayed in by the mobile communication facility and wherein the remote speech recognition facility is further configured to determine at least one application to invoke on the mobile communication facility to perform an operation on the mobile communication facility based on the contextual information, the command, and the subject of the speech; and

an operating system of the mobile communication facility, the operating system comprising a plurality of functions of which one is selected and controlled by the user based on the results.

29. The system of claim 28 wherein before the selected function is initiated, the user is presented with an option to verify or correct the recognized speech.

30. The system of claim 28 wherein the function is selected by prompting the user to interact with a menu on the mobile communication facility to select an input to which results generated by the speech recognition facility will be delivered.

31. The system of claim 28 wherein the function is selected based on inferring a function based on the content of the results generated by the speech recognition facility.

32. The system of claim 28 wherein the function is selected based on stating the name of the function near the beginning of recording the speech.