Voice action biasing system

• US 10,089,982 B2
• Filed: 06/08/2017
• Issued: 10/02/2018
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Wang, Bo, Vemuri, Sunil, James, Barnaby John, Gupta, Pravir Kumar, Huffman, Scott B.
• Primary Examiner(s): Abebe, Daniel
• Application Number:
  US15/617,496
• Publication Number:
  US 20180053507A1
• Time in Patent Office: 481 Days
• Field of Search: 704275
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Methods, systems, and apparatus for determining that a software application installed on a user device is compatible with a new voice action, wherein the new voice action is specified by an application developer of the software application. One or more trigger terms for triggering the software application to perform the new voice action are identified. An automatic speech recognizer is biased to prefer the identified trigger terms of the new voice action over trigger terms of other voice actions. A transcription of an utterance generated by the biased automatic speech recognizer is obtained. The transcription of the utterance generated by the biased automatic speech recognizer is determined to include a particular trigger term included in the identified trigger terms. Based at least on determining that the transcription of the utterance generated by the biased automatic speech recognizer includes the particular trigger term, execution of the new voice action is triggered.

Claims

1. A computer-implemented method comprising:

receiving a user utterance for a new voice action, the user utterance received in a voice action system that is separate from and communicatively coupled to a user device, the user device having an application installed which is associated with trigger terms for performing voice actions;

determining, using the voice action system, multiple candidate transcriptions of the user utterance;

determining, using the voice action system, whether any of the multiple candidate transcriptions contains one or more of the trigger terms; and

biasing the voice action system based on the determination, such that the voice action system favors one of the multiple candidate transcriptions that is associated with the new voice action, wherein upon the voice action system receiving the user utterance from the user device, the voice action system sends an action trigger to the user device, the action trigger causing the application to perform the new voice action.

2. The computer-implemented method of claim 1, wherein biasing the voice action system to favor the one of the multiple candidate transcriptions comprises adjusting a language model used by the voice action system such that the voice action system when using the adjusted language model has an increased likelihood of detecting trigger terms of the new voice action.

3. The computer-implemented method of claim 1, wherein causing the application to perform the new voice action comprises triggering the application to perform an action associated with the new voice action.

4. The computer-implemented method of claim 1, comprising:

determining, by the voice action system, that the application installed on the user device is compatible with one or more other voice actions;

identifying, by the voice action system, one or more trigger terms for triggering the application to perform the one or more other voice actions; and

ranking, by the voice action system, the new voice action and the one or more other voice actions, wherein biasing the voice action system is done based at least on the ranking.

5. The computer-implemented method of claim 4, comprising:

receiving information indicating a status of the user device or of the application installed on the user device; and

ranking the new voice action and the one or more other voice actions based at least on the received information indicating the status of the user device or of the application installed on the user device.

6. The computer-implemented method of claim 1, wherein the voice action system generates, based on the received user utterance, a graph comprising nodes and edges between some of the nodes, and wherein the multiple candidate transcriptions are determined, and the voice action system is biased, using the generated graph.

7. The computer-implemented method of claim 6, wherein the nodes correspond to connections between terms, wherein the edges correspond to a candidate term that corresponds to a portion of audio data from the user utterance, and wherein the nodes or the edges, or both, are associated with probabilities indicating a determined confidence that the user utterance includes a particular term or connection between terms corresponding to the nodes or edges.

8. The computer-implemented method of claim 7, further comprising ranking the determined multiple candidate transcriptions, wherein biasing the voice action system comprises changing the ranking to prefer the candidate transcription associated with the new voice action.

9. The computer-implemented method of claim 7, wherein biasing the voice action system comprises adjusting probabilities of the respective candidate transcriptions based on a presence or absence therein of the trigger terms corresponding to eligible voice actions.

10. The computer-implemented method of claim 9, wherein adjusting the probabilities is done for individual nodes or edges.

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations, the system being separate from and communicatively coupled to a user device, the user device having an application installed which is associated with trigger terms for performing voice actions, the operations comprising;

receiving a user utterance for a new voice action;

determining multiple candidate transcriptions of the user utterance;

determining whether any of the multiple candidate transcriptions contains one or more of the trigger terms; and

biasing the system based on the determination, such that the system favors one of the multiple candidate transcriptions that is associated with the new voice action, wherein upon the system receiving the user utterance from the user device, the system sends an action trigger to the user device, the action trigger causing the application to perform the new voice action.

12. The system of claim 11, wherein the system is communicatively coupled to multiple user devices, wherein the user utterance is received from one of the multiple user devices, and wherein the system sends the action trigger to the user device of the multiple user devices from which the user utterance was received.

13. The system of claim 11, further comprising a voice action selector, a first database of application voice actions, and a second database of operating system voice actions.

14. The system of claim 13, further comprising a biasing engine and a speech recognition engine, wherein the biasing engine is configured to provide biasing information to the speech recognition engine.

15. The system of claim 11, wherein the operations further include generating, based on the received user utterance, a graph comprising nodes and edges between some of the nodes, and wherein the multiple candidate transcriptions are determined, and the system is biased, using the generated graph, and wherein the nodes correspond to connections between terms, wherein the edges correspond to a candidate term that corresponds to a portion of audio data from the user utterance, and wherein the nodes or the edges, or both, are associated with probabilities indicating a determined confidence that the user utterance includes a particular term or connection between terms corresponding to the nodes or edges.

16. The system of claim 15, wherein the operations further include ranking the determined multiple candidate transcriptions, wherein biasing the system comprises changing the ranking to prefer the candidate transcription associated with the new voice action.

17. The system of claim 11, wherein biasing the system comprises adjusting probabilities of the respective candidate transcriptions based on a presence or absence therein of the trigger terms corresponding to eligible voice actions.

18. The system of claim 17, wherein adjusting the probabilities is done for individual nodes or edges.

19. A non-transitory computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving a user utterance for a new voice action, the user utterance received in a voice action system that is separate from and communicatively coupled to a user device, the user device having an application installed which is associated with trigger terms for performing voice actions;

determining, using the voice action system, multiple candidate transcriptions of the user utterance;

determining, using the voice action system, whether any of the multiple candidate transcriptions contains one or more of the trigger terms; and

biasing the voice action system based on the determination, such that the voice action system favors one of the multiple candidate transcriptions that is associated with the new voice action, wherein upon the voice action system receiving the user utterance from the user device, the voice action system sends an action trigger to the user device, the action trigger causing the application to perform the new voice action.

20. The non-transitory computer-readable storage device of claim 19, wherein the voice action system is communicatively coupled to multiple user devices, wherein the user utterance is received from one of the multiple user devices, and wherein the voice action system sends the action trigger to the user device of the multiple user devices from which the user utterance was received.