TASK INITIATION USING LONG-TAIL VOICE COMMANDS

• US 20180096681A1
• Filed: 10/03/2016
• Published: 04/05/2018
• Est. Priority Date: 10/03/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ni, Yuzhao, Wang, Bo, James, Barnaby, Gupta, Pravir, Schairer, David
• Granted Patent: US 10,297,254 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

In various implementations, upon receiving a given voice command from a user, a voice-based trigger may be selected from a library of voice-based triggers previously used across a population of users. The library may include association(s) between each voice-based trigger and responsive action(s) previously performed in response to the voice-based trigger. The selecting may be based on a measure of similarity between the given voice command and the selected voice-based trigger. One or more responsive actions associated with the selected voice-based trigger in the library may be determined. Based on the one or more responsive actions, current responsive action(s) may be performed by the client device. Feedback associated with performance of the current responsive action(s) may be received from the user and used to alter a strength of an association between the selected voice-based trigger and the one or more responsive actions.

Claims

1. A method comprising:

receiving, at a client device, a given voice command from a user;

selecting, from a library of voice-based triggers previously used across a population of users, a selected voice-based trigger, wherein the library includes one or more associations between each given voice-based trigger of the library of voice-based triggers and one or more responsive actions previously invoked automatically by one or more other client devices operated by one or more other users of the population of users in response to the given voice-based trigger, and wherein the selecting is based at least in part on a measure of similarity between the given voice command and the selected voice-based trigger;

determining one or more responsive actions associated with the selected voice-based trigger in the library;

selecting, by the client device, one or more current responsive actions to be performed by the client device based on the one or more responsive actions;

invoking, by the client device, the one or more current responsive actions;

receiving, by the client device from the user, feedback associated with the client device invoking the one or more current responsive actions; and

altering a strength of an association between the selected voice-based trigger and the one or more responsive actions based on the feedback;

wherein the selected voice-based trigger is selected from the library of voice-based triggers further based on a context of the user detected using one or more sensors of the client device.

2. The method of claim 1, wherein the one or more current responsive actions are the same as the one or more responsive actions.

3. The method of claim 1, wherein the selected voice-based trigger includes one or more terms or tokens that are analogous to one or more terms or tokens in the given voice command.

4. The method of claim 1, further comprising analyzing a plurality of distinct applications installed on the client device to identify one or more applications to perform the one or more current responsive actions.

5. The method of claim 1, wherein the altering comprises weakening the association between the selected voice-based trigger and the one or more responsive actions based on the feedback.

6. The method of claim 1, wherein the altering comprises strengthening the association between the selected voice-based trigger and the one or more responsive actions based on the feedback.

7. The method of claim 1, wherein the measure of similarity is a measure of syntactic similarity.

8. The method of claim 1, wherein the measure of similarity is a measure of semantic similarity.

10. The method of claim 1, wherein determining the one or more responsive actions associated with the selected voice-based trigger in the library comprises selecting at least one responsive action from a plurality of candidate responsive actions associated with the selected voice-based trigger in the library.

11. The method of claim 10, wherein the at least one responsive action is selected based on the context of the user.

12. The method of claim 1, wherein the given voice command fails to specify at least one parameter required by the one or more current responsive actions, and the method further includes identifying, based on a context of the user detected using one or more sensors of the client device, a value to use as the at least one parameter.

13. The method of claim 1, wherein the given voice command fails to specify at least one parameter required by the one or more current responsive actions, and the method further includes identifying, based on a personal library associated with the user, a value to use as the at least one parameter.

9. (canceled)

14. A system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to operate an interactive assistant module configured to:

receive a given voice command from a user;

select, from a library of voice-based triggers previously used across a population of users, a selected voice-based trigger, wherein the library includes one or more associations between each given voice-based trigger of the library of voice-based triggers and one or more responsive actions previously invoked automatically by one or more other client devices operated by one or more other users of the population of users in response to the given voice-based trigger, and wherein the selecting is based at least in part on a measure of similarity between the given voice command and the selected voice-based trigger;

determine one or more responsive actions associated with the selected voice-based trigger in the library;

select one or more current responsive actions to be invoked by the one or more processors based on the one or more responsive actions;

invoke the one or more current responsive actions;

receive, from the user, feedback associated with the one or more processors invoking the one or more current responsive actions; and

alter a strength of an association between the selected voice-based trigger and the one or more responsive actions based on the feedback;

wherein the selected voice-based trigger is selected from the library of voice-based triggers further based on a context of the user detected using one or more sensors of the system.

15. The system of claim 14, wherein the one or more current responsive actions are the same as the one or more responsive actions.

16. The system of claim 14, wherein the one or more current responsive actions are analogous to the one or more responsive actions.

17. The system of claim 14, further comprising analyzing a plurality of applications installed on the system to identify one or more applications to perform the one or more current responsive actions.

18. The system of claim 14, wherein the interactive assistant module is further configured to weaken the association between the selected voice-based trigger and the one or more responsive actions based on the feedback.

19. The system of claim 14, wherein the given voice command fails to specify at least one parameter required by the one or more current responsive actions, and the method further includes identifying, based on a context of the user detected using one or more sensors of the system or based on a personal library associated with the user, a value to use as the at least one parameter.

20. A method comprising:

receiving, from a population of users, a plurality of voice-based triggers and responsive actions invoked automatically at client devices operated by users of the population of users in response to the voice-based triggers;

storing associations between the voice-based triggers and the responsive actions in a library, wherein the associations are endorsed by the population of users;

receiving a new voice command provided by a given user at a given client device;

selecting, from the library of voice-based triggers, a selected voice-based trigger, wherein the selecting is based at least in part on a measure of similarity between the new voice command and the selected voice-based trigger and on a context of the given user detected using one or more sensors of the given client device;

determining one or more responsive actions associated with the selected voice-based trigger in the library;

selecting one or more current responsive actions to be invoked by the given client device based on the determined one or more responsive actions;

receiving, by the client device from the user, feedback associated with the given client device invoking the one or more current responsive actions; and

altering a strength of an association between the selected voice-based trigger and the one or more determined responsive actions based on the feedback.