VOICE COMMANDS ACROSS DEVICES

• US 20170249940A1
• Filed: 05/15/2017
• Published: 08/31/2017
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: CHEN, Jennifer Shien-Ming, KUSCHER, Alexander Friedrich, OSHIMA, Mitsuru
• Granted Patent: US 10,714,083 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 12/2832 Interconnection of the control functionalities between home networks single bridge functionality H04L12/4625
• CPC Class Codes: H04L 12/4625 Single bridge functionality, e.g. connection of two networks over a single bridge
• CPC Class Codes: H04L 67/306 User profiles

Abstract

Aspects of the subject technology relate to a method for using a voice command for multiple computing devices. First voice input data is received from a first computing device associated with a user account, where the first voice input data comprises a first voice command captured at the first computing device. Second voice input data is received from a second computing device associated with the user account where the second voice input data comprises a second voice command captured at the second computing device. An intended voice command is determined based on the obtained first and second voice input data. Based on the intended voice command, a first target computing device is determined. First instructions associated with the intended voice command are provided to the first target computing device for execution.

Claims

1. A computer-implemented method, comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first voice command captured at the first computing device;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second voice command captured at the second computing device;

determining an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

2. The method of claim 1, wherein the first voice input data further comprises a first timestamp associated with the first voice command, wherein the second voice input data further comprises a second timestamp associated with the second voice command.

3. The method of claim 2, further comprising:

determining that the first voice command comprises a first part of the intended voice command and the second voice command comprises a second part of the intended voice command based on a comparison of the first timestamp and the second timestamp, and a comparison of a first location of the first computing device and a second location of the second computing device, wherein determining the intended voice command further comprises;

merging the first part of the intended voice command and the second part of the intended voice command based on the first timestamp and the second timestamp.

4. The method of claim 2, further comprising:

determining that each of the first and second voice commands comprises the intended voice command based on a first comparison of the first timestamp and the second timestamp and a second comparison of a first location of the first computing device and a second location of the second computing device.

5. The method of claim 4, wherein the first voice input data further comprises a first recognized word segment and a first confidence value, wherein the second voice input data further comprises a second recognized word segment and a second confidence value, and wherein determining the intended voice command comprises:

determining that the first recognized word segment is different from the second recognized word segment; and

selecting one of the first recognized word segment or the second recognized word segment based on a comparison of the first confidence value and the second confidence value, wherein the intended voice command comprises the selected one of the first recognized word segment or the second recognized word segment.

6. The method of claim 4, wherein the first voice input data further comprises a first plurality of recognized word segments and their respective first confidence values and the second voice input data further comprises a second plurality recognized word segments and their respective second confidence values, and wherein determining the intended voice command further comprises:

merging the first plurality of recognized word segments and the second plurality of recognized word segments based on the respective first confidence values and the respective second confidence values, wherein the intended voice command comprises the merged first plurality of recognized word segments and second plurality of recognized word segments.

7. The method of claim 1, wherein the intended voice command comprises a device identifier associated with the first target computing device.

8. The method of claim 1, wherein the first voice input data further comprises a first sound level associated with the first voice command, wherein the second voice input data further comprises a second sound level associated with the second voice command, and wherein determining the first target computing device is further based on a comparison of the first sound level and the second sound level.

9. The method of claim 1, wherein determining the first target computing device is further based on a first comparison of commands available to the user on the first computing device and the intended voice command and a second comparison of commands available to the user on the second computing device and the intended voice command.

10. The method of claim 1, further comprising:

determining a second target computing device based on the intended voice command; and

providing second instructions associated with the intended voice command to the second target computing device for execution.

11. The method of claim 1, wherein determining the first target computing device is further based on a first location of the first computing device and a second location of the second computing device.

12. A system, comprising:

one or more processors; and

a non-transitory computer-readable medium comprising instructions stored therein, which when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first voice command captured at the first computing device and a first timestamp associated with the first voice command;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second voice command captured at the second computing device and a second timestamp associated with the second voice command;

determining an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

13. The system of claim 12, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations further comprising:

determining that the first voice command comprises a first part of the intended voice command and the second voice command comprises a second part of the intended voice command based on a comparison of the first timestamp and the second timestamp, and a comparison of a first location of the first computing device and a second location of the second computing device, wherein determining the intended voice command further comprises;

merging the first part of the intended voice command and the second part of the intended voice command based on the first timestamp and the second timestamp.

14. The system of claim 12, further comprising:

determining that the first and second voice commands comprise the intended voice command based on a first comparison of the first timestamp and the second timestamp and a second comparison of a first location of the first computing device and a second location of the second computing device.

15. The system of claim 14, wherein the first voice input data further comprises a first recognized word segment and a first confidence value, wherein the second voice input data further comprises a second recognized word segment and a second confidence value, and wherein determining the intended voice command comprises:

determining that the first recognized word segment is different from the second recognized word segment; and

selecting one of the first recognized word segment or the second recognized word segment based on a comparison of the first confidence value and the second confidence value, wherein the intended voice command comprises the selected one of the first recognized word segment or the second recognized word segment.

16. The system of claim 12, wherein the first voice input data further comprises a first sound level associated with the first voice command, wherein the second voice input data further comprises a second sound level associated with the second voice command, and wherein determining the first target computing device is further based on a comparison of the first sound level and the second sound level.

17. The system of claim 12, wherein determining the first target computing device is further based on a first comparison of commands available to a user on the first computing device and the intended voice command and a second comparison of commands available to the user on the second computing device and the intended voice command.

18. A non-transitory machine-readable medium comprising instructions stored therein, which when executed by a processor, cause the processor to perform operations comprising:

receiving first voice input data from a first computing device associated with a plurality of user accounts;

determining a first intended voice command associated with a first user account of the plurality of user accounts and a second intended voice command associated with a second user account of the plurality of user accounts based on the first voice input data;

determining a first target computing device based on the first intended voice command;

determining a second target computing device based on the second intended voice command;

providing first instructions associated with the first intended voice command to the first target computing device for execution; and

providing second instructions associated with the second intended voice command to the second target computing device for execution.

19. The non-transitory machine-readable medium of claim 18, wherein the operations further comprise:

receiving second voice input data from the first computing device, the second voice input data comprising a third voice command captured at the first computing device;

determining a third intended voice command associated with the first user account;

determining that there is a conflict between the second intended voice command and the third intended voice command; and

determining that the second user account has a higher priority than the first user account at the second target computing device,

wherein the providing second instructions to the second target computing device for execution is provided in response to the determination that the second user account has the higher priority than the first user account.

20. The non-transitory machine-readable medium of claim 18, wherein the operations further comprise:

receiving second voice input data from the first computing device, the second voice input data comprising a third voice command captured at the first computing device;

determining a third intended voice command associated with the first user account;

determining that the first intended voice command is associated with the third intended voice command;

determining a third target computing device based on the first intended voice command and the third intended voice command; and

providing third instructions associated with the third intended voice command to the third target computing device for execution.