Voice commands across devices

• US 10,714,083 B2
• Filed: 05/15/2017
• Issued: 07/14/2020
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Chen, Jennifer Shien-Ming, Kuscher, Alexander Friedrich, Oshima, Mitsuru
• Primary Examiner(s): Baker, Matthew H
• Application Number:
  US15/595,802
• Publication Number:
  US 20170249940A1
• Time in Patent Office: 1,156 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 12/2832 Interconnection of the control functionalities between home networks single bridge functionality H04L12/4625
• CPC Class Codes: H04L 12/4625 Single bridge functionality, e.g. connection of two networks over a single bridge
• CPC Class Codes: H04L 67/306 User profiles

Abstract

Aspects of the subject technology relate to a method for using a voice command for multiple computing devices. First voice input data is received from a first computing device associated with a user account, where the first voice input data comprises a first voice command captured at the first computing device. Second voice input data is received from a second computing device associated with the user account where the second voice input data comprises a second voice command captured at the second computing device. An intended voice command is determined based on the obtained first and second voice input data. Based on the intended voice command, a first target computing device is determined. First instructions associated with the intended voice command are provided to the first target computing device for execution.

Claims

1. A computer-implemented method, the method comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first voice command captured at the first computing device;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second voice command captured at the second computing device;

determining an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution,

wherein determining the first target computing device is further based on a first comparison of commands available to the user on the first computing device and the intended voice command and a second comparison of commands available to the user on the second computing device and the intended voice command.

2. The method of claim 1, wherein the intended voice command comprises a device identifier associated with the first target computing device.

3. The method of claim 1, wherein determining the first target computing device is further based on a first location of the first computing device and a second location of the second computing device.

4. A computer-implemented method, the method comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising;

a first voice command captured at the first computing device; and

a first timestamp associated with the first voice command;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising;

a second voice command captured at the second computing device; and

a second timestamp associated with the second voice command;

determining that the first voice command comprises a first part of the intended voice command and the second voice command comprises a second part of the intended voice command based on a comparison of the first timestamp and the second timestamp, and a comparison of a first location of the first computing device and a second location of the second computing device;

determining an intended voice command based on the obtained first and second voice input data by merging the first part of the intended voice command and the second part of the intended voice command based on the first timestamp and the second timestamp;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

5. A computer-implemented method, the method comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising;

a first voice command captured at the first computing device;

a first timestamp associated with the first voice command;

a first recognized word segment; and

a first confidence value;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising;

a second voice command captured at the second computing device;

a second timestamp associated with the second voice command;

a second recognized word segment; and

a second confidence value;

determining an intended voice command based on the obtained first and second voice input data by;

determining that the first recognized word segment is different from the second recognized word segment; and

selecting one of the first recognized word segment or the second recognized word segment based on a comparison of the first confidence value and the second confidence value, wherein the intended voice command comprises the selected one of the first recognized word segment or the second recognized word segment;

determining that each of the first and second voice commands comprises the intended voice command based on a first comparison of the first timestamp and the second timestamp and a second comparison of a first location of the first computing device and a second location of the second computing device;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

6. A computer-implemented method, the method comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising;

a first voice command captured at the first computing device;

a first timestamp associated with the first voice command; and

a first plurality of recognized word segments and their respective first confidence values;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising;

a second voice command captured at the second computing device;

a second timestamp associated with the second voice command; and

a second plurality of recognized word segments and their respective first confidence values;

determining an intended voice command based on the obtained first and second voice input data by merging the first plurality of recognized word segments and the second plurality of recognized word segments based on the respective first confidence values and the respective second confidence values, wherein the intended voice command comprises the merged first plurality of recognized word segments and second plurality of recognized word segments;

determining that each of the first and second voice commands comprises the intended voice command based on a first comparison of the first timestamp and the second timestamp and a second comparison of a first location of the first computing device and a second location of the second computing device;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

7. The method of claim 6, wherein the first voice input data further comprises a first sound level associated with the first voice command, wherein the second voice input data further comprises a second sound level associated with the second voice command, and wherein determining the first target computing device is further based on a comparison of the first sound level and the second sound level.

8. A computer-implemented method, the method comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first voice command captured at the first computing device;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second voice command captured at the second computing device;

determining an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on the intended voice command;

providing first instructions associated with the intended voice command to the first target computing device for execution;

determining a second target computing device based on the intended voice command; and

providing second instructions associated with the intended voice command to the second target computing device for execution.

9. A system comprising:

one or more processors; and

a non-transitory computer-readable medium comprising instructions stored therein, which when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first voice command captured at the first computing device and a first timestamp associated with the first voice command;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second voice command captured at the second computing device and a second timestamp associated with the second voice command;

determining an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on a first comparison of commands available to a user on the first computing device and the intended voice command and a second comparison of commands available to the user on the second computing device and the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

10. The system of claim 9, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations further comprising:

determining that the first voice command comprises a first part of the intended voice command and the second voice command comprises a second part of the intended voice command based on a comparison of the first timestamp and the second timestamp, and a comparison of a first location of the first computing device and a second location of the second computing device, wherein determining the intended voice command further comprises;

merging the first part of the intended voice command and the second part of the intended voice command based on the first timestamp and the second timestamp.

11. A system comprising:

one or more processors; and

a non-transitory computer-readable medium comprising instructions stored therein, which when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising;

a first voice command captured at the first computing device;

a first timestamp associated with the first voice command;

a first recognized word segment; and

a first confidence value;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising;

a second voice command captured at the second computing device;

a second timestamp associated with the second voice command;

a second recognized word segment; and

a second confidence value;

determining an intended voice command based on the obtained first and second voice input data by;

determining that the first recognized word segment is different from the second recognized word segment; and

selecting one of the first recognized word segment or the second recognized word segment based on a comparison of the first confidence value and the second confidence value, wherein the intended voice command comprises the selected one of the first recognized word segment or the second recognized word segment;

determining that each of the first and second voice commands comprises the intended voice command based on a first comparison of the first timestamp and the second timestamp and a second comparison of a first location of the first computing device and a second location of the second computing device;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

12. The system of claim 11, wherein the first voice input data further comprises a first sound level associated with the first voice command, wherein the second voice input data further comprises a second sound level associated with the second voice command, and wherein determining the first target computing device is further based on a comparison of the first sound level and the second sound level.