Voice commands across devices

• US 9,653,075 B1
• Filed: 11/06/2015
• Issued: 05/16/2017
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Oshima, Mitsuru, Kuscher, Alexander Friedrich, Chen, Jennifer Shien-Ming
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US14/935,350
• Publication Number:
  US 20170133011A1
• Time in Patent Office: 557 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 12/2832 Interconnection of the control functionalities between home networks single bridge functionality H04L12/4625
• CPC Class Codes: H04L 12/4625 Single bridge functionality, e.g. connection of two networks over a single bridge
• CPC Class Codes: H04L 67/306 User profiles

Abstract

Aspects of the subject technology relate to a method for using a voice command for multiple computing devices. First voice input data is received from a first computing device associated with a user account, where the first voice input data comprises a first voice command captured at the first computing device. Second voice input data is received from a second computing device associated with the user account where the second voice input data comprises a second voice command captured at the second computing device. An intended voice command is determined based on the obtained first and second voice input data. Based on the intended voice command, a first target computing device is determined. First instructions associated with the intended voice command are provided to the first target computing device for execution.

Claims

1. A computer-implemented method, comprising:

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first recognized word segment and a first confidence value corresponding to the first recognized word segment, the first recognized word segment obtained from a first voice command associated with the first voice input data;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second recognized word segment and a second confidence value corresponding to the second recognized word segment, the second recognized word segment obtained from a second voice command associated with second voice input data;

determining that the first recognized word segment is different from the second recognized word segment;

selecting one of the first recognized word segment or the second recognized word segment based on a comparison of the first confidence value and the second confidence value;

determining, from the selected one of the first recognized word segment or the second recognized word segment, an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

2. The method of claim 1, wherein the first voice input data further comprises a first timestamp associated with the first voice command, wherein the second voice input data further comprises a second timestamp associated with the second voice command.

3. The method of claim 2, wherein the first voice input data further comprises a first location of the first computing device, and the second voice input data further comprises a second location of the second computing device, and the method further comprising:

determining the intended voice command further based on a first comparison of the first timestamp and the second timestamp and a second comparison of the first location of the first computing device and the second location of the second computing device.

4. The method of claim 3, wherein the first voice input data further comprises a first plurality of recognized word segments obtained from the first voice command, and their respective first confidence values and the second voice input data further comprises a second plurality recognized word segments obtained from the second voice command, and their respective second confidence values, and wherein determining the intended voice command further comprises:

merging the first plurality of recognized word segments and the second plurality of recognized word segments based on the respective first confidence values and the respective second confidence values, wherein the intended voice command comprises the merged first plurality of recognized word segments and second plurality of recognized word segments.

5. The method of claim 1, wherein the intended voice command comprises a device identifier associated with the first target computing device.

6. The method of claim 1, wherein the first voice input data further comprises a first sound level associated with the first voice command, wherein the second voice input data further comprises a second sound level associated with the second voice command, and wherein determining the first target computing device is further based on a comparison of the first sound level and the second sound level.

7. The method of claim 1, wherein determining the first target computing device is further based on a first comparison of commands available to the user on the first computing device and the intended voice command and a second comparison of commands available to the user on the second computing device and the intended voice command.

8. The method of claim 1, further comprising:

determining a second target computing device based on the intended voice command; and

providing second instructions associated with the intended voice command to the second target computing device for execution.

9. The method of claim 1, wherein determining the first target computing device is further based on a first location of the first computing device and a second location of the second computing device.

10. A system, comprising:

one or more processors; and

a non-transitory computer-readable medium comprising instructions stored therein, which when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving first voice input data from a first computing device associated with a user account, the first voice input data comprising a first recognized word segment, a first confidence value corresponding to the first recognized word segment, and a first timestamp associated with a first voice command, the first recognized word segment obtained from the first voice command that is associated with first voice input data;

receiving second voice input data from a second computing device associated with the user account, the second voice input data comprising a second recognized word segment, a second confidence value corresponding to the second recognized word segment, and a second timestamp associated with a second voice command, the second recognized word segment obtained from the second voice command that is associated with second voice input data;

determining that the first recognized word segment is different from the second recognized word segment;

selecting one of the first recognized word segment or the second recognized word segment based on a comparison of the first confidence value and the second confidence value;

determining, from the selected one of the first recognized word segment or the second recognized word segment, an intended voice command based on the obtained first and second voice input data;

determining a first target computing device based on the intended voice command; and

providing first instructions associated with the intended voice command to the first target computing device for execution.

11. The system of claim 10, further comprising:

determining the intended voice command further based on a first comparison of the first timestamp and the second timestamp and a second comparison of a first location of the first computing device and a second location of the second computing device.

12. The system of claim 10, wherein the first voice input data further comprises a first sound level associated with the first voice command, wherein the second voice input data further comprises a second sound level associated with the second voice command, and wherein determining the first target computing device is further based on a comparison of the first sound level and the second sound level.

13. The system of claim 10, wherein determining the first target computing device is further based on a first comparison of commands available to a user on the first computing device and the intended voice command and a second comparison of commands available to the user on the second computing device and the intended voice command.