Grouping devices for voice control

• US 10,031,722 B1
• Filed: 06/26/2015
• Issued: 07/24/2018
• Est. Priority Date: 03/17/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Mutagi, Rohan, Lu, He, Lew Yuk Vong, Willy, Whiteley, Michael Dale, Torok, Fred, Sitoke, Shikher, Bronaugh, David Ross, Li, Bo
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Kim, Jonathan
• Application Number:
  US14/752,198
• Time in Patent Office: 1,124 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00

Abstract

Techniques for creating groups of devices for controlling these groups with voice commands are described herein. For instance, an environment may include an array of secondary devices (or “smart appliances”, or simply “devices”) that are configured to perform an array of operations. Users may request to create different groups of these devices, such that the users may control entire groups at a single time with individual voice commands.

Claims

1. A method comprising:

determining that a first device, in an environment remote from at least one network-based device, has a first device capability corresponding to a first predefined device capability, wherein the at least one network-based device stores a first indication of a first association between the first predefined device capability and a first device driver;

storing, at the at least one network-based device, a second indication of a second association between the first device and the first predefined device capability;

determining that a second device in the environment has a second device capability corresponding to the first predefined device capability;

storing, at the at least one network-based device, a third indication of a third association between the second device and the first predefined device capability, wherein the at least one network-based device stores a fourth indication of a fourth association between one or more words and the first predefined device capability;

receiving, over a network and by the at least one network-based device, at least one audio signal, the at least one audio signal representing at least one voice command uttered within the environment;

performing, at the at least one network-based device, speech-recognition on the at least one audio signal to identify the at least one voice command, the at least one voice command including (i) a request to create a group of devices within the environment, (ii) the one or more words, and (iii) a name to associate with the group of devices;

based at least in part on the at least one voice command including the one or more words and on the fourth indication of the fourth association between the one or more words and the first predefined device capability, identifying the first device and the second device as being associated with the first predefined device capability;

receiving, at the at least one network-based device, at least one additional audio signal generated based on sound from the environment;

performing, at the at least one network-based device, speech recognition on the at least one additional audio signal to identify at least one additional voice command, the at least one additional voice command including a specified operation to be performed and the name, the specified operation comprising using the group of devices;

based at least in part on the first association between the first predefined device capability and the first device driver;

causing the first device driver, stored at the at least one network-based device and associated with the first device, to execute to generate a first command for the first device, the first command configured to, when executed by the first device, cause the first device to operate; and

causing the first device driver, stored at the at least one network-based device and associated with the second device, to execute to generate a second command for the second device, the second command configured to, when executed by the second device, cause the second device to operate.

2. The method as recited in claim 1, wherein the at least one additional audio signal was generated by a voice-controlled device in the environment, and further comprising:

sending the first command to the voice-controlled device for sending to the first device for causing the first device to operate; and

sending the second command to the voice-controlled device for sending to the second device for causing the second device to operate.

3. The method as recited in claim 1, wherein the group of devices comprises one or more of a light, a television, an audio system, a lock, a garage door opener, a washing machine, a dryer, a dishwasher, a coffee maker, a refrigerator, a door, or automated window shades.

4. The method as recited in claim 1, wherein the at least one voice command was uttered by a user, the method further comprising:

based at least in part on the identifying the at least one voice command including the request to create the group of devices, causing sound to be output into the environment requesting the user to indicate which devices are to be included in the group of devices;

receiving a third audio signal representing a third voice command; and

performing speech-recognition on the third audio signal to identify the third voice command, the third voice command including an indication of the first device and the second device.

5. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising;

determining that a first device, in an environment remote from at least one network-based device, has a first device capability corresponding to a first predefined device capability, wherein the at least one network-based device stores a first indication of a first association between the first predefined device capability and a first device driver;

storing a second indication of a second association between the first device and the first predefined device capability;

determining that a second device in the environment has a second device capability corresponding to the first predefined device capability;

storing a third indication of a third association between the second device and the first predefined device capability, wherein the at least one network-based device stores a fourth indication of a fourth association between one or more words and the first predefined device capability;

receiving at least one initial audio signal representing at least one voice command uttered by a sound source;

performing-speech recognition on the at least one initial audio signal to identify the at least one voice command, the at least one voice command including (i) a request to create a group of devices controllable by another voice command and (ii) the one or more words;

based at least in part on the at least one voice command including the one or more words and on the fourth indication of the fourth association between the one or more words and the first predefined device capability, identifying the first device and the second device as being associated with the first predefined device capability;

receiving at least one additional audio signal generated within the environment;

performing speech recognition on the at least one additional audio signal to identify at least one additional voice command, the at least one additional voice command requesting that a specified operation be performed by the group of devices;

based at least in part on the first association between the first predefined device capability and the first device driver;

causing the first device driver to execute to generate a first command configured to, when executed by the first device, cause the first device to perform the specified operation; and

causing the first device driver to execute to generate a second command configured to, when executed by the second device, cause the second device to perform the specified operation.

6. The system as recited in claim 5, further comprising:

sending the first command to one or more devices within the environment for causing the first device to perform the specified operation; and

sending the second command to the one or more devices within the environment for causing the second device to perform the specified operation.

7. The system as recited in claim 6, wherein the at least one audio signal is received from a voice-controlled device within the environment, and the sending comprises sending each respective command to the voice-controlled device for sending to each of the devices of the group of devices.

8. The system as recited in claim 5, wherein the one or more words included in the request includes an indication of the first predefined device capability, and the acts further comprise identifying devices to associate with the group of devices based on the devices being associated with the first predefined device capability.

9. The system as recited in claim 8, wherein the first predefined device capability includes turning on or off, changing a volume, changing a channel, or changing a brightness.

10. The system as recited in claim 5, wherein devices associated with the group of devices comprise one or more of a light, a television, an audio system, a lock, a garage door opener, a washing machine, a dryer, a dishwasher, a coffee maker, a refrigerator, a door, or automated window shades.

11. The system as recited in claim 5, wherein the sound source comprises a user, and the acts further comprising determining that the user resides in the environment of a set of environments, the determining including at least one of:

determining that the at least one initial audio signal was generated based at least in part on sound captured in the environment of the set of environments;

or

analyzing motion data received from a motion sensor to determine that the user resides in the environment of the set of environments.

12. The system as recited in claim 5, wherein the performing the speech-recognition on the at least one additional audio signal to identify the at least one additional voice command comprises performing speech-recognition on the at least one additional audio signal to identify an indication of the first predefined device capability associated with the specific operation to be performed.