Mapping device capabilities to a predefined set

• US 9,984,686 B1
• Filed: 06/26/2015
• Issued: 05/29/2018
• Est. Priority Date: 03/17/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Mutagi, Rohan, Lew Yuk Vong, Willy, Whiteley, Michael Dale, Lu, He
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): Kim, Jonathan
• Application Number:
  US14/752,257
• Time in Patent Office: 1,068 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00

Abstract

Techniques for defining a set of predefined device capabilities generally offered by available voice-controllable devices are described herein. Thereafter, as a particular user introduces new secondary devices into his environment and registers these devices, the techniques may identify the capabilities of the new device and map these capabilities to one or more of the predefined device capabilities of the set.

Claims

1. A method implemented at least in part by a speech-recognition service, the method comprising:

receiving a first indication that a first device is in an environment;

based at least in part on receiving the first indication;

identifying a first capability associated with the first device;

determining that the first capability corresponds to a first predefined device capability included within a predefined set of predefined device capabilities;

generating a first association between the first capability associated with the first device and the first predefined capability; and

storing the first association between the first capability associated with the first device and the first predefined device capability;

identifying a second capability associated with a second device;

determining that the second capability corresponds to the first predefined device capability of the set of predefined device capabilities;

generating a second association between the second capability associated with the second device and the first predefined capability;

storing the second association between the second capability associated with the second device and the first predefined device capability;

storing, based at least in part on the first association, a third indication of a device driver associated with the first device, the device driver being configured to, when executed, generate a first command to cause the first device to perform the first predefined device capability;

receiving an audio signal from the environment in which the first device resides;

performing speech-recognition on the audio signal;

determining, based at least in part on performing the speech-recognition on the audio signal, that the audio signal includes a request to associate a word or phrase with at least the first device and the second device performing an operation associated with the first predefined device capability; and

associating, the word or phrase with the first device and the second device performing the operation.

2. The method as recited in claim 1, further comprising:

storing, based at least in part on the second association, a fourth indication of a second device driver associated with the second device, the second device driver being configured to, when executed, generate a second command to cause the second device to perform the second predefined device capability.

3. The method as recited in claim 1, wherein the audio signal comprises a first audio signal, further comprising:

receiving a second audio signal from the environment in which the second device resides;

performing speech-recognition on the second audio signal;

determining, based at least in part on performing the speech-recognition on the second audio signal, that the second audio signal includes the word or phrase;

causing the device driver to execute to generate the command; and

sending the command to a device in the environment in which the first device resides.

5. The system as recited in claim 1, the acts further comprising storing one or more fifth indications of additional voice commands to cause the first device to perform the first predefined device capability, each of the additional voice commands being unique relative to one another and relative to the first voice command.

4. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising;

receiving a first indication that a first device is in an environment;

identifying a first capability of the first device;

generating a first association between the first capability of the first device and a first predefined capability included within predefined device capabilities;

storing the first association between the first capability and the first predefined device capability;

storing a second indication of a first voice command to cause the first device to perform the first predefined device capability;

determining that a second capability associated with a second device is configured to perform the first predefined device capability;

generating a second association between the second capability of the second device and the first predefined capability;

storing the second association between the second capability and the first predefined device capability;

storing a third indication of a second voice command to cause the second device to perform the first predefined device capability, the second voice command being different from the first voice command; and

storing, based at least in part on the first association, a fourth indication of a first device driver associated with the first device, the first device driver being configured to, when executed by the one or more processors, generate a first command to cause the first device to perform the first predefined device capability.

6. The system as recited in claim 4, the acts further comprising:

storing a fifth indication of a second device driver, associated with the second device, configured to generate a second command to cause the second device to perform the first predefined device capability.

7. The system as recited in claim 6, the acts further comprising:

receiving a second audio signal generated in the environment in which the second device resides;

identifying, from the second audio signal, the second voice command;

sending information indicative of the second voice command to the second device driver; and

receiving, from the second device driver, the second command to cause the second device to perform the first predefined device capability.

8. The system as recited in claim 4, the acts further comprising:

receiving a first audio signal generated in the environment in which the first device resides;

identifying, from the first audio signal, the first voice command;

sending information indicative of the first voice command to the first device driver; and

receiving, from the first device driver, the first command to cause the first device to perform the first predefined device capability.

9. The system as recited in claim 4, wherein the first predefined device capability comprises turning on or off, changing a volume, changing a channel, changing a brightness, changing a color, and changing a temperature.

10. The system as recited in claim 4, wherein the first device comprises a light, a television, an audio system, a lock, a garage door, a washing machine, a dryer, a dishwasher, a coffee maker, a refrigerator, a door, or shades.

11. The system as recited in claim 4, the acts further comprising:

receiving a first audio signal generated in the environment in which the first device resides;

performing-speech recognition on the audio signal;

determining, based at least in part on performing-speech recognition on the audio signal, that the audio signal includes the first voice command;

based at least in part on determining that the audio signal includes the voice command, the second indication of the first voice command to cause the first device to perform the first predefined device capability, and the fourth indication of the first device driver configured to generate the first command to cause the first device to perform the first predefined device capability, causing the first device driver to execute to generate a command that, when executed by the first device, causes the first device to perform the first predefined device capability; and

sending the command to at least one of the first device or a device in the environment of the first device.

12. A method comprising:

receiving a first indication that a first device is in an environment;

based at least in part on receiving the first indication;

identifying one or more capabilities of the first device;

analyzing predefined device capabilities and the one or more capabilities of the first device to determine that the first device is configured to perform;

(i) a first predefined device capability of the predefined device capabilities, and (ii) a second predefined device capability of the predefined device capabilities;

generating a first association between the first capability of the first device and the first predefined capability;

generating a second association between the second capability of the first device and the second predefined capability; and

storing the first association and the second association;

storing one or more second indications of;

(i) a first voice command that, when uttered within the environment, is to result in the first device performing the first predefined device capability, and (ii) a second voice command that, when uttered within the environment, is to result in the first device performing the second predefined device capability; and

storing, based at least in part on the first association, an association between the first device and a device driver, the device driver being configured to generate a first command that, when executed by the first device, causes the first device to perform the first predefined device capability.

13. The method as recited in claim 12, further comprising storing an association between the first device and the device driver configured to generate a second command to cause the first device to perform the second predefined device capability.

14. The method as recited in claim 12, wherein the first device comprises a light, a television, an audio system, a lock, a garage door, a washing machine, a dryer, a dishwasher, a coffee maker, a refrigerator, a door, or shades.

15. The method as recited in claim 12, further comprising:

receiving an audio signal;

performing speech-recognition on the audio signal to identify the first voice command; and

causing the device driver to execute to generate the command to cause the first device to perform the first predefined device capability.

16. The method as recited in claim 15, wherein the audio signal comprises a first audio signal and the command comprises a first command, and further comprising:

receiving a second audio signal;

performing speech-recognition on the second audio signal to identify the second voice command; and

generating a second command to cause the first device to perform the second predefined device capability.

17. The method as recited in claim 12, further comprising:

receiving a third indication that a second device is in the environment;

determining that the second device is configured to perform the first predefined device capability; and

storing a fourth indication of a third voice command that, when uttered within the environment, is to result in the second device performing the first predefined device capability.

18. The method as recited in claim 17, wherein the third voice command is different from the first voice command.

19. The method as recited in claim 17, further comprising:

receiving an audio signal;

performing speech-recognition on the audio signal to identify the third voice command; and

generating a command to cause the second device to perform the first predefined device capability.

20. The method as recited in claim 12, wherein the analyzing the predefined device capabilities and the one or more capabilities of the first device comprises analyzing the predefined device capabilities and the one or more capabilities of the first device to determine that the first device is configured to perform a third device capability that does not correspond to a predefined device capability of the predefined device capabilities;

and further comprising;

modifying the predefined device capabilities by adding the third device capability to the predefined device capabilities to create updated predefined device capabilities.