Voice control of remote device

• US 10,580,405 B1
• Filed: 12/27/2016
• Issued: 03/03/2020
• Est. Priority Date: 12/27/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wang, Peng, Naidu, Pathivada Rajsekhar
• Primary Examiner(s): Le, Thuykhanh
• Application Number:
  US15/390,875
• Time in Patent Office: 1,162 Days
• Field of Search: None
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A system configured to enable remote control to allow a first user to provide assistance to a second user. The system may receive a command from the second user granting remote control to the first user, enabling the first user to initiate a voice command on behalf of the second user. In some examples, the system may enable the remote control by treating a voice command originating from the first user as though it originated from the second user instead. For example, the system may receive the voice command from a first device associated with the first user but may route the voice command as though it was received by a second device associated with the second user.

Claims

1. A computer-implemented method comprising:

receiving a first command enabling remote control of a first user profile, the remote control authorizing a first speech-controlled device that is associated with a second user profile to initiate commands for the first user profile, the first user profile associated with a second speech-controlled device, the second user profile being different than the first user profile;

receiving first audio data corresponding to communication from the first speech-controlled device to the second speech-controlled device, the first audio data originating from the first speech-controlled device;

sending the first audio data to the second speech-controlled device;

receiving second audio data associated with the communication from the first speech-controlled device to the second speech-controlled device, the second audio originating from the first speech-controlled device;

performing speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform an action associated with the first user profile that is associated with the second speech-controlled device;

determining, based on the first user profile, a recipient device configured to perform the action, the recipient device associated with the first user profile; and

sending an instruction to the recipient device to perform the action.

2. The computer-implemented method of claim 1, further comprising:

storing an indicator in the second user profile indicating that commands originating from the first speech-controlled device are to be associated with the first user profile;

determining that the second audio data originated from the first speech-controlled device;

determining that the first speech-controlled device is associated with the second user profile;

determining, based on the indicator, that commands originating from the first speech-controlled device are to be associated with the use profile; and

determining, based on the first user profile, the recipient device.

3. The computer-implemented method of claim 1, further comprising:

determining, based on an indication stored in the first user profile, that the first speech-controlled device is only authorized to initiate a first plurality of commands associated with the first user profile;

receiving third audio data originating from the first speech-controlled device;

performing speech processing using the third audio data to determine that the third audio data corresponds to a third command to perform a second action associated with the first user profile;

determining that the third command is not included in the first plurality of commands; and

determining not to perform the third command.

4. The computer-implemented method of claim 1, further comprising:

determining, using voice recognition processing, that the second audio is associated with a first speaker identifier;

determining, based on first speaker identifier being associated with the first user profile, that the first speaker identifier is authorized to initiate commands associated with the first user profile;

sending the instruction to the recipient device to perform the action;

receiving third audio data originating from the first device;

performing speech processing using the third audio data to determine that the third audio data corresponds to a third command to perform a second action associated with the first user profile;

determining, using the voice recognition processing, that the third audio is associated with a second speaker identifier;

determining, based on the second speaker identifier not being associated with the first user profile, that the second speaker identifier is not authorized to initiate commands associated with the first user profile; and

determining not to perform the third command.

5. A computer-implemented method, comprising:

receiving first audio data associated with a communication from a first device to a second device, the first audio data originating from the first device that is associated with a first user profile;

performing speech processing using the first audio data to determine that the first audio data corresponds to a command to perform an action associated with a second user profile that is associated with the second device, the second user profile being different than the first user profile;

determining that the first device is granted remote control of the second user profile; and

executing the command, comprising at least one of;

(a) performing the action, or

(b) determining, based on the second user profile, a recipient device configured to perform the action, the recipient device associated with the second user profile, and

sending an instruction to the recipient device to perform the action.

6. The computer-implemented method of claim 5, further comprising:

determining, based on the second user profile, that the second device is the recipient device; and

sending the instruction to the second device to perform the action.

7. The computer-implemented method of claim 5, further comprising:

receiving second audio data originating from the second device;

performing speech processing using the second audio data to determine that the second audio data corresponds to a second command to grant remote control of the second user profile to the first device; and

storing an indication in the first user profile associated with the first device, the indication indicating that audio data originating from the first device is to be associated with the second user profile.

8. The computer-implemented method of claim 5, further comprising:

receiving second audio data originating from the first device;

performing speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform a second action associated with the first user profile;

performing the second command based on the first user profile;

receiving third audio data originating from the first device;

performing speech processing using the third audio data to determine that the third audio data corresponds to a third command to perform a third action associated with the second user profile; and

performing the third command based on the second user profile.

9. The computer-implemented method of claim 5, further comprising:

determining that the first device is granted remote control of the second user profile;

determining a first user associated with the first device;

associating a speaker identifier corresponding to the first user with the second user profile; and

determining, using voice recognition, that the speaker identifier is associated with the first audio data.

10. The computer-implemented method of claim 5, further comprising:

determining that the first device is only granted limited remote control of the second user profile, the limited remote control corresponding to a first plurality of commands;

receiving second audio data originating from the first device;

performing speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform a second action associated with the second user profile;

determining that the second command is not included in the first plurality of commands; and

determining not to perform the second command.

11. The computer-implemented method of claim 5, further comprising:

determining, using voice recognition, a first speaker identifier associated with the first audio;

determining that the first speaker identifier is granted remote control of the second user profile;

performing the command;

receiving second audio data originating from the first device;

performing second speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform a second action associated with the second user profile;

determining, using the voice recognition, a second speaker identifier associated with the second audio;

determining that the second speaker identifier is not granted remote control of the second user profile; and

determining not to perform the second command.

12. The computer-implemented method of claim 5, further comprising, by the first device:

capturing second audio data using one or more microphones;

sending at least a portion of the second audio data to the second device, the second audio data corresponding to the communication from the first device to the second device;

detecting that a wakeword is represented in the second audio data;

generating the first audio data including at least a portion of the second audio data following the wakeword; and

sending the first audio data to a server.

13. A system comprising:

at least one processor; and

memory including instructions operable to be executed by the at least one processor to perform a set of actions to configure the system to;

receive first audio data associated with a communication from a first device to a second device, the first audio data originating from the first device that is associated with a first user profile;

perform speech processing using the first audio data to determine that the first audio data corresponds to a command to perform an action associated with a second user profile that is associated with the second device, the second user profile being different than the first user profile;

determine that the first device is granted remote control of the second user profile; and

execute the command, comprising at least one of;

(a) perform the action, or

(b) determine, based on the second user profile, a recipient device configured to perform the action, the recipient device associated with the second user profile, and

send an instruction to the recipient device to perform the action.

14. The system of claim 13, wherein the system is further configured to:

determine, based on the second user profile, that the second device is the recipient device; and

send the instruction to the second device to perform the action.

15. The system of claim 13, wherein the system is further configured to:

receive second audio data originating from the second device;

perform speech processing using the second audio data to determine that the second audio data corresponds to a second command to grant remote control of the second user profile to the first device; and

storing an indication in the first user profile associated with the first device, the indication indicating that audio data originating from the first device is to be associated with the second user profile.

16. The system of claim 13, wherein the system is further configured to:

receive second audio data originating from the first device;

perform speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform a second action associated with the first user profile;

perform the second command based on the first user profile;

receive third audio data originating from the first device;

perform speech processing using the third audio data to determine that the third audio data corresponds to a third command to perform a third action associated with the second user profile; and

perform the third command based on the second user profile.

17. The system of claim 13, wherein the system is further configured to:

determine that the first device is granted remote control of the second user profile;

determine a first user associated with the first device;

associate a speaker identifier corresponding to the first user with the second user profile; and

determine, using voice recognition, that the speaker identifier is associated with the first audio data.

18. The system of claim 13, wherein the system is further configured to:

determine that the first device is only granted limited remote control of the second user profile, the limited remote control corresponding to a first plurality of commands;

receive second audio data originating from the first device;

perform speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform a second action associated with the second user profile;

determine that the second command is not included in the first plurality of commands; and

determine not to perform the second command.

19. The system of claim 13, wherein the system is further configured to:

determine, using voice recognition, a first speaker identifier associated with the first audio;

determine that the first speaker identifier is granted remote control of the second user profile;

perform the command;

receive second audio data originating from the first device;

perform second speech processing using the second audio data to determine that the second audio data corresponds to a second command to perform a second action associated with the second user profile;

determine, using the voice recognition, a second speaker identifier associated with the second audio;

determine that the second speaker identifier is not granted remote control of the second user profile; and

determine not to perform the second command.

20. The system of claim 13, wherein a first device of the system is further configured to:

capture second audio data using one or more microphones;

send at least a portion of the second audio data to the second device, the second audio data corresponding to the communication from the first device to the second device;

detect that a wakeword is represented in the second audio data;

generate the first audio data including at least a portion of the second audio data following the wakeword; and

send the first audio data to a server.