Accepting voice commands based on user identity

• US 9,098,467 B1
• Filed: 12/19/2012
• Issued: 08/04/2015
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Blanksteen, Scott Ian, Hoffmeister, Bjorn
• Primary Examiner(s): Guerra-Erazo, Edgar
• Application Number:
  US13/720,841
• Time in Patent Office: 958 Days
• Field of Search: 704/233, 704/235, 704/246, 704/247, 704/270, 704/275
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

Techniques for determining when to perform an action associated with a voice command and when to disregard the voice command. In some instances, the techniques reference an identity of a user that utters a command when making this determination. For instance, if a first user awakens a device or initially begins providing voice commands to the device, the device or another computing device may perform actions associated with subsequent voice commands uttered by the same user, while disregarding voice commands from other users. That is, because the device is engaging in a dialog with a first user, the device may refrain from engaging in a dialog with a different user. Additionally or alternatively, the device may begin a new dialog with the different user, either upon identifying the command of the other user or upon finishing the dialog with the first user.

Claims

1. One or more computing devices comprising:

one or more processors;

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

receiving a first audio signal generated based on a first sound captured within an environment;

identifying first speech from the first audio signal, the first speech including a wake word;

receiving a second audio signal generated based on a second sound captured within the environment;

determining that the second audio signal includes second speech;

determining whether a user that spoke the first speech is the same as a user that spoke the second speech;

causing performance of an action associated with the second speech at least partly in response to determining that the user that spoke the first speech is the same as the user that spoke the second speech; and

refraining from causing performance of the action associated with the second speech at least partly in response to determining that the user that spoke the first speech is not the same as the user that spoke the second speech.

2. One or more computing devices as recited in claim 1, the acts further comprising:

receiving a third audio signal generated based on a third sound captured within the environment;

determining that the third audio signal includes third speech;

determining whether the user that spoke the first speech is the same as a user that spoke the third speech;

causing performance of an action associated with the third speech at least partly in response to determining that the user that spoke the first speech is the same as the user that spoke the third speech; and

refraining from causing performance of the action associated with the third speech at least partly in response to determining that the user that spoke the first speech is not the same as the user that spoke the third speech.

3. One or more computing devices as recited in claim 1, the acts further comprising:

issuing a query to the user that spoke the second speech at least partly in response to determining that the user that spoke the first speech is not the same as the user that spoke the second speech.

4. One or more computing devices as recited in claim 3, wherein the query requests that the user that spoke the second speech confirm that the user that spoke the second speech would like to request performance of the action associated with the second speech.

5. One or more computing devices as recited in claim 1, wherein the determining is based at least in part on at least one of a Gaussian mixture model of the first speech compared to a Gaussian mixture model of the second speech, a cMLLR matrix created from the first speech compared to a cMLLR matrix created from the second speech, or a location within the environment of the user that spoke the first speech compared to a location within the environment of the user that spoke the second speech.

6. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

determining whether a user that uttered a first audible command is the same as a user that uttered a second audible command;

causing performance of an action associated with the second audible command at least partly in response to determining that the user that uttered the first audible command is the same as the user that uttered the second audible command; and

refraining from causing performance of the action associated with the second audible command at least partly in response to determining that the user that uttered the first audible command is not the same as the user that uttered the second audible command.

7. One or more non-transitory computer-readable media as recited in claim 6, wherein the first audible command comprises a command to awaken a device proximate to the user that provided the first audible command, the device configured to provide audio signals generated based on one or more sounds captured within an environment to one or more remote computing devices upon recognizing the command to awaken the device.

8. One or more non-transitory computer-readable media as recited in claim 7, wherein the command to awaken the device comprises one or more predefined words.

9. One or more non-transitory computer-readable media as recited in claim 6, the acts further comprising:

determining whether a user that uttered a third audible command is the same as the user that uttered the first audible command;

causing performance of an action associated with the third audible command at least partly in response to determining that the user that uttered the third audible command is the same as the user that uttered the first audible command; and

refraining from causing performance of the action associated with the third audible command at least partly in response to determining that the user that uttered the third audible command is not the same as the user that uttered the first audible command.

10. One or more non-transitory computer-readable media as recited in claim 6, wherein the determining is based at least in part on at least one of a Gaussian mixture model, a cMLLR matrix, or directionality relative to a device that captures one or more sounds associated with the first and second audible commands.

11. One or more non-transitory computer-readable media as recited in claim 6, wherein the determining comprises:

calculating a similarity between a voice print associated with the first audible command and a voice print associated with the second audible command;

comparing the similarity to a threshold; and

determining that the user that uttered the first audible command is the same as the user that uttered the second audible command at least partly in response to determining that the similarity is greater than the threshold.

12. One or more non-transitory computer-readable media as recited in claim 6, wherein the determining is performed at least partly using a support vector machine (SVM) configured to classify different units of speech as originating from a same user or from different users.

13. One or more non-transitory computer-readable media as recited in claim 6, wherein the action associated with the second audible command comprises setting a reminder, playing a media file, adding an item to a list, providing a recommendation to the user, purchasing an item, or placing a telephone call.

14. One or more non-transitory computer-readable media as recited in claim 6, wherein at least a portion of the one or more computer-readable media resides within a device that captures one or more sounds associated with the first and second audible commands.

15. One or more non-transitory computer-readable media as recited in claim 6, wherein at least a portion of the one or more computer-readable media resides remote from a device that captures one or more sounds associated with the first and second audible commands, the device providing audio signals over a network and to the at least a portion of the one or more computer-readable media.

16. A method comprising:

under control of one or more computing devices configured with executable instructions,

determining that an audio signal includes a predefined utterance;

performing a first subsequent command at least partially in response to the first subsequent command being uttered by a user that uttered the predefined utterance; and

disregarding a second subsequent command at least partially in response to the second subsequent command being uttered by a different user from the user that uttered the predefined utterance.

17. A method as recited in claim 16, further comprising determining, for the first subsequent command, whether the user that uttered the first subsequent command is the same as the user that uttered the predefined utterance.

18. A method as recited in claim 16, further comprising:

calculating, for the first subsequent command, a similarity between a voice print associated with the first subsequent command and a voice print associated with the predefined utterance; and

comparing the similarity to a threshold.

19. A method as recited in claim 16, wherein the disregarding comprises refraining from performing the second subsequent command for the different user, and further comprising requesting that the different user utter the predefined utterance or another predefined utterance.

20. A method as recited in claim 19, further comprising performing the second subsequent command uttered by the different user at least partly in response to the different user uttering the predefined utterance or the other predefined utterance.