Identification using Audio Signatures and Additional Characteristics

• US 20140249817A1
• Filed: 03/04/2013
• Published: 09/04/2014
• Est. Priority Date: 03/04/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Blanksteen, Scott Ian, Potter, Stephen Frederick, Hart, Gregory Michael, Barton, William Folwell
• Granted Patent: US 9,460,715 B2
• Time in Patent Office: Days
• US Class Current: 704/239
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Techniques for using both speaker-identification information and other characteristics associated with received voice commands to determine how and whether to respond to the received voice commands. A user may interact with a device through speech by providing voice commands. After beginning an interaction with the user, the device may detect subsequent speech, which may originate from the user, from another user, or from another source. The device may then use speaker-identification information and other characteristics associated with the speech to attempt to determine whether or not the user interacting with the device uttered the speech. The device may then interpret the speech as a valid voice command and may perform a corresponding operation in response to determining that the user did indeed utter the speech. If the device determines that the user did not utter the speech, however, then the device may refrain from taking action on the speech.

Claims

1. One or more computing devices comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

receiving a first voice command requesting performance of a first operation, the first voice command being associated with a first voice signature;

causing performance of the first operation at least partly in response to receiving the first voice command;

receiving a second voice command requesting performance of a second operation, the second voice command being associated with a second voice signature;

calculating a similarity between the first voice signature and the second voice signature;

determining, based at least in part on the calculated similarity, whether to;

(i) cause performance of the second operation, or (ii) refrain from causing performance of the second operation.

2. One or more computing devices as recited in claim 1, the acts further comprising identifying a characteristic, other than the voice signature of the second voice command, associated with the second voice command, and wherein the determining whether to cause performance of the second operation or refrain from causing performance of the second operation is also based at least in part on comparing the characteristic to a characteristic associated with the first voice command.

3. One or more computing devices as recited in claim 2, wherein the characteristic comprises one or more of:

a grammar of the second voice command or a time of day at which the second voice command is uttered.

4. One or more computing devices as recited in claim 1, wherein the calculating comprises at least one of comparing a volume of the first voice command to a volume of the second voice command or comparing a frequency of the first voice command to a frequency of the second voice command.

5. One or more computing devices as recited in claim 1, the acts further comprising issuing a query regarding whether or not to perform the second operation after determining to refrain from causing performance of the second operation.

6. One or more computing devices as recited in claim 1, wherein the first and second voice signatures are based at least in part on at least one of a volume or frequency of the respective voice commands.

7. One or more computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

receiving an audio signal generated by a device residing within an environment, the audio signal including speech uttered while the device performs a first operation for a user within the environment;

performing speech recognition on the audio signal to identify the speech within the audio signal;

calculating a confidence level that the user uttered the speech; and

determining whether or not to perform a second operation, specified by the speech, based at least in part on the calculated confidence level.

8. One or more computer-readable media as recited in claim 7, wherein the calculating comprises comparing a voice signature of the speech to a voice signature associated with the user, wherein the voice signature of the speech is based at least in part on at least one of a volume or frequency of the speech and the voice signature associated with the user is based at least in part on at least one of a volume or frequency of previous speech uttered by the user.

9. One or more computer-readable media as recited in claim 8, the acts further comprising identifying a characteristic, other than the voice signature of the speech, associated with the speech, and wherein the calculating further comprises calculating the confidence level with reference to the characteristic.

10. One or more computer-readable media as recited in claim 9, wherein calculating the confidence level with reference to the characteristic comprises calculating a similarity between the characteristic and a corresponding characteristic associated with the user.

11. One or more computer-readable media as recited in claim 9, wherein the characteristic comprises a grammar of the speech, and the calculating includes comparing a similarity between the grammar and a grammar of one or more previous voice commands spoken by the user.

12. One or more computer-readable media as recited in claim 9, wherein the characteristic comprises a location within the environment from which the speech was uttered, and the calculating includes comparing a similarity between the location and a location within the environment from which the user uttered a previous voice command.

13. A method comprising:

under control of one or more computing devices configured with executable instructions,

identifying a voice command uttered by a user in an environment;

performing speech recognition to identify subsequent speech uttered within the environment;

attempting to determine whether the user uttered the subsequent speech; and

interpreting the subsequent speech as a valid voice command at least partly in response to determining that the first user uttered the subsequent speech.

14. A method as recited in claim 13, further comprising refraining from interpreting the subsequent speech as a valid voice command at least partly in response to determining that the first user did not utter the subsequent speech.

15. A method as recited in claim 13, further comprising:

identifying a user that uttered the subsequent speech; and

communicating with the user regarding whether or not to perform an action associated with the subsequent speech.

16. A method as recited in claim 15, wherein the communicating comprises causing output of audio directed to the user that uttered the subsequent speech or sending a communication to a device or application associated with the user that uttered the subsequent speech.

17. A method as recited in claim 13, wherein the attempting comprises comparing a voice signature of the voice command uttered by the user to a voice signature of the subsequent speech.

18. A method as recited in claim 17, wherein the voice signature of the voice command is based at least in part on a volume or frequency of the voice command and the voice signature of the subsequent speech is based at least in part on a volume or frequency of the subsequent speech.

19. A method as recited in claim 13, wherein the attempting comprises:

comparing a voice signature of the voice command uttered by the user to a voice signature of the subsequent speech; and

identifying a characteristic associated with the subsequent speech and comparing the characteristic to a known characteristic associated with the user.

20. A method as recited in claim 19, wherein the characteristic comprises a grammar of the subsequent speech, a choice of words of the subsequent speech, or a location within the environment from which the subsequent speech was uttered.

21. A method as recited in claim 13, wherein the attempting comprises comparing past voice commands uttered by the user to one or more characteristics associated with the subsequent speech.

22. A method comprising:

under control of one or more computing devices configured with executable instructions,

receiving a first voice command uttered by a user in an environment;

performing a first action at least partly in response to receiving the voice command;

receiving a second voice command uttered within the environment, the second voice command requesting performance of a second action that is related to the first action;

identifying one or more characteristics associated with at least one of the second voice command or the second action; and

determining whether or not to perform the second action based at least in part on the one or more characteristics.

23. A method as recited in claim 22, wherein the one or more characteristics comprise whether a user that uttered the second voice command is the same as the user that uttered the first voice command.

24. A method as recited in claim 22, wherein the one or more characteristics comprise an identity of a user that uttered the second voice command, and the determining comprises determining whether the user that uttered the second voice command is include in a group of one or more users authorized to cause performance of the second action.

25. A method as recited in claim 22, wherein the one or more characteristics comprise whether or not a user that uttered the second voice command is recognized to the one or more computing devices, and the determining is based at least in part on whether or not the user that uttered the second voice command is recognized.