Detecting self-generated wake expressions

• US 9,747,899 B2
• Filed: 06/27/2013
• Issued: 08/29/2017
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Pogue, Michael Alan, Hilmes, Philip Ryan
• Primary Examiner(s): JACKSON, JAKIEDA R
• Application Number:
  US13/929,540
• Publication Number:
  US 20150006176A1
• Time in Patent Office: 1,524 Days
• Field of Search: 704275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2021/02087 the noise being separate speech, e.g. cocktail party
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming

Abstract

A speech-based audio device may be configured to detect a user-uttered wake expression and to respond by interpreting subsequent words or phrases as commands. In order to distinguish between utterance of the wake expression by the user and generation of the wake expression by the device itself, directional audio signals may by analyzed to detect whether the wake expression has been received from multiple directions. If the wake expression has been received from many directions, it is declared as being generated by the audio device and ignored. Otherwise, if the wake expression is received from a single direction or a limited number of directions, the wake expression is declared as being uttered by the user and subsequent words or phrase are interpreted and acted upon by the audio device.

Claims

1. An audio device configured to respond to a trigger expression uttered by a user, comprising:

a speaker configured to generate output audio;

a microphone array configured to produce a plurality of input audio signals;

an audio beamformer configured to produce a plurality of directional audio signals based at least in part on the input audio signals, wherein the directional audio signals represent audio from respectively corresponding directions relative to the audio device;

one or more speech recognition components configured to detect that the trigger expression occurs in the audio represented by each of the respective directional audio signals; and

an expression detector configured to;

identify two or more directional audio signals, of the plurality of directional audio signals, that each include the trigger expression,

determine that a total number of the two or more directional audio signals is less than a threshold number,

in response to determining that the total number of the two or more directional audio signals is less than the threshold number, determine that the occurrence of the trigger expression in the audio is a result of an utterance of the trigger expression by a user, and

based at least in part on determining that the occurrence of the trigger expression in the audio is a result of an utterance of the trigger expression by the user, provide a wake notification corresponding to the trigger expression to at least one processor of the audio device to change a state of the audio device.

2. The audio device of claim 1, wherein the expression detector is further configured to determine that the trigger expression has been uttered by the user if the trigger expression occurs in the audio from multiple directions that are within a single cone shape extending from an apex at the audio device.

3. The audio device of claim 1, wherein the expression detector is further configured to determine that the trigger expression has been uttered by the user if the trigger expression occurs in the audio from multiple directions that are within two cone shapes extending from apexes at the audio device.

4. The audio device of claim 1, wherein the expression detector is further configured to determine that the trigger expression has been generated by the speaker if the trigger expression occurs in the audio represented by more than half of the plurality of directional audio signals.

5. A method comprising:

producing, by a speaker included in an audio device, output audio in a user environment;

receiving, from a microphone array of the audio device, a plurality of audio signals representing input audio;

generating, for each of the plurality of audio signals, a recognition parameter indicating whether a corresponding audio signal includes a predefined expression;

identifying, based at least in part on the recognition parameters, two or more audio signals, of the plurality of audio signals, that include the predefined expression;

determining that a total number of the two or more audio signals is more than a threshold number; and

based at least in part on the determining that the total number of the two or more audio signals is more than the threshold number, determining that an occurrence of the predefined expression in the input audio is a result of the predefined expression occurring in the output audio from the speaker.

6. The method of claim 5, wherein the identifying the two or more audio signals includes determining that all of the plurality of audio signals include the predefined expression.

7. The method of claim 5, further comprising:

determining that the two or more audio signals and a reference signal set have the same one or more members, wherein the reference signal set has one or more members comprising one or more of the audio signals that contain the predefined expression during an occurrence of the predefined expression in the output audio,

wherein the determining that the occurrence of the predefined expression in the input audio is the result of the predefined expression occurring in the output audio from the speaker is based at least in part on the determining that the two or more audio signals and the reference signal set have the same one or more members.

8. The method of claim 7, further comprising identifying the one or more members of the reference signal set during a known occurrence of the predefined expression in the output audio, wherein the one or more members of the reference signal set comprise one or more of the audio signals that are indicated by the recognition parameters to include the predefined expression during the known occurrence of the predefined expression in the output audio.

9. The method of claim 5, wherein the recognition parameters indicate one or more of the following:

loudness of the output audio;

whether the output audio is known to contain speech;

loudness of the input audio;

or

echo characteristics of the audio signals.

10. The method of claim 9, further comprising using machine learning to perform the determining that the occurrence of the predefined expression in the input audio is the result of the predefined expression occurring in the output audio from the speaker.

11. The method of claim 5, wherein each of the recognition parameters indicates a probability of the predefined expression being present in a corresponding audio signal.

12. An audio device comprising:

one or more processors;

a plurality of microphones;

an audio speaker configured to produce output audio;

memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

receiving, from the plurality of microphones, a plurality of audio signals representing input audio;

generating, by the one or more processors for each of the plurality of audio signals, an indication indicating whether a corresponding audio signal contains a predefined expression;

generating a parameter that indicates at least one of whether or not the output audio is currently being produced, whether or not the output audio contains speech, loudness of the output audio, or an echo characteristic of one or more of the plurality of audio signals;

identifying, based at least in part on the indications, two or more audio signals, of the plurality of audio signals, that include the predefined expression;

determining, based at least in part on the two or more audio signals and the parameter, that an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user; and

in response to the determining, providing a wake notification corresponding to the predefined expression to the one or more processors to change a state of the audio device.

13. The audio device of claim 12, wherein each of the indications comprises a binary parameter indicating whether the predefined expression occurs in the corresponding audio signal.

14. The audio device of claim 12, wherein each of the indications comprises a probability that the predefined expression occurs in the corresponding audio signal.

15. The audio device of claim 12, the acts further comprising:

determining that the two or more audio signals and a reference signal set have the same one or more members, wherein the one or more members of the reference signal set comprise one or more of the audio signals that contain the predefined expression during production of the predefined expression by the audio speaker.

16. The audio device of claim 15, the acts further comprising identifying the one or more members of the reference signal set during known production of the predefined expression by the audio speaker, wherein the one or more members of the reference signal set comprise one or more of the audio signals that are indicated by the indications to contain the predefined expression during the known production of the predefined expression by the audio speaker.

17. The one audio device of claim 12, wherein the wake notification indicates a probability that the predefined expression has been uttered by the user.