Multi-layer keyword detection for distributed voice communications

• US 10,679,625 B1
• Filed: 09/14/2018
• Issued: 06/09/2020
• Est. Priority Date: 12/06/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Lockhart, Christopher Wayne, Cole, Matthew Joseph, Liu, Xulei
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Ogunbiyi, Oluwadamilola M
• Application Number:
  US16/131,705
• Time in Patent Office: 634 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A system and method for temporarily disabling keyword detection to avoid detection of machine-generated keywords. A local device may operate two keyword detectors. The first keyword detector operates on input audio data received by a microphone to capture keywords uttered by a user. In these instances, the keyword may be detected by the first detector and the audio data may be transmitted to a remote device for processing. The remote device may generate output audio data to be sent to the local device. The local device may process the output audio data to determine that it also includes the keyword. The device may then disable the first keyword detector while the output audio data is played back by an audio speaker of the local device. Thus the local device may avoid detection of a keyword originating from the output audio. The first keyword detector may be reactivated after a time interval during which the keyword might be detectable in the output audio.

Claims

1. A first device, comprising:

at least one processor; and

at least one memory including instructions that, when executed by the at least one processor, cause the first device to;

receive, from a second device, first audio data originating from a third device;

determine, using a first detector, that the first audio data includes a first representation of a keyword;

disable a second detector after determining the first audio data includes the first representation, the second detector configured to determine second audio data includes a second representation of the keyword; and

after disabling the second detector, cause audio, corresponding to the first audio data, to be output.

2. The first device of claim 1, wherein:

the first device is associated with a first user profile; and

the third device is associated with a second user profile.

3. The first device of claim 1, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the first device to:

determine the first representation will be output as a portion of the audio during a first time period; and

enable the second detector following the first time period.

4. The first device of claim 1, wherein the first device comprises a microphone and an audio output component, wherein the second detector is coupled to the microphone, and wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the first device to:

cause the first audio data to be sent to the audio output component.

5. The first device of claim 4, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the first device to:

disable the second detector prior to causing the first audio data to be sent to the audio output component.

6. The first device of claim 1, wherein the first detector is configured to send an instruction to the second detector based at least in part on the first detector determining the first audio data includes the first representation, the instruction causing the second detector to ignore the second representation in the second audio data.

7. The first device of claim 6, wherein the instruction prevents the second detector from ceasing output of the audio in response to determining the second representation.

8. The first device of claim 1, wherein the at least one memory further includes instructions that, when executed by the at least one processor, further cause the first device to:

receive the first audio data based at least in part on the third device determining the first representation in the first audio data.

9. The first device of claim 1, wherein the third device is a content source.

10. A method performed by a first device, comprising:

receiving, from a second device, first audio data originating from a third device;

determining, using a first detector, that the first audio data includes a first representation of a keyword;

disabling a second detector after determining the first audio data includes the first representation, the second detector configured to determine second audio data includes a second representation of the keyword; and

after disabling the second detector, causing audio, corresponding to the first audio data, to be output.

11. The method of claim 10, wherein:

the first device is associated with a first user profile; and

the third device is associated with a second user profile.

12. The method of claim 10, further comprising:

determining the first representation will be output as a portion of the audio during a first time period; and

enabling the second detector following the first time period.

13. The method of claim 10, wherein the first device comprises a microphone and an audio output component, wherein the second detector is coupled to the microphone, and wherein the method further comprises:

causing the first audio data to be sent to the audio output component.

14. The method of claim 13, further comprising:

disabling the second detector prior to causing the first audio data to be sent to the audio output component.

15. The method of claim 10, wherein the first detector is configured to send an instruction to the second detector based at least in part on the first detector determining the first audio data includes the first representation, the instruction causing the second detector to ignore the second representation in the second audio data.

16. The method of claim 15, wherein the instruction prevents the second detector from ceasing output of the audio in response to determining the second representation.

17. The method of claim 10, further comprising:

receiving the first audio data based at least in part on the third device determining the first representation in the first audio data.

18. The method of claim 10, wherein the third device is a content source.