Audio signal transmission techniques

• US 9,570,071 B1
• Filed: 08/17/2015
• Issued: 02/14/2017
• Est. Priority Date: 03/26/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hart, Gregory Michael, Bezos, Jeffrey P.
• Primary Examiner(s): Colucci, Michael
• Application Number:
  US14/828,263
• Time in Patent Office: 547 Days
• Field of Search: 704/275, 704/201, 704/215, 704/224, 704/226, 704/231, 704/233, 704/234, 704/250, 704/251, 704/254, 704/270, 704/500, 375/285, 381/56, 381/66, 381/92, 381/94.7, 455/570
• US Class Current: 1/1
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

A voice interaction architecture that compiles multiple audio signals captured at different locations within an environment, determines a time offset between a primary audio signal and other captured audio signals and identifies differences between the primary signal and the other signal(s). Thereafter, the architecture may provide the primary audio signal, an indication of the determined time offset(s) and the identified differences to remote computing resources for further processing. For instance, the architecture may send this information to a network-accessible distributed computing platform that performs beamforming and/or automatic speech recognition (ASR) on the received audio. The distributed computing platform may in turn determine a response to provide based upon the beamforming and/or ASR.

Claims

1. One or more computing devices comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising;

receiving a first signal generated by a first audio transducer of an audio transducer array, the audio transducer array comprising the first audio transducer and a second audio transducer, the first signal generated by the first audio transducer based at least in part on audio captured by the first audio transducer, and the second audio transducer generating a second signal based at least in part on the audio as captured by the second audio transducer;

receiving information indicative of a difference between the first signal and the second signal;

generating a representation of the second signal based at least in part on (i) the first signal, and (ii) the information indicative of the difference between the first signal and the second signal; and

performing signal processing on at least one of the first signal or the representation of the second signal.

2. The one or more computing devices as recited in claim 1, the acts further comprising performing automatic speech recognition (ASR) on the at least one of the first signal or the representation of the second signal to identify a voice command uttered within the environment.

3. The one or more computing devices as recited in claim 1, wherein the performing the signal processing comprises determining a direction of a source of the audio within the environment using the at least one of the first signal or the representation of the second signal.

4. The one or more computing devices as recited in claim 1, wherein the performing the signal processing comprises at least one of performing beamforming, echo cancellation, noise reduction, dereverberation, or blind source separation on the at least one of the first signal or the representation of the second signal.

5. The one or more computing devices as recited in claim 1, wherein the receiving the information indicative of the difference between the first signal and the second signal includes receiving individual segments of the second signal.

6. The one or more computing devices as recited in claim 1, the acts further comprising receiving an indication of a time offset between the first signal and the second signal, and wherein the generating comprises generating the representation of the second signal based at least in part on (i) the first signal, (ii) the information indicative of the difference between the first signal and the second signal, and (iii) the indication of the time offset between the first signal and the second signal.

7. The one or more computing devices as recited in claim 1, the acts further comprising:

receiving information indicative of a difference between the first signal and a third signal, the third signal representing the audio captured by a third audio transducer at a third location within the environment;

receiving an indication of a time offset between first signal and the third signal;

generating a representation of the third audio signal based at least in part on (i) the information indicative of the difference between the first signal and the third signal, and (ii) the indication of the time offset between the first signal and the third signal.

8. The one or more computing devices as recited in claim 1, the acts further comprising:

determining, based at least in part on the performing of the signal processing, a response to provide to one or more computing devices at the environment; and

providing data indicating the response to the one or more computing devices at the environment.

9. A method comprising:

receiving a first signal generated by a first audio transducer of an audio transducer array, the audio transducer array comprising the first audio transducer and a second audio transducer, the first signal generated by the first audio transducer based at least in part on audio captured by the first audio transducer, and the second audio transducer generating a second signal based at least in part on the audio as captured by the second audio transducer;

receiving information indicative of a difference between the first signal and the second signal;

generating a representation of the second signal based at least in part on (i) the first signal, and (ii) the information indicative of the difference between the first signal and the second signal; and

performing signal processing on at least one of the first signal or the representation of the second signal.

10. The method as recited in claim 9, further comprising performing automatic speech recognition (ASR) on the at least one of the first signal or the representation of the second signal to identify a voice command uttered within the environment.

11. The method as recited in claim 9, wherein the performing the signal processing comprises determining a direction of a source of the audio within the environment using the at least one of the first signal or the representation of the second signal.

12. The method as recited in claim 9, wherein the performing the signal processing comprises at least one of performing beamforming, echo cancellation, noise reduction, dereverberation, or blind source separation on the at least one of the first signal or the representation of the second signal.

13. The method as recited in claim 9, wherein the receiving the information indicative of the difference between the first signal and the second signal includes receiving individual segments of the second signal.

14. The method as recited in claim 9, further comprising receiving an indication of a time offset between the first signal and the second signal, and wherein the generating comprises generating the representation of the second signal based at least in part on (i) the first signal, (ii) the information indicative of the difference between the first signal and the second signal, and (iii) the indication of the time offset between the first signal and the second signal.

15. The method as recited in claim 9, further comprising:

receiving information indicative of a difference between the first signal and a third signal, the third signal representing the audio captured by a third audio transducer at a third location within the environment;

receiving an indication of a time offset between first signal and the third signal;

generating a representation of the third audio signal based at least in part on (i) the information indicative of the difference between the first signal and the third signal, and (ii) the indication of the time offset between the first signal and the third signal.

16. The method as recited in claim 9, further comprising:

determining, based at least in part on the performing of the signal processing, a response to provide to one or more computing devices at the environment; and

providing data indicating the response to the one or more computing devices at the environment.

17. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

receiving a first signal generated by a first audio transducer of an audio transducer array, the audio transducer array comprising the first audio transducer and a second audio transducer, the first signal generated by the first audio transducer based at least in part on audio captured by the first audio transducer, and the second audio transducer generating a second signal based at least in part on the audio as captured by the second audio transducer;

receiving information indicative of a difference between the first signal and the second signal;

generating a representation of the second signal based at least in part on (i) the first signal, and (ii) the information indicative of the difference between the first signal and the second signal; and

performing signal processing on at least one of the first signal or the representation of the second signal.

18. The one or more non-transitory computer-readable media as recited in claim 17, the acts further comprising performing automatic speech recognition (ASR) on the at least one of the first signal or the representation of the second signal to identify a voice command uttered within the environment.

19. The one or more non-transitory computer-readable media as recited in claim 17, wherein the performing the signal processing comprises determining a direction of a source of the audio within the environment using the at least one of the first signal or the representation of the second signal.

20. The one or more non-transitory computer-readable media as recited in claim 17, wherein the performing the signal processing comprises at least one of performing beamforming, echo cancellation, noise reduction, dereverberation, or blind source separation on the at least one of the first signal or the representation of the second signal.