Distributed speaker synchronization

• US 9,319,782 B1
• Filed: 12/20/2013
• Issued: 04/19/2016
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Crump, Edward Dietz, Hilmes, Philip Ryan
• Primary Examiner(s): King, Simon
• Application Number:
  US14/137,587
• Time in Patent Office: 851 Days
• Field of Search: 381/73.1
• US Class Current: 1/1
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/02 for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453
• CPC Class Codes: H04R 3/04 for correcting frequency response

Abstract

Audio of electronic audio devices may be synchronized by a signal synchronization component that receives one or more signals corresponding to elements of the audio output by the electronic audio devices. The signal synchronization component may perform calculations to align signals corresponding to the output audio of the electronic audio devices and then determine a delay for the output audio transmitted from the electronic audio devices with respect to each other. Additionally, the signal synchronization component may operate in conjunction with audio sources of the electronic audio devices to modify the timing for transmitting output audio by one or more of the electronic audio devices based, at least in part, on the delay. In this way, the output audio transmitted by the electronic audio devices may be synchronized.

Claims

1. An audio device comprising:

a first speaker to output first audio;

a first microphone to capture elements of the first audio and to capture elements of the second audio from a second speaker of an additional audio device, wherein the first microphone produces an audio input signal corresponding to the elements of the first audio and the elements of the second audio;

a second microphone to capture the elements of the first audio and to capture a portion of the elements of the second audio, wherein the second microphone produces a reference signal that corresponds to the elements of the first audio and the portion of the elements of the second audio;

one or more processors;

one of more computer-readable storage media in communication with the one or more processors, the one or more computer-readable storage media including instructions executable by the one or more processors to perform operations comprising;

isolating a portion of the audio input signal corresponding to one or more of the elements of the second audio to produce a modified input signal by subtracting a portion of the reference signal corresponding to the elements of the first audio from the audio input signal;

generating a cross-correlation function that indicates, for each of a plurality of delays, an amount of correlation between the portion of the reference signal corresponding to the elements of the first audio and the modified input signal;

determining a delay of the plurality of delays corresponding to the amount of correlation between the portion of the reference signal corresponding to the elements of the first audio and the modified input signal being at a maximum; and

outputting additional audio from the first speaker that is delayed by an amount of time of the delay.

2. The audio device of claim 1, wherein:

the audio device is located at a first location; and

the operations further comprise determining a second location that is remote from the first location by receiving a signal including a distance measurement indicating a distance between the first location and the second location or receiving a signal indicating a difference between a time of arrival of the first audio from the first speaker at the second location and a time of arrival of the second audio from the second speaker at the second location.

3. The audio device of claim 2, wherein:

the operations further comprise determining an estimated amount of time for sound to travel from the first location to the second location; and

the additional audio output from the first speaker is delayed by an amount of time between the second microphone capturing the elements of the first audio and the first microphone capturing the elements of the second audio and the estimated amount of time for sound to travel from the first location to the second location.

4. A computing device, comprising:

one or more processors;

one of more computer-readable storage media in communication with the one or more processors, the one or more computer-readable storage media including instructions executable by the one or more processors to perform operations comprising;

receiving an audio input signal corresponding to elements of first audio and elements of second audio;

receiving a reference signal corresponding to the elements of the first audio;

aligning at least a portion of the audio input signal that corresponds to at least a portion of the elements of the second audio with at least a portion of the reference signal that corresponds to at least a portion of the elements of the first audio; and

determining a delay between the first audio and the second audio based, at least in part, on the aligning.

5. The computing device of claim 4, wherein:

the computing device is a first audio device, the first audio is produced by the first audio device, and the second audio is produced by a second audio device; and

the operations further comprise receiving an additional reference signal corresponding to the elements of the second audio.

6. The computing device of claim 5, wherein:

the delay is a first delay, the first delay indicating that the elements of the second audio are delayed by a first period of time with respect to the elements of the first audio;

the audio input signal includes elements of third audio produced by a third audio device, and

the operations further comprise;

determining a second delay between the first audio and the third audio by aligning at least a portion of the audio input signal that corresponds to at least a portion of elements of the third audio with the at least a portion of the reference signal that corresponds to the at least a portion of the elements of the first audio, the second delay indicating that the elements of the third audio are delayed by a second period of time with respect to the elements of the first audio; and

determining a third delay between the second audio and the third audio by aligning the at least a portion of the audio input signal that corresponds to the at least a portion of elements of the third audio with at least a portion of the additional reference signal that corresponds to at least a portion of the elements of the second audio, the third delay indicating that the elements of the third audio are delayed by a third period of time with respect to the elements of the second audio.

7. The computing device of claim 5, wherein the operations further comprise:

determining that the second period of time is greater than the first period of time and that the second period of time is greater than the third period of time; and

in response to determining that the second period of time is greater than the first period of time and that the second period of time is greater than the third period of time, delaying transmission of the first audio according to the second period of time.

8. The computing device of claim 7, wherein the operations further comprise:

in response to determining that the second period of time is greater than the first period of time and that the second period of time is greater than the third period of time, sending a signal to the second audio device to delay transmission of the second audio according to the third period of time.

9. The computing device of claim 4, wherein the operations further comprise delaying output of the first audio from a speaker of the computing device according to the delay.

10. The computing device of claim 4, wherein the operations further comprise generating a cross-correlation function to align the at least a portion of the audio input signal that corresponds to the at least a portion of the elements of the second audio with at least the portion of the reference signal that corresponds to the at least a portion of the elements of the first audio.

11. The computing device of claim 10, wherein the operations further comprise identifying a maximum of the cross-correlation function that indicates the delay.

12. A method, comprising:

receiving an audio input signal corresponding to elements of respective audio from a plurality of audio devices and elements of audio from an additional audio source;

receiving a reference signal corresponding to one or more elements of first audio produced by a first audio device of the plurality of audio devices;

isolating a portion of the audio input signal corresponding to one or more elements of second audio produced by a second audio device of the plurality of audio devices by subtracting from the audio input signal a portion of the reference signal corresponding to the one or more elements of the first audio from the audio input signal and by subtracting from the audio input signal a portion of the audio input signal corresponding to at least a portion of the elements of the audio from the additional audio source; and

determining a delay between the first audio and the second audio at least partly in response to performing calculations to determine a maximum amount of correlation between the portion of the input audio signal corresponding to the one or more elements of the second audio and the portion of the reference signal corresponding to the one or more elements of the first audio, the delay indicating a period of time that the first audio is to be delayed from transmission or output with respect to the second audio.

13. The method of claim 12, wherein the first audio is generated from first audio content;

and the second audio is generated from second audio content different from the first audio content.

14. The method of claim 12, wherein the period of time is a first period of time, and the method further comprising:

receiving an additional reference signal corresponding to the one or more elements of the second audio;

isolating a portion of the audio input signal corresponding to the one or more elements of the first audio by subtracting from the audio input signal a portion of the additional reference signal corresponding to the one or more elements of the second audio and subtracting from the audio input signal the portion of the audio input signal corresponding to the at least a portion of the elements of the audio from the additional audio source; and

determining an additional delay between the first audio and the second audio at least partly in response to performing additional calculations to determine a maximum amount of correlation between the portion of the input audio signal corresponding to the one or more elements of the first audio with the portion of the additional reference signal corresponding to the one or more elements of the second audio, the additional delay indicating a second period of time that the elements of the second audio are to be delayed from transmission or output with respect to the first audio.

15. The method of claim 14, further comprising:

determining that the second period of time is greater than the first period of time; and

in response to determining that the second period of time is greater than the first period of time, delaying transmission or output of the first audio for a third period of time based at least in part on a difference between the second period of time and the first period of time.

16. The method of claim 14, further comprising:

sending a first signal to the first audio device to delay transmitting or outputting the first audio according to the delay; and

sending a second signal to the second audio device to delay transmitting or outputting the second audio according to the additional delay.

17. The method of claim 12, further comprising:

determining that the delay is greater than or equal to a threshold delay; and

transmitting the first audio according to the delay at least partly in response to determining that the delay is greater than or equal to the threshold delay.

18. The method of claim 12, further comprising:

transmitting a first portion of the first audio according to a first portion of the delay; and

transmitting a second portion of the first audio according to a second portion of the delay.

19. The method of claim 12, wherein the elements of the audio from the additional source include human speech.

20. The method of claim 12, wherein the audio input signal is received from an array of microphones receiving the respective audio from the plurality of audio devices, the array of microphones being remote from each audio device of the plurality of audio devices.