Estimation of time delay of arrival for microphone arrays

• US 9,319,787 B1
• Filed: 12/19/2013
• Issued: 04/19/2016
• Est. Priority Date: 12/19/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Chu, Wai Chung
• Primary Examiner(s): Addy, Thjuan K
• Application Number:
  US14/135,320
• Time in Patent Office: 852 Days
• Field of Search: 381/92, 381/120, 381/122, 381/98
• US Class Current: 1/1
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

The accuracy and computationally efficient estimation of time different (or delay) of arrival (TDOA) data is improved for localization of a sound. In one aspect, for each acoustic source event, multiple sets of TDOA data are generated, where each set uses a different sensor or microphone to be the reference. One of the microphones is ultimately selected to be the reference microphone based, in part, on correlation functions of the various sets of TDOA data. The selected reference microphone is then used in sound source localization or other signal processing applications. The direction of the sound source is found using a VMRL finding algorithm as a function of a channel vector containing information of the selected channels, the reference channel and a TDOA vector.

Claims

1. One or more non-transitory computer-readable media storing computer-executable instructions executable by one or more processors to perform operations comprising:

receiving acoustic signals from an array of at least first, second, and third microphones, the acoustic signals being associated with an acoustic source in an environment;

generating at least first, second, and third sets of time-difference-of-arrival (TDOA) data, wherein the first set of TDOA data is derived from time differences between the acoustic signals of the first microphone and the second microphone relative to the acoustic signal of the third microphone, wherein the second set of TDOA data is derived from time differences between the acoustic signals of the first microphone and the third microphone relative to the acoustic signal of the second microphone, wherein the third set of TDOA data is derived from time differences between the acoustic signals of the second microphone and the third microphone relative to the acoustic signal of the first microphone;

for the first set of TDOA data, computing a correlation function between the acoustic signal from the first microphone and the acoustic signal from the second microphone, while excluding the acoustic signal from the third microphone, to produce a first correlation value;

for the second set of TDOA data, computing a correlation function between the acoustic signal from the first microphone and the acoustic signal from the third microphone, while excluding the acoustic signal from the second microphone, to produce a second correlation value;

for the third set of TDOA data, computing a correlation function between the acoustic signal from the second microphone and the acoustic signal from the third microphone, while excluding the acoustic signal from the first microphone, to produce a third correlation value;

wherein a comparatively higher correlation value implies that two acoustic signals share similar structure when offset by a time lag, and a comparatively lower correlation value implies that two acoustic signals do not share similar structure when offset by the time lag;

determining that the first correlation value is lowest;

selecting, as a reference microphone, the third microphone; and

localizing the acoustic source in the environment by computing, in part, a direction to the acoustic source based on one of the first, second, and third sets of TDOA data associated with the reference microphone.

2. The one or more non-transitory computer-readable media of claim 1, wherein generating the first, second, and third sets of time-difference-of-arrival (TDOA) data comprises:

for the first set of TDOA data, subtracting a time at which the acoustic signal reaches the first microphone from a time at which the acoustic signal reaches the third microphone and subtracting a time at which the acoustic signal reaches the second microphone from the time at which the acoustic signal reaches the third microphone;

for the second set of TDOA data, subtracting the time at which the acoustic signal reaches the first microphone from the time at which the acoustic signal reaches the second microphone and subtracting the time at which the acoustic signal reaches the third microphone from the time at which the acoustic signal reaches the second microphone; and

for the third set of TDOA data, subtracting the time at which the acoustic signal reaches the second microphone from the time at which the acoustic signal reaches the first microphone and subtracting the time at which the acoustic signal reaches the second microphone from the time at which the acoustic signal reaches the first microphone.

3. The one or more non-transitory computer-readable media of claim 1, further storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

excluding the acoustic signal from the first microphone when a ratio of the correlation value of the first microphone to the correlation value of the selected reference microphone satisfies a predetermined criteria;

excluding the acoustic signal from the second microphone when a ratio of the correlation value of the second microphone to the correlation value of the selected reference microphone satisfies the predetermined criteria; and

excluding the acoustic signal from the third microphone when a ratio of the correlation value of the third microphone to the correlation value of the selected reference microphone satisfies the predetermined criteria.

4. The one or more non-transitory computer-readable media of claim 3, wherein the predetermined criteria is a threshold with a value between 1.0 and 1.5, and at least one of the first acoustic signal, the second acoustic signal, and the third acoustic signal are excluded when an associated ratio exceeds the threshold.

5. A computer-implemented method comprising:

receiving acoustic signals from an array of at least first, second, and third microphones, the acoustic signals being associated with an acoustic source in an environment;

generating at least first, second, and third sets of time-difference-of-arrival (TDOA) data, wherein the first set of TDOA data is derived from time differences between the acoustic signals of the first microphone and the second microphone relative to the acoustic signal of the third microphone, wherein the second set of TDOA data is derived from time differences between the acoustic signals of the first microphone and the third microphone relative to the acoustic signal of the second microphone, wherein the third set of TDOA data is derived from time differences between the acoustic signals of the second microphone and the third microphone relative to the acoustic signal of the first microphone;

selecting one of the first, second, and third microphones from the array to be a reference microphone and an associated set of the TDOA data such that if the first microphone is selected, the third set of TDOA data is associated with the first microphone, if the second microphone is selected, the second set of TDOA data is associated with the second microphone, and if the third microphone is selected, the first set of TDOA data is associated with the third microphone; and

outputting an identity of the selected reference microphone and the associated set of the TDOA data.

6. The computer-implemented method of claim 5, wherein generating the first, second, and third sets of time-difference-of-arrival (TDOA) data comprises:

for the first set of TDOA data, subtracting a time at which the acoustic signal reaches the first microphone from a time at which the acoustic signal reaches the third microphone and subtracting a time at which the acoustic signal reaches the second microphone from the time at which the acoustic signal reaches the third microphone;

for the second set of TDOA data, subtracting the time at which the acoustic signal reaches the first microphone from the time at which the acoustic signal reaches the second microphone and subtracting the time at which the acoustic signal reaches the third microphone from the time at which the acoustic signal reaches the second microphone; and

for the third set of TDOA data, subtracting the time at which the acoustic signal reaches the second microphone from the time at which the acoustic signal reaches the first microphone and subtracting the time at which the acoustic signal reaches the second microphone from the time at which the acoustic signal reaches the third microphone.

7. The computer-implemented method of claim 5, wherein selecting the reference microphone comprises:

for the first set of TDOA data, computing a correlation function between the acoustic signal from the first microphone and the acoustic signal from the second microphone, while excluding the acoustic signal from the third microphone, to produce a first correlation value;

for the second set of TDOA data, computing a correlation function between the acoustic signal from the first microphone and the acoustic signal from the third microphone, while excluding the acoustic signal from the second microphone, to produce a second correlation value;

for the third set of TDOA data, computing a correlation function between the acoustic signal from the second microphone and the acoustic signal from the third microphone, while excluding the acoustic signal from the first microphone, to produce a third correlation value;

wherein a comparatively higher correlation value implies that two acoustic signals share similar structure when offset by a time lag, and a comparatively lower correlation value implies that two acoustic signals do not share similar structure when offset by the time lag;

determining which of the first, second, and third correlation values is lowest; and

selecting, as a reference microphone, one of the first microphone, the second microphone, or the third microphone that was excluded in the computation of the first, second, and third correlation values that is determined to be lowest.

8. The computer-implemented method of claim 7, further comprising:

excluding the acoustic signal from the first microphone when a ratio of the correlation value of the first microphone to the correlation value of the selected reference microphone satisfies a predetermined criteria;

excluding the acoustic signal from the second microphone when a ratio of the correlation value of the second microphone to the correlation value of the selected reference microphone satisfies the predetermined criteria; and

excluding the acoustic signal from the third microphone when a ratio of the correlation value of the third microphone to the correlation value of the selected reference microphone satisfies the predetermined criteria.

9. The computer-implemented method of claim 5, further comprising localizing the acoustic source, at least in part, by computing a Valin-Michaud-Rouat-Letourneau (VMRL) direction finding algorithm.

10. A system comprising:

a plurality of sensors to detect a sound emanating from an acoustic source in an environment, the plurality of sensors including at least a first sensor, a second sensor and a third sensor;

a time-difference-of-arrival estimation module coupled to receive, from the plurality of sensors, signals indicative of a detected sound, wherein the time-difference-of-arrival estimation module is configured to;

generate multiple sets of time-difference-of-arrival (TDOA) data;

associate the first sensor as a first reference sensor with a first set of the multiple sets of TDOA data;

associate the second sensor as a second reference sensor with a second set of the multiple sets of TDOA data, wherein the first reference sensor is different from the second reference sensor;

associate the third sensor as a third reference sensor with a third set of the multiple sets of TDOA data; and

select, based on the multiple sets of TDOA data, one of the first, second or third sensors to be a reference sensor for the detected sound.

11. The system of claim 10, wherein the TDOA estimation module is further configured to compute correlation sums for the first, second and third set of the multiple sets of the TDOA data and select, as the reference sensor for the detected sound, one of the first, second or third sensors associated with the first, second or third set of the multiple sets of the TDOA data that yields the lowest correlation sum.

12. The system of claim 10, further comprising a TDOA localization module configured to localize the acoustic source in the environment using, at least in part, the reference sensor for the detected sound and the associated set of the first, second or third sets of the multiple sets of the TDOA data.

13. The system of claim 10, wherein the TDOA estimation module is further configured to determine whether to exclude a signal from a particular one of the first, second or third sensors as a function of a ratio of a correlation sum of the particular one sensor to a correlation sum of the reference sensor for the detected sound.

18. The system of claim 10, wherein associating the first sensor as the first reference sensor is predetermined.

19. The system of claim 10, wherein the second reference sensor is different from the third reference sensor.

20. The system of claim 10, wherein the first set of the multiple sets of the TDOA data is derived from time differences between the acoustic signals of the first sensor and the second sensor relative to the acoustic signal of the third sensor, wherein the second set of TDOA data is derived from time differences between the acoustic signals of the first sensor and the third sensor relative to the acoustic signal of the second sensor, and wherein the third set of TDOA data is derived from time differences between the acoustic signals of the second sensor and the third sensor relative to the acoustic signal of the first sensor.

14. A system comprising:

a plurality of sensors to detect a sound emanating from an acoustic source in an environment; and

a time-difference-of-arrival estimation module coupled to receive, from the plurality of sensors, signals indicative of the detected sound and configured to generate multiple sets of time-difference-of-arrival (TDOA) data, wherein each of the sets of TDOA data chooses a different sensor from the plurality of sensors to be a reference sensor, and to evaluate the multiple sets of TDOA data to select one of the sensors to be the reference sensor; and

a TDOA localization module configured to localize the acoustic source in the environment using, at least in part, the reference sensor and an associated set of the TDOA data, the TDOA localization module finding a direction to the acoustic source by computing a matrix M as follows;

15. The system of claim 14, wherein the TDOA localization module further computes the inverse matrix M⁻

1.

16. The system of claim 15, wherein the TDOA localization module further computes the M matrices and the inverse matrices M⁻

1 on demand for each new set of inputs.

17. The system of claim 15, wherein the TDOA localization module accesses a codebook that maintains the M matrices and the inverse matrices M⁻

1.