Time difference of arrival determination with direct sound

• US 9,194,938 B2
• Filed: 06/24/2011
• Issued: 11/24/2015
• Est. Priority Date: 06/24/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Velusamy, Kavitha
• Primary Examiner(s): Alsomiri, Isam
• Assistant Examiner(s): Baghdasaryan, Hovhannes
• Application Number:
  US13/168,759
• Publication Number:
  US 20120327746A1
• Time in Patent Office: 1,614 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 3/86 with means for eliminating undesired waves, e.g. disturbing noises
• CPC Class Codes: G01S 5/22 Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements G01S5/28 takes precedence

Abstract

Acoustic signals may be localized such that their position in space is determined. Time-difference-of-arrival data from multiple microphones may be used for this localization. Signal data from the microphones may be degraded by reverberation and other environmental distortions, resulting in erroneous localization. By detecting a portion of the signal resulting from sound directly reaching a microphone rather than from a reverberation, accuracy of the localization is improved.

Claims

1. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

acquiring, from a plurality of microphones, a plurality of audio signals associated with an acoustic source;

filtering the plurality of audio signals with a band-pass filter;

estimating a noise floor of the plurality of audio signals;

for individual audio signals of the plurality of audio signals;

identifying an event of interest;

determining that the event of interest rises above the noise floor;

adjusting the noise floor at a rate that is based at least partly on an extent to which the event of interest rises above the noise floor;

detecting a peak after the event of interest rose above the noise floor; and

determining a time at which the peak occurs; and

determining time-difference-of-arrival (TDOA) values for the event of interest based at least in part on a difference between the time at which the peaks occur in the individual audio signals.

2. The one or more non-transitory computer-readable storage media of claim 1, wherein the band-pass filter is configured to have a bandwidth extending from about 800 Hertz to about 2 Kilohertz.

3. The one or more non-transitory computer-readable storage media of claim 1, wherein the event of interest comprises an acoustic signal having a duration of less than about 250 milliseconds.

4. The one or more non-transitory computer-readable storage media of claim 3, wherein the acoustic signal is generated by a human gesture.

5. The one or more non-transitory computer-readable storage media of claim 1, the acts further comprising removing noise from the plurality of audio signals.

6. The one or more non-transitory computer-readable storage media of claim 1, the acts further comprising filtering the TDOA values based at least in part on one or more physical attributes of the plurality of microphones or a room in which the plurality of microphones reside.

7. The one or more non-transitory computer-readable storage media of claim 6, wherein the physical attributes comprise known distances between the plurality of microphones.

8. The one or more non-transitory computer-readable storage media of claim 6, wherein the physical attributes comprise known dimensions of the room.

9. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

acquiring a plurality of audio signals associated with an acoustic source;

for individual audio signals of the plurality of audio signals;

identifying an event of interest by identifying a portion of the audio signal that corresponds to direct sound from the acoustic source and disregarding portions of the audio signal that correspond to indirect sound from the acoustic source;

determining that the event of interest rises above a noise floor;

detecting a peak after the event of interest rose above the noise floor; and

determining a time at which the peak occurs; and

determining time-difference-of-arrival (TDOA) values for the event of interest based at least in part on a difference between the time at which the peaks occur in the plurality of filtered audio signals.

10. The one or more non-transitory computer-readable storage media of claim 9, the acts further comprising filtering the plurality of audio signals with a band-pass filter.

11. The one or more non-transitory computer-readable storage media of claim 10, wherein the band-pass filter is configured to have a bandwidth extending from about 800 Hertz to about 2 Kilohertz.

12. The one or more non-transitory computer-readable storage media of claim 9, the acts further comprising estimating the noise floor of the plurality of audio signals.

13. The one or more non-transitory computer-readable storage media of claim 9, the acts further comprising removing noise from the plurality of audio signals.

14. The one or more non-transitory computer-readable storage media of claim 9, wherein the event of interest further comprises an acoustic signal generated by the acoustic source having a duration of less than about 250 milliseconds.

15. The one or more non-transitory computer-readable storage media of claim 14, wherein the acoustic signal is generated by a user physically striking an object within an environment.

16. The one or more non-transitory computer-readable storage media of claim 9, the acts further comprising filtering the TDOA values based at least in part on at least one of one or more physical attributes of a plurality of microphones receiving the plurality of audio signals or physical attributes of a room in which the plurality of microphones reside.

17. The one or more non-transitory computer-readable storage media of claim 16, wherein the physical attributes comprise known distances between the plurality of microphones.

18. The one or more non-transitory computer-readable storage media of claim 16, wherein the physical attributes comprise known dimensions of the room.

19. A system comprising:

a plurality of sensors;

a time-difference-of-arrival module coupled to the sensors and configured to;

acquire, via the sensors, a plurality of signals associated with an acoustic source;

for individual audio signals of the plurality of signals;

identify an event of interest;

determine that the event of interest rises above a noise floor;

based at least partly on the event of interest rising above the noise floor, increase the noise floor at a first rate;

determine that the event of interest falls below the noise floor; and

based at least partly on the event of interest falling below the noise floor, decrease the noise floor at a second rate different from the first rate.

20. The system of claim 19, wherein the plurality of sensors comprise one or more microphones.

21. The system of claim 19, the time-difference-of-arrival module being further configured to:

filter the plurality of signals with a band-pass filter; and

estimate the noise floor of the plurality of signals.

22. The system of claim 19, wherein the time-difference-of-arrival module is further configured to:

detect a peak after the event of interest rose above the noise floor;

determine a time at which the peak occurs;

determine time-difference-of-arrival (TDOA) values based at least in part on a difference between the time at which the peaks of the signals occur; and

localize the source based at least in part upon the determined TDOA values.

23. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

acquiring, from a plurality of microphones, a plurality of audio signals associated with an acoustic source generated by a human speaker;

estimating a noise floor of the plurality of audio signals; and

for individual audio signals of the plurality of audio signals;

identifying an event of interest that comprises human speech;

determining a time that the event of interest rises above the noise floor;

designating a window of samples starting within a threshold amount of time from the time that the event of interest rises above the noise floor; and

adjusting the noise floor at a rate that is based at least partly on an extent to which the event of interest rises above the noise floor.

24. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising determining that a quiet period is present before the event of interest, wherein the quiet period is less than about 100 milliseconds in duration.

25. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising determining that a quiet period is present before the event of interest, wherein the quiet period comprises a period during which any audio signals are at or below the noise floor.

26. The one or more non-transitory computer-readable storage media of claim 23, wherein the window of samples is less than about 2 milliseconds in duration.

27. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising filtering the plurality of audio signals with a band-pass filter configured to have a bandwidth extending from about 2 Kilohertz to about 8 Kilohertz.

28. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising removing noise from the plurality of audio signals.

29. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising:

calculating time-difference-of-arrival (TDOA) values based at least in part on samples within the window of samples;

sliding the window of samples forward by a time, t;

determining that the samples are within a region of interest that comprises at least a portion of a direct signal from the event of interest; and

again calculating TDOA values based at least in part upon the samples within the window after sliding the window forward by the time, t.

30. The one or more non-transitory computer-readable storage media of claim 29, the acts further comprising filtering the TDOA values based at least in part on at least one of one or more physical attributes of the plurality of microphones or of a room in which the plurality of microphones reside.

31. The one or more non-transitory computer-readable storage media of claim 30, wherein the physical attributes comprise known distances between the plurality of microphones.

32. The one or more non-transitory computer-readable storage media of claim 30, wherein the physical attributes comprise known dimensions of the room.

33. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising:

based at least partly on a determination that a stable set of TDOA from a pre-determined number of consecutive windows values is reached, localizing the acoustic source with use of the stable TDOA values.

34. The one or more non-transitory computer-readable storage media of claim 23, the acts further comprising continuously tracking the acoustic source.

35. The one or more non-transitory computer-readable storage media of claim 23, wherein identifying the event of interest that comprises human speech includes identifying a portion of the audio signal that corresponds to direct sound traveling directly from the acoustic source and disregarding portions of the audio signal that correspond to indirect sound traveling indirectly from the acoustic source.

36. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

acquiring, from a plurality of microphones, a plurality of audio signals associated with an acoustic source, wherein a portion of an audio signal of the plurality of audio signals is indicative of direct sound traveling directly from the acoustic source and portions of the audio signal are indicative of indirect sound traveling indirectly from the acoustic source;

calculating a time-difference-of-arrival value for the audio signal at least partly by disregarding the portions of the audio signal that are indicative of the indirect sound; and

determining a location of the acoustic source based at least in part upon the time-difference-of arrival value; and

for an audio signal of the plurality of audio signals;

estimating a noise floor of the audio signal;

identifying an event of interest that comprises human speech; and

adjusting the noise floor at a rate that is based at least partly on an extent to which the event of interest rises above the noise floor.

37. The one or more non-transitory computer-readable storage media of claim 36, wherein the portion of the audio signal that is indicative of the direct sound is received from the acoustic source free from reflection.

38. The one or more non-transitory computer-readable storage media of claim 36, wherein the calculating comprises applying a phase transform.

39. The one or more non-transitory computer-readable storage media of claim 36, wherein the calculating comprises applying a generalized cross correlation technique.

40. The one or more non-transitory computer-readable storage media of claim 36, wherein the determining the location further comprises filtering the time-difference-of-arrival value based at least in part upon one or more physical attributes of at least one of the plurality of the microphones or of a room in which the plurality of microphones reside.

41. The one or more non-transitory computer-readable storage media of claim 40, wherein the physical attributes comprise known distances between the plurality of microphones.

42. The one or more non-transitory computer-readable storage media of claim 40, wherein the physical attributes comprise known dimensions of the room.

43. A system comprising:

a plurality of sensors;

a time-difference-of-arrival module coupled to the sensors and configured to;

acquire, from a plurality of sensors, a plurality of audio signals associated with an acoustic source generated by a human speaker;

for individual ones of the plurality of the audio signals;

identify an event of interest that comprises human speech;

determine a time that the event of interest rises above a noise floor; and

adjusting the noise floor at a rate that is based at least partly on an extent to which the event of interest rises above the noise floor.

44. The system of claim 43, wherein the sensors comprise microphones.

45. The system of claim 43, wherein the event of interest comprises a portion of an audio signal that is received directly by a microphone free from reflection.

46. The system of claim 43, the time-difference-of-arrival module further configured to:

filter the plurality of audio signals with a band-pass filter; and

estimate the noise floor of the plurality of audio signals.

47. The system of claim 43, further comprising a wherein the time-difference-of-arrival module is further configured to:

determine that a quiet period is present before the event of interest;

designate a window of samples starting within a threshold amount of time from the time that the event of interest rises above the quiet period;

calculate time-difference-of-arrival (TDOA) values based at least in part on samples within the window of samples;

slide the window of samples forward by a time, t;

determine that the samples are within a region of interest that comprises at least a portion of a direct signal from the event of interest;

again calculate TDOA values based at least in part upon the samples within the window of samples after sliding the window forward by the time, t; and

localize the acoustic source based at least in part upon the calculated TDOA values.