Sound source localization using multiple microphone arrays

• US 8,983,089 B1
• Filed: 11/28/2011
• Issued: 03/17/2015
• Est. Priority Date: 11/28/2011
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Chu, Wai C., Crump, Edward Dietz
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): HAMID, AMMAR T
• Application Number:
  US13/305,189
• Time in Patent Office: 1,205 Days
• Field of Search: 381/92, 381/58, 381/71.1, 381/19, 381/23, 455/456.1, 455/563, 348/14.08, 348/14.01, 348/14.11, 700/94, 700/258
• US Class Current: 381/92
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04R 1/08 Mouthpieces; Microphones; Attachments therefor
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2420/03 Connection circuits to selectively connect loudspeakers or headphones to amplifiers
• CPC Class Codes: H04R 2430/21 Direction finding using differential microphone array [DMA]
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

An augmented reality environment allows interaction between virtual and real objects. Multiple microphone arrays of different physical sizes are used to acquire signals for spatial tracking of one or more sound sources within the environment. A first array with a larger size may be used to track an object beyond a threshold distance, while a second array having a size smaller than the first may be used to track the object up to the threshold distance. By selecting different sized arrays, accuracy of the spatial location is improved.

Claims

1. A system comprising:

one or more processors;

a first microphone array comprising a first plurality of microphones in a first arrangement, the first plurality of microphones being coupled to the one or more processors and configured to generate first signal data from a sound source;

a second microphone array comprising a second plurality of microphones in a second arrangement, the second plurality of microphones being coupled to the one or more processors and configured to generate second signal data from the sound source;

a localization module executable by the one or more processors to;

determine a first spatial location of the sound source based at least in part on the first signal data, the first spatial location of the sound source being calculated as a distance from an origin; and

based at least in part on a determination that the distance from the origin is below a threshold value, determine a second spatial location of the sound source based at least in part on the second signal data.

2. The system of claim 1, wherein the first arrangement encompasses an area greater than the second arrangement.

3. The system of claim 1, further comprising additional microphone arrays, the additional microphone arrays disposed such that one or more additional microphone arrays of the additional microphone arrays are at least one of disposed within at least a portion of the first microphone array, disposed within at least a portion of the second microphone array, or disposed within at least a portion of the first microphone array and the second microphone array.

4. The system of claim 1, wherein at least a portion of the first microphone array encompasses at least a portion of the second microphone array.

5. The system of claim 1, wherein the second microphone array is disposed at least partially within the first microphone array and at least partially outside of the first microphone array.

6. The system of claim 1, wherein at least one microphone is shared between the first microphone array and the second microphone array.

7. The system of claim 1, wherein the spatial location is determined based at least in part on a time difference of arrival between two microphones, and wherein each of the two microphones belong to one or more of the first plurality of microphones or the second plurality of microphones.

8. The system of claim 1, wherein the first plurality of microphones, the second plurality of microphones, or both are disposed equidistant from one another around a perimeter of a symmetric quadrilateral shape.

9. The system of claim 8, wherein the first plurality of microphones, the second plurality of microphone, or both are disposed in a single plane.

10. A system comprising:

three or more microphones configured such that each microphone is individually addressable and configured to acquire data from a sound source; and

an array selection module configured to access the data acquired by a first subset of the three or more microphones and a second, different subset of the three or more microphones; and

a localization module coupled to one or more processors and configured to;

determine a spatial location of the sound source based at least in part on the data from the first subset of microphones; and

based at least in part on a determination that one or more attributes of the sound source are below a threshold value, determine the spatial location of the sound source based at least in part on the data from the second subset of microphones.

11. The system of claim 10, wherein at least a portion of the second subset is disposed within the first subset.

12. The system of claim 10, wherein at least one microphone of the three or more microphones is excluded from both the first subset and the second subset.

13. The system of claim 10, wherein each of the three or more microphones is configured to have substantially a same response characteristics.

14. The system of claim 10, wherein at least a portion of the three or more microphones are configured to move from one location to another location.

15. The system of claim 10, wherein the first subset, the second subset, or both comprise the three or more microphones disposed equidistant from one another around a perimeter of a symmetric quadrilateral shape.

16. The system of claim 10, wherein the three or more microphones are disposed in a single plane.

17. The system of claim 10, wherein at least one microphone is shared between the first subset and the second subset.

18. The system of claim 10, wherein the one or more attributes comprise at least one of a determined spatial location of the sound source, a distance of the sound source from a pre-determined point, or a signal-to-noise-ratio of a signal in the data.

19. The system of claim 10, wherein the spatial location is determined based at least in part on a time difference of arrival between two or more of the three or more microphones.

24. The system of claim 10, wherein the second subset comprises microphones physically disposed in a spatial arrangement encompassing an area that is less than an area defined by a spatial arrangement of the first subset.

20. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

retrieving first signal data of a signal source acquired at a first microphone array;

determining, from the first signal data, a first distance to the signal source relative to a pre-determined point;

based at least in part on a determination that the first distance exceeds a threshold distance, determining a first location of the source based at least in part on the first signal data; and

based at least in part on a determination that the first distance is less than the threshold distance;

retrieving second signal data of the signal source acquired at a second microphone array; and

determining, from the second signal data, a second location of the signal source based at least in part on the second signal data.

21. The one or more non-transitory computer-readable storage media of claim 20, wherein the first signal data and the second signal data are acquired at substantially a same time.

22. The one or more non-transitory computer-readable storage media of claim 20, the acts further comprising determining the threshold distance at least partly by:

determining a candidate location of the signal source using a depth sensor; and

setting the threshold distance based at least in part upon a correspondence of the candidate location to the first location or second location.

23. The one or more non-transitory computer-readable storage media of claim 22, wherein the depth sensor comprises a projector configure to project structured light.