Sound source localization using phase spectrum

• US 9,817,100 B2
• Filed: 08/19/2016
• Issued: 11/14/2017
• Est. Priority Date: 07/14/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Regunathan, Shankar, Koishida, Kazuhito, Kikkeri, Harshavardhana Narayana
• Primary Examiner(s): Lun-See, Lao
• Application Number:
  US15/241,332
• Publication Number:
  US 20170052245A1
• Time in Patent Office: 452 Days
• Field of Search: 381 92, 381309, 381 26, 381 56- 58, 381 74, 381310, 381356, 700 94
• CPC Class Codes: G01S 3/8006 Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G01S 3/82 with means for adjusting phase or compensating for time-lag errors
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

An array of microphones placed on a mobile robot provides multiple channels of audio signals. A received set of audio signals is called an audio segment, which is divided into multiple frames. A phase analysis is performed on a frame of the signals from each pair of microphones. If both microphones are in an active state during the frame, a candidate angle is generated for each such pair of microphones. The result is a list of candidate angles for the frame. This list is processed to select a final candidate angle for the frame. The list of candidate angles is tracked over time to assist in the process of selecting the final candidate angle for an audio segment.

Claims

1. A process for sound source localization with a plurality of pairs of microphones with known spatial relationship, comprising:

receiving signals from the plurality of pairs of microphones into a memory as a plurality of frames;

processing each frame of the signals from the plurality of pairs of microphones to identify when the received signals are active in the frame;

computing frequency spectrum data for each frame of the received signals;

for each pair of active signals in a frame, determining a candidate angle of sound arrival on the plurality of pairs of microphones using the frequency spectrum data; and

selecting, for a current frame, an angle of sound arrival on the plurality of pairs of microphone from among the candidate angles determined for the current frame, by;

tracking a history of candidate angles determined for multiple frames;

updating the history based on the candidate angles determined for the current frame; and

selecting, as the angle for the current frame, an angle from the history having a phase distortion less than or equal to a minimum phase distortion of the candidate angles and similar to a highest ranked candidate angle determined for the current frame.

2. The computer-implemented process of claim 1, wherein processing, computing, determining and selecting are performed on a per frame basis.

3. The computer-implemented process of claim 1, wherein selecting the angle from the history is further based on the selected candidate angle having a presence score greater than or equal to a maximum presence score of candidate angles in the history.

4. The process of claim 1, wherein the history comprises, for each candidate angle, a phase distortion, a presence score and a presence counter.

5. The process of claim 4, wherein updating the history comprises:

for candidate angles in the history other than the selected candidate angle for the current frame, decrementing the presence counter for the candidate angles.

6. The process of claim 5, wherein updating the history comprises:

for the selected candidate angle for the current frame, incrementing a presence counter for the selected candidate angle.

7. The process of claim 6, further comprising, in response to a determination that a presence counter for a selected candidate angle for the current frame exceeds a threshold, reporting the selected candidate angle for the current frame as a detected angle of sound arrival on the plurality of pairs of microphone.

8. The process of claim 4 wherein updating the history comprises:

for a target candidate angle in the history having a lowest phase distortion, updating the presence score for the target candidate angle based on a candidate angle for the current frame having an angle similar to the target candidate angle.

9. The process of claim 4 wherein updating the history comprises:

for the target candidate angle in the history having a lowest phase distortion, updating the phase distortion for the target candidate angle based on a candidate angle for the current frame having an angle similar to the target candidate angle.

10. The process of claim 4 wherein updating the history comprises:

for the target candidate angle in the history having a lowest phase distortion, updating the angle of the target candidate angle based on a candidate angle for the current frame having an angle similar to the target candidate angle.

11. A computing machine comprising:

a memory;

an input for receiving signals from a plurality of pairs of microphones into the memory as a plurality of frames;

a processing unit configured to process each frame of the received signals from the plurality of pairs of microphones to identify when the received signals are active in the frame and to compute frequency spectrum data for each frame of the received signals;

wherein the processing unit is further configured to, for each pair of active signals in a frame, determine a candidate angle of sound arrival on the plurality of pairs of microphones using the frequency spectrum data, and to select, for a current frame, an angle of sound arrival on the plurality of pairs of microphones from among the candidate angles determined for the current frame, by;

tracking a history of candidate angles determined for multiple frames;

updating the history based on the candidate angles determined for the current frame; and

selecting, as the angle for the current frame, an angle from the history having a phase distortion less than or equal to a minimum phase distortion of the candidate angles and similar to a highest ranked candidate angle determined for the current frame.

12. The computing machine of claim 11, wherein the processing unit is configured to process the signals on a per frame basis.

13. The computing machine of claim 11, wherein to select the angle from the history, the processing unit is further configured to select the candidate angle having a presence score greater than or equal to a maximum presence score of candidate angles in the history.

14. The computing machine of claim 11, wherein the history comprises, for each candidate angle, a phase distortion, a presence score and a presence counter.

15. The computing machine of claim 14, wherein updating the history comprises:

for candidate angles in the history other than the selected candidate angle for the current frame, decrementing the presence counter for the candidate angles.

16. The computing machine of claim 15, wherein updating the history comprises:

for the selected candidate angle for the current frame, incrementing a presence counter for the selected candidate angle.

17. The computing machine of claim 16, further comprising, in response to a determination that a presence counter for a selected candidate angle for the current frame exceeds a threshold, reporting the selected candidate angle for the current frame as a detected angle of sound arrival on the plurality of pairs of microphone.

18. The computing machine of claim 14 wherein updating the history comprises:

for a target candidate angle in the history having a lowest phase distortion, updating the presence score for the target candidate angle based on a candidate angle for the current frame having an angle similar to the target candidate angle.

19. The computing machine of claim 14 wherein updating the history comprises:

for the target candidate angle in the history having a lowest phase distortion, updating the phase distortion for the target candidate angle based on a candidate angle for the current frame having an angle similar to the target candidate angle.

20. The computing machine of claim 14 wherein updating the history comprises:

for the target candidate angle in the history having a lowest phase distortion, updating the angle of the target candidate angle based on a candidate angle for the current frame having an angle similar to the target candidate angle.