Determining speaker direction using a spherical microphone array

• US 9,560,441 B1
• Filed: 12/24/2014
• Issued: 01/31/2017
• Est. Priority Date: 12/24/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Leutnant, Volker Sebastian, Garimell, Sri Venkata Surya Siva Rama Krishna, Matsoukas, Spyridon, McDonough, John Walter Jr.
• Primary Examiner(s): Sniezek, Andrew L
• Application Number:
  US14/582,305
• Time in Patent Office: 769 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 3/803 using amplitude comparison of signals derived from receiving transducers or transducer systems having differently-oriented directivity characteristics
• CPC Class Codes: G01S 3/805 using adjustment of real or effective orientation of directivity characteristics of a transducer or transducer system to give a desired condition of signal derived from that transducer or transducer system, e.g. to give a maximum or minimum signal
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: H04R 1/20 Arrangements for obtaining desired frequency or directional characteristics for stereophonic purpose H04R5/00
• CPC Class Codes: H04R 1/32 for obtaining desired directional characteristic only specially adapted for hearing aids H04R25/40
• CPC Class Codes: H04R 1/326 for microphones H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 25/407 Circuits for combining signals of a plurality of transducers
• CPC Class Codes: H04R 29/005 Microphone arrays
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads

Abstract

A system that detects audio including speech using a spherical sensor array estimates a direction of arrival of the speech using a Kalman filter. To improve the estimates of the Kalman filter, the system estimates a noise covariance matrix, representing noise detected by the array. The structure of the noise covariance matrix is determined, using an assumption of spherically isotropic diffuse noise. The intensity of the noise covariance matrix is estimated based on the intensity of audio detected by the array.

Claims

1. A computer-implemented method for estimating a direction of arrival of detected speech using a spherical microphone array, the computer-implemented method comprising:

determining, at a first time, a first noise covariance matrix comprising a plurality of first values, wherein each of the first values describes a covariance between a pair of microphones detecting diffuse noise, wherein the microphones are part of a spherical microphone array;

detecting, at a second time after the first time, audio using the spherical microphone array;

determining a signal from the audio using the spherical microphone array, wherein the signal comprises a speech component and a noise component and the signal has a signal intensity calculated from pressure detected by the spherical microphone array;

estimating, based on the signal intensity, an estimated noise intensity of the noise component;

multiplying the estimated noise intensity by the plurality of first values to obtain a plurality of second values, wherein a second noise covariance matrix includes the plurality of second values; and

estimating a direction of arrival of the speech component relative to the spherical microphone array using an extended Kalman filter and the second noise covariance matrix.

2. The computer-implemented method of claim 1, wherein determining the first noise covariance matrix comprises:

determining an angular offset between a first position of a first microphone on the spherical microphone array and a second position of a second microphone on the spherical microphone array;

determining a radius of the spherical microphone array;

determining a wave number of diffuse noise;

calculating a first covariance value using the angular offset, the radius and the wave number; and

including the first covariance value as one of the plurality of first values.

3. The computer-implemented method of claim 1, wherein estimating the estimated noise intensity comprises:

determining a previous estimated noise intensity;

adjusting the previous estimated noise intensity by a first weighting factor to determine a weighted previous estimated noise intensity;

adjusting the signal intensity by a second weighting factor to determine a weighted signal intensity; and

calculating the estimated noise intensity by adding the weighted previous estimated noise intensity to the weighted signal intensity.

4. The computer-implemented method of claim 1, wherein estimating the estimated noise intensity {circumflex over (α

)} at time K comprises evaluating the equation;

5. A system comprising:

a sensor array comprising a plurality of sensors, the plurality of sensors configured to detect sound;

at least one processor; and

a memory including instruction operable to be executed by the at least one processor to perform a set of actions comprising;

determining an audio signal using the sensor array, the audio signal having a signal intensity and comprising a speech component and a noise component;

estimating, based on the signal intensity, an estimated noise intensity of the noise component;

determining a first noise covariance matrix comprising a plurality of first values, wherein each of the first values describes a covariance between a pair of sensors of the sensor array;

multiplying the estimated noise intensity by the first noise covariance matrix to obtain a second noise covariance matrix; and

estimating a direction of arrival of the speech component relative to the sensor array using a Kalman filter and the first noise covariance matrix and the second noise covariance matrix.

6. The system of claim 5, wherein the sensor array is generally spherical and the plurality of sensors are located proximate to a surface of the sensor array.

7. The system of claim 6, wherein determining the first noise covariance matrix comprises:

determining an angular offset between a first position of a first sensor of the sensor array and a second position of a second sensor of the sensor array;

determining a radius of the sensor array;

determining a wave number of diffuse noise;

calculating a first covariance value using the angular offset, the radius and the wave number; and

including the first covariance value as one of the plurality of first values.

8. The system of claim 5, wherein estimating the estimated noise intensity comprises:

determining a previous estimated noise intensity;

adjusting the previous estimated noise intensity by a first weighting factor to determine a weighted previous estimated noise intensity;

adjusting the signal intensity by a second weighting factor to determine a weighted signal intensity; and

calculating the estimated noise intensity by adding the weighted previous estimated noise intensity to the weighted signal intensity.

9. The system of claim 5, wherein the estimated noise intensity {circumflex over (α

)} at time K is given by the equation;

10. The system of claim 5, wherein:

first noise covariance matrix is determined at a first time;

the audio signal is detected at a second time; and

the second time is after the first time.

11. The system of claim 5, wherein the Kalman filter is an extended Kalman filter.

12. A computer-implemented method, comprising:

determining an audio signal using a sensor array, the sensor array comprising a plurality of sensors to detect sound and the audio signal having a signal intensity and comprising a speech component and a noise component;

estimating, based on the signal intensity, an estimated noise intensity of the noise component;

determining a first noise covariance matrix comprising a plurality of first values, wherein each of the first values describes a covariance between a pair of sensors of the sensor array;

multiplying the estimated noise intensity by the first noise covariance matrix to obtain a second noise covariance matrix; and

estimating a direction of arrival of the speech component relative to the sensor array using a Kalman filter and the first noise covariance matrix and the second noise covariance matrix.

13. The computer-implemented method of claim 12, wherein the sensor array is generally spherical and the plurality of sensors are located proximate to a surface of the sensor array.

14. The computer-implemented method of claim 13 wherein determining the first noise covariance matrix comprises:

determining an angular offset between a first position of a first sensor of the sensor array and a second position of a second sensor of the sensor array;

determining a radius of the sensor array;

determining a wave number of diffuse noise;

calculating a first covariance value using the angular offset, the radius and the wave number; and

including the first covariance value as one of the plurality of first values.

15. The computer-implemented method of claim 12, wherein estimating the estimated noise intensity comprises:

determining a previous estimated noise intensity;

adjusting the previous estimated noise intensity by a first weighting factor to determine a weighted previous estimated noise intensity;

adjusting the signal intensity by a second weighting factor to determine a weighted signal intensity; and

calculating the estimated noise intensity by adding the weighted previous estimated noise intensity to the weighted signal intensity.

16. The computer-implemented method of claim 12, wherein the estimated noise intensity {circumflex over (α

)} at time K is given by the equation;

17. The computer-implemented method of claim 12, wherein:

first noise covariance matrix is determined at a first time;

the audio signal is detected at a second time; and

the second time is after the first time.

18. The computer-implemented method of claim 12, wherein the Kalman filter is an extended Kalman filter.