Direction of arrival estimation for multiple audio content streams

• US 10,455,325 B2
• Filed: 12/27/2018
• Issued: 10/22/2019
• Est. Priority Date: 12/28/2017
• Status: Active Grant

• Current Assignee: Knowles Electronics Llc (Knowles Corporation)
• Original Assignee: Knowles Electronics Llc (Knowles Corporation)
• Inventors: Woodruff, John, Shah, Pratik, Pulugurtha, Sai Ravi Teja, Rao, Harsha
• Primary Examiner(s): Kim, Paul
• Application Number:
  US16/234,509
• Publication Number:
  US 20190208317A1
• Time in Patent Office: 299 Days
• Field of Search: 381 92
• CPC Class Codes: G01S 3/80 using ultrasonic, sonic or infrasonic waves
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: G10L 25/30 using neural networks
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/04 for correcting frequency response

Abstract

A method for estimating directions of arrival of audio signals based on categories is disclosed herein. The method includes receiving an audio signal; generating a plurality of filters based on the audio signal, each of the filters corresponding to one of a plurality of sound content categories; separating the audio signal into a plurality of content-based audio signals by applying the filters to the audio signal, each of the content-based audio signals contains a content of a corresponding sound content category among the plurality of sound content categories; for each of the content-based audio signals, generating a direction of arrival (DOA) for a sound source of the content-based audio signal.

Claims

1. A method for estimating directions of arrival based on categories, comprising:

receiving an audio signal;

providing the audio signal to a neural network that has been trained using known sound content corresponding to the plurality of sound content categories;

generating, by the neural network, a plurality of filters based on the audio signal, each of the filters corresponding to one of the plurality of sound content categories; and

separating the audio signal into a plurality of category specific audio signals using the filters, the category specific audio signals respectively corresponding to the plurality of sound content categories; and

for each of the category specific audio signals, generating a direction of arrival (DOA) for a sound source of the category specific audio signal.

2. The method of claim 1, wherein generating the DOA further includes, for each of the category specific audio signals, generating a probability value indicating a likelihood of the DOA of the sound source.

3. The method of claim 2, further comprising extracting a plurality of features from the audio signal, wherein generating the DOA for the sound source and the probability value is further based on the extracted features.

4. The method of claim 1, wherein generating the DOA comprises:

extracting a plurality of features from the audio signal;

generating one or more estimated DOAs for one or more sound source directions and probabilities associated with the estimated DOAs based on the extracted features; and

selecting the DOA from the one or more estimated DOAs by comparing the probabilities associated with the estimated DOAs.

5. The method of claim 1, further comprising:

tracking the DOA of the sound source of the category specific audio signal across multiple time frames.

6. The method of claim 5, wherein the DOA of the sound source of the category specific audio signal is tracked using a Kalman filter.

7. The method of claim 1, wherein the method further comprises generating a plurality of features from the audio signal, and wherein generating the plurality of filters is further based on the generated features.

8. The method of claim 1, further comprising converting the audio signal from a time domain to a frequency domain, wherein the plurality of filters comprise frequency domain filters, and wherein the plurality of filters are applied to the frequency domain representation of the audio signal.

9. The method of claim 8, further comprising converting the plurality of category specific audio signals from the frequency domain into the time domain.

10. The method of claim 1, wherein the neural network has been trained by:

combining at least a first training signal of a first known sound content category and a second training signal of a second known sound content category into a combined training audio signal; and

training the neural network by feeding the combined training audio signal into the neural network and optimizing parameters of the neural network.

11. The method of claim 10, wherein optimizing parameters of the neural network includes comparing a frequency mask generated by the neural network to an optimal frequency mask associated with one of the first and second audio signals.

12. The method of claim 10, wherein the first and second training signals are clean signals having sound content corresponding the first and second known sound content categories, respectively.

13. The method of claim 1, wherein each of the filters is a time-varying real-valued function of frequency, and a value of the time-varying real-valued function for a corresponding frequency and a corresponding time frame represents a level of signal attenuation for the corresponding frequency at the corresponding time frame.

14. The method of claim 13, wherein the separating the audio signal into a plurality of category specific audio signals by applying the filters to the audio signal comprises:

separating the audio signal into a plurality of category-specific audio signals by multiplying the audio signal by the time-varying real-valued functions.

15. The method of claim 1, further comprising:

reproducing a virtual reality sound stage using the category specific audio signals and the DOAs of the sound sources.

16. A method of reproducing a virtual sound environment, comprising:

receiving an audio signal;

generating a plurality of filters based on the audio signal, each of the filters corresponding to one of a plurality of sound content categories, the sound content categories including a target sound content category and one or more ambient sound content categories; and

separating the audio signal into a plurality of category specific audio signals using the filters, the category specific audio signals includes a target category specific audio signal of the target sound content category and one or more ambient category specific audio signals of the ambient sound content categories;

for each of the category specific audio signals, generating a direction of arrival (DOA) for a sound source of the category specific audio signal and a probability value indicating a likelihood of the DOA of the sound source; and

reproducing a virtual sound environment by replaying the category specific audio signals using a plurality of speakers, wherein for each of the category specific audio signals, designation of one or more of the speakers that replay the category specific audio signal and determination of replay parameters are based on the DOA of the sound source of the category specific audio signal.

17. The method of claim 16, wherein generating the plurality of filters is performed using a neural network that has been trained using known sound content corresponding to the plurality of sound content categories.

18. The method of claim 17, wherein the neural network has been trained by:

combining a first training signal of the target sound content category and a second training signal of one of the ambient sound content categories into a combined training audio signal; and

training the neural network by feeding the combined training audio signal into the neural network and optimizing parameters of the neural network.

19. The method of claim 18, wherein optimizing parameters of the neural network includes comparing a frequency mask generated by the neural network to an optimal frequency mask associated with one of the first and second training signals.

20. The method of claim 18, wherein the first and second training signals are clean signals having sound content corresponding to the first and second known sound content categories, respectively.