Method for enhancing audio signal using phase information

• US 9,881,631 B2
• Filed: 02/12/2015
• Issued: 01/30/2018
• Est. Priority Date: 10/21/2014
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Erdogan, Hakan, Hershey, John, Watanabe, Shinji, Le Roux, Jonathan
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US14/620,526
• Publication Number:
  US 20160111108A1
• Time in Patent Office: 1,083 Days
• Field of Search: None
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0324 Details of processing therefor
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: G10L 25/30 using neural networks

Abstract

A method transforms a noisy audio signal to an enhanced audio signal, by first acquiring the noisy audio signal from an environment. The noisy audio signal is processed by an enhancement network having network parameters to jointly produce a magnitude mask and a phase estimate. Then, the magnitude mask and the phase estimate are used to obtain the enhanced audio signal.

Claims

1. A method for transforming a noisy audio signal to an enhanced audio signal, comprising steps:

acquiring the noisy audio signal from an environment;

inputting the noisy audio signal to a deep neural network having network parameters to produce a magnitude mask and a phase estimate, wherein the deep neural network is a deep recurrent neural network (DRNN), a bidirectional long short-term memory (BLSTM) deep recurrent neural network (DRNN) or a long short-term memory (LSTM) network, wherein the deep neural network uses a phase-sensitive objective function based on an error in a complex spectrum that includes an error in amplitude and a phase of the noisy audio signal;

using the magnitude mask and the phase estimate to obtain the enhanced audio signal, wherein the steps are performed in a processor.

2. The method of claim 1, wherein the phase estimate is obtained directly through the deep neural network.

3. The method of claim 1, wherein the phase estimate is jointly obtained with an amplitude of the noisy audio signal using a complex valued mask.

4. The method of claim 1, wherein

the step of inputting.

11. The method of claim 1, wherein the deep neural network is the LSTM network when a system is online applications.

12. The method of claim 1, wherein the deep neural network is the BLSTM network when the system is non-online applications.

5. An audio signal transformation system comprising:

a sound detecting device configured to acquire a noisy audio signal from an environment;

a signal input interface device configured to receive and transmit the noisy audio signal;

an audio signal processing device configured to process the noisy audio signal, wherein the audio signal processing device comprises;

a processor configured to connected to a memory, the memory being configured to input/output data, wherein the processor executes the steps of;

inputting the noisy audio signal to a deep neural network having network parameters to produce a magnitude mask and a phase estimate, wherein the deep neural network is a bidirectional long short-term memory (BLSTM) deep recurrent neural network (DRNN) or a long short-term memory (LSTM) network, wherein the deep neural network uses a phase-sensitive objective function based on an error in a complex spectrum that includes an error in amplitude and a phase of the noisy audio signal;

using the magnitude mask and the phase estimate to obtain an enhanced audio signal, and

a signal output device configured to output the enhanced audio signal.

6. The audio signal transformation system of claim 5, wherein the phase estimate is obtained directly through the deep neural network.

7. The audio signal transformation system of claim 5, wherein the phase estimate is jointly obtained with the amplitude of the noisy audio signal using a complex valued mask.

8. The audio signal transformation system of claim 5, wherein the deep neural network is the LSTM network when the system is online applications.

9. The audio signal transformation system of claim 5, wherein the deep neural network is the BLSTM network when the system is non-online applications.

10. The audio signal transformation system of claim 5, wherein the input step jointly produces the magnitude mask and the phase estimate.