Audio processing apparatus and method

• US 8,275,148 B2
• Filed: 07/28/2009
• Issued: 09/25/2012
• Est. Priority Date: 07/28/2009
• Status: Active Grant

• Current Assignee: Fortemedia, Inc.
• Original Assignee: Fortemedia, Inc.
• Inventors: Li, Xi-Lin, Liu, Sheng
• Primary Examiner(s): Tsang, Fan
• Assistant Examiner(s): Zhao, Eugene
• Application Number:
  US12/510,449
• Publication Number:
  US 20110026730A1
• Time in Patent Office: 1,155 Days
• Field of Search: 381 91- 949, 381/98, 381/66, 381/83, 702/189, 702/190, 702/194, 702/196, 704/233, 704/226
• US Class Current: 381/92
• CPC Class Codes: H04R 2410/05 Noise reduction with a separate noise microphone
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

An audio processing apparatus is provided, comprising: a main microphone for receiving sounds from a source and noises from non-source sources and generating a main input; a reference microphone for receiving the sounds and the noises and generating a reference input; a short-time Fourier transformation (STFT) unit for applying short time Fourier transformation to convert the main input of a time domain signals into a main signal of a frequency domain and convert the reference input of the time domain signals into a reference signal of the frequency domain; a sensitivity calibrating unit for performing sensitivity calibration on the main signal and the reference signal and generating a main calibrated signal and a reference calibrated signal; and a voice active detector (VAD) for generating a voice active signal according to the main calibrated signal, the reference calibrated signal and a direction of arrival (DOA) signal.

Claims

1. An audio processing apparatus, comprising:

a main microphone for receiving sounds from a source and noises from non-source sources and generating a main input;

a reference microphone for receiving the sounds and the noises and generating a reference input;

a short-time Fourier transformation (STFT) unit for applying short time Fourier transformation to convert the main input of a time domain signals into a main signal of a frequency domain and convert the reference input of the time domain signals into a reference signal of the frequency domain;

a sensitivity calibrating unit for performing sensitivity calibration on the main signal and the reference signal and generating a main calibrated signal and a reference calibrated signal;

a voice active detector (VAD) for generating a voice active signal according to the main calibrated signal, the reference calibrated signal and a direction of arrival (DOA) signal; and

a beamformer for converting the main calibrated signal into a main channel and converting the reference calibrated signal into a reference channel according to the voice active signal.

2. The audio processing apparatus as claimed in claim 1, wherein the main microphone is disposed closer to the source than the reference microphone.

3. The audio processing apparatus as claimed in claim 1, wherein the sensitivity calibrating unit further comprises a spatial spectrum estimator for generating a spatial spectrum according to the main signal and the reference signal, wherein the spatial spectrum depicts a functional relationship between power distribution and angles of incident of the main signal and the reference signal, where a substantially flat curve in the spatial spectrum is caused by far field noises, and sharp and dominant peaks in the spatial spectrum is caused by near field sounds of a speaker'"'"'s voice and spot noises from the environment.

4. The audio processing apparatus as claimed in claim 3, wherein the sensitivity calibrating unit further comprises a diffuse noise detector for inspecting the spatial spectrum to indicate whether diffuse noises exist or not.

5. The audio processing apparatus as claimed in claim 4, wherein the sensitivity calibrating unit further comprises a sensitivity mismatch calculator for calculating a sensitivity mismatch between the main signal and the reference signal when the diffuse noise detector indicates that the diffuse noises exist.

6. The audio processing apparatus as claimed in claim 5, wherein the sensitivity calibrating unit further comprises a sensitivity mismatch remover used for receiving the main signal and the reference signal, removing the sensitivity mismatch between the main signal and the reference signal and generating the main calibrated signal and the reference calibrated signal.

7. The audio processing apparatus as claimed in claim 3, further comprising, a DOA estimator for inspecting the spatial spectrum and generating the DOA signal D1, wherein the DOA signal D1 indicates whether there is a dominant peak in the spatial spectrum.

8. The audio processing apparatus as claimed in claim 1, wherein the VAD compares a power ratio between the main calibrated signal and the reference calibrated signal with a predetermined threshold;

where the voice active signal will be turned on when the power ratio is larger than the pre-defined threshold, and the voice active signal will be turned off when the power ratio is smaller than the pre-defined threshold.

9. The audio processing apparatus as claimed in claim 1, wherein the beamformer further comprises an array manifold matrix identification unit for tracking signal subspace and generating a steering vector signal according to the voice active signal.

10. The audio processing apparatus as claimed in claim 9, wherein the beamformer further comprises:

a main channel generator for receiving the main calibrated signal and the reference calibrated signal and generating the main channel according to the steering vector signal, wherein the main channel is corresponding to the sounds received from the source; and

a reference channel generator for receiving the main calibrated signal and the reference calibrated signal and generating the reference channel according to the steering vector signal, wherein the reference channel is corresponding to the noises received from non-source sources.

11. The audio processing apparatus as claimed in claim 1, further comprising, a noise suppressing unit used for suppressing stationary and non-stationary noises in the main channel and the reference channel according to the voice active signal and integrating the main channel and the reference channel into a final signal.

12. The audio processing apparatus as claimed in claim 11, further comprising, an inverse STFT unit for applying inverse short time Fourier transformation to convert the final signal of the frequency domain signals into a final output of the time domain.

13. The audio processing apparatus as claimed in claim 9, wherein the array manifold matrix identification unit uses a projection approximation subspace tracking (PAST) algorithm.

14. The audio processing apparatus as claimed in claim 10, wherein the main channel generator and the reference channel generator use a minimal variance distortionless response (MVDR) beamforming method to generate the main channel and the reference channel.

15. The audio processing apparatus as claimed in claim 11, wherein the noise suppressing unit is a Wiener post filter.

16. An audio processing method, comprising:

receiving sounds from a source and noises from non-source sources and generating a main input;

receiving the sounds and the noises and generating a reference input;

applying short time Fourier transformation to convert the main input of a time domain signals into a main signal of a frequency domain and convert the reference input of the time domain signals into a reference signal of the frequency domain;

performing sensitivity calibration on the main signal and the reference signal and generating a main calibrated signal and a reference calibrated signal;

generating a voice active signal according to the main calibrated signal, the reference calibrated signal and a direction of arrival (DOA) signal; and

converting the main calibrated signal into a main channel and converting the reference calibrated signal into a reference channel according to the voice active signal.

17. The audio processing method as claimed in claim 16, further comprising, generating a spatial spectrum according to the main signal and the reference signal, wherein the spatial spectrum depicts a functional relationship between power distribution and angles of incident of the main signal and the reference signal, where a substantially flat curve in the spatial spectrum is caused by far field noises, and sharp and dominant peaks in the spatial spectrum is caused by near field sounds of a speaker'"'"'s voice and spot noises from the environment.

18. The audio processing method as claimed in claim 17, further comprising, inspecting the spatial spectrum to indicate whether diffuse noises exist or not.

19. The audio processing method as claimed in claim 18, further comprising, calculating a sensitivity mismatch between the main signal and reference signal when the diffuse noise detector indicates that the diffuse noises exist.

20. The audio processing method as claimed in claim 19, further comprising, removing a sensitivity mismatch between a main signal and a reference signal and generating a main calibrated signal and the reference calibrated signal.

21. The audio processing method as claimed in claim 17, further comprising, inspecting the spatial spectrum and generating the DOA signal D1, wherein the DOA signal D1 indicates whether there is a dominant peak in the spatial spectrum.

22. The audio processing method as claimed in claim 21, further comprising, comparing power ratio between the main calibrated signal and the reference calibrated signal with a predetermined threshold;

where the voice active signal will be turned on when the power ratio is larger than the pre-defined threshold, and the voice active signal will be turned off when the power ratio is smaller than the pre-defined threshold.

23. The audio processing method as claimed in claim 16, further comprising, tracking signal subspace and generating a steering vector signal according to the voice active signal.

24. The audio processing method as claimed in claim 23, further comprising, receiving the main calibrated signal and the reference calibrated signal and generating the main channel and the reference channel according to the steering vector signal, wherein the main channel is corresponding to the sounds received from the source, and the reference channel is corresponding to the noises received from non-source sources.

25. The audio processing method as claimed in claim 16, further comprising, suppressing stationary and non-stationary noises in the main channel and the reference channel according to the voice active signal and integrating the main channel and the reference channel into a final signal.

26. The audio processing method as claimed in claim 25, further comprising, applying inverse short time Fourier transformation to convert the final signal of the frequency domain signals into a final output of the time domain.