Audio Signal Processing Method and Apparatus and Differential Beamforming Method and Apparatus

• US 20160173978A1
• Filed: 02/22/2016
• Published: 06/16/2016
• Est. Priority Date: 09/18/2013
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Li, Haiting, Zhang, Deming
• Granted Patent: US 9,641,929 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/025 Transducer mountings or cabinet supports enabling variable orientation of transducer of cabinet
• CPC Class Codes: H04R 2201/405 Non-uniform arrays of transducers or a plurality of uniform arrays with different transducer spacing
• CPC Class Codes: H04R 2430/21 Direction finding using differential microphone array [DMA]

Abstract

An audio signal processing method and apparatus and a differential beamforming method and apparatus to resolve a problem that an existing audio signal processing system cannot process audio signals in multiple application scenarios at the same time. The method includes determining a super-directional differential beamforming weighting coefficient, acquiring an audio input signal and determining a current application scenario and an audio output signal, acquiring, a weighting coefficient corresponding to the current application scenario, performing super-directional differential beamforming processing on the audio input signal using the acquired weighting coefficient in order to obtain a super-directional differential beamforming signal in the current application scenario, and performing processing on the formed signal to obtain a final audio signal required by the current application scenario. By using this method, a requirement that different application scenarios require different audio signal processing manners can be met.

Claims

1. An audio signal processing apparatus, comprising

a non-transitory memory storing instructions;

and

a processor coupled to the non-transitory memory and configured to execute the instructions to;

store a super-directional differential beamforming weighting coefficient;

acquire an audio input signal;

output the audio input signal;

determine a current application scenario and an output signal type required by the current application scenario;

transmit the current application scenario and the output signal type required by the current application scenario;

acquire, according to the output signal type required by the current application scenario, a weighting coefficient corresponding to the current application scenario;

perform super-directional differential beamforming processing on the audio input signal using the acquired weighting coefficient in order to obtain a super-directional differential beamforming signal;

transmit the super-directional differential beamforming signal; and

output the super-directional differential beamforming signal.

2. The apparatus according to claim 1, wherein the processor is further configured to execute the instructions to:

acquire an audio-left channel super-directional differential beamforming weighting coefficient and an audio-right channel super-directional differential beamforming weighting coefficient when the output signal type required by the current application scenario is a dual-channel signal type;

perform super-directional differential beamforming processing on the audio input signal according to the audio-left channel super-directional differential beamforming weighting coefficient in order to obtain an audio-left channel super-directional differential beamforming signal;

perform super-directional differential beamforming processing on the audio input signal according to the audio-right channel super-directional differential beamforming weighting coefficient in order to obtain an audio-right channel super-directional differential beamforming signal;

transmit the audio-left channel super-directional differential beamforming signal and the audio-right channel super-directional differential beamforming signal; and

output the audio-left channel super-directional differential beamforming signal and the audio-right channel super-directional differential beamforming signal.

3. The apparatus according to claim 1, wherein the processor is further configured to execute the instructions to:

acquire a mono super-directional differential beamforming weighting coefficient corresponding to the current application scenario when the output signal type required by the current application scenario is a mono signal type;

perform super-directional differential beamforming processing on the audio input signal according to the mono super-directional differential beamforming weighting coefficient in order to form one mono super-directional differential beamforming signal;

transmit the one mono super-directional differential beamforming signal; and

output the one mono super-directional differential beamforming signal.

4. The apparatus according to claim 1, wherein the processor is further configured to execute the instructions to:

adjust a microphone array to form a first subarray and a second subarray, wherein an end-fire direction of the first subarray is different from an end-fire direction of the second subarray, and wherein the first subarray and the second subarray each collect an original audio signal; and

transmit the original audio signal as the audio input signal.

5. The apparatus according to claim 1, wherein the processor is further configured to execute the instructions to:

adjust an end-fire direction of a microphone array, such that the end-fire direction points to a target sound source;

collect an original audio signal emitted from the target sound source; and

transmit the original audio signal as the audio input signal.

6. The apparatus according to claim 1, wherein the processor is further configured to execute the instructions to:

determine whether an audio collection area is adjusted;

determine a geometric shape of a microphone array, a position of a loudspeaker, and an adjusted audio collection effective area when the audio collection area is adjusted;

adjust a beam shape according to the audio collection effective area, or adjust the beam shape according to the audio collection effective area and the position of the loudspeaker in order to obtain an adjusted beam shape;

determine the super-directional differential beamforming weighting coefficient according to the geometric shape of the microphone array and the adjusted beam shape in order to obtain an adjusted weighting coefficient;

transmit the adjusted weighting coefficient; and

store the adjusted weighting coefficient.

7. An audio signal processing method, comprising:

determining a super-directional differential beamforming weighting coefficient;

acquiring an audio input signal;

determining a current application scenario and an output signal type required by the current application scenario;

acquiring, according to the output signal type required by the current application scenario, a weighting coefficient corresponding to the current application scenario;

performing super-directional differential beamforming processing on the audio input signal using the acquired weighting coefficient in order to obtain a super-directional differential beamforming signal; and

outputting the super-directional differential beamforming signal.

8. The audio signal processing method according to claim 7, wherein acquiring, according to the output signal type required by the current application scenario, the weighting coefficient corresponding to the current application scenario, wherein performing super-directional differential beamforming processing on the audio input signal using the acquired weighting coefficient in order to obtain the super-directional differential beamforming signal, and wherein outputting the super-directional differential beamforming signal further comprises:

acquiring an audio-left channel super-directional differential beamforming weighting coefficient and an audio-right channel super-directional differential beamforming weighting coefficient when the output signal type required by the current application scenario is a dual-channel signal type;

performing super-directional differential beamforming processing on the audio input signal according to the audio-left channel super-directional differential beamforming weighting coefficient in order to obtain an audio-left channel super-directional differential beamforming signal;

performing super-directional differential beamforming processing on the audio input signal according to the audio-right channel super-directional differential beamforming weighting coefficient in order to obtain an audio-right channel super-directional differential beamforming signal; and

outputting the audio-left channel super-directional differential beamforming signal and the audio-right channel super-directional differential beamforming signal.

9. The audio signal processing method according to claim 7, wherein acquiring, according to the output signal type required by the current application scenario, the weighting coefficient corresponding to the current application scenario, wherein performing super-directional differential beamforming processing on the audio input signal using the acquired weighting coefficient in order to obtain the super-directional differential beamforming signal, and wherein outputting the super-directional differential beamforming signal further comprises:

acquiring a mono super-directional differential beamforming weighting coefficient for forming a mono signal in the current application scenario when the output signal type required by the current application scenario is a mono signal type;

performing super-directional differential beamforming processing on the audio input signal according to the acquired mono super-directional differential beamforming weighting coefficient in order to form one mono super-directional differential beamforming signal; and

outputting the one mono super-directional differential beamforming signal.

10. The audio signal processing method according to claim 7, wherein before acquiring the audio input signal, the method further comprises:

adjusting a microphone array to form a first subarray and a second subarray, wherein an end-fire direction of the first subarray is different from an end-fire direction of the second subarray;

collecting an original audio signal using each of the first subarray and the second subarray; and

using the original audio signal as the audio input signal.

11. The audio signal processing method according to claim 7, wherein before acquiring the audio input signal, the method further comprises:

adjusting an end-fire direction of a microphone array, such that the end-fire direction points to a target sound source;

collecting an original audio signal of the target sound source; and

using the original audio signal as the audio input signal.

12. The audio signal processing method according to claim 7, wherein before acquiring, according to the output signal type required by the current application scenario, the weighting coefficient corresponding to the current application scenario, the method further comprises:

determining whether an audio collection area is adjusted;

determining a geometric shape of a microphone array, a position of a loudspeaker, and an adjusted audio collection effective area when the audio collection area is adjusted;

adjusting a beam shape according to the audio collection effective area, or adjusting the beam shape according to the audio collection effective area and the position of the loudspeaker in order to obtain an adjusted beam shape;

determining the super-directional differential beamforming weighting coefficient according to the geometric shape of the microphone array and the adjusted beam shape in order to obtain an adjusted weighting coefficient; and

performing super-directional differential beamforming processing on the audio input signal using the adjusted weighting coefficient.

13. The audio signal processing method according to claim 7, further comprising:

performing echo cancellation on an original audio signal collected by a microphone array;

or

performing echo cancellation on the super-directional differential beamforming signal.

14. The audio signal processing method according to claim 7, wherein after the super-directional differential beamforming signal is formed, the method further comprises performing echo suppression processing and/or noise suppression processing on the super-directional differential beamforming signal.

15. The audio signal processing method according to claim 7, further comprising:

forming, in another direction, except a direction of a sound source, in adjustable end-fire directions of a microphone array, at least one beamforming signal as a reference noise signal; and

performing noise suppression processing on the super-directional differential beamforming signal using the reference noise signal.

16. A differential beamforming apparatus, comprising:

a non-transitory memory storing instructions; and

a processor coupled to the non-transitory memory and configured to execute the instructions to;

determine a differential beamforming weighting coefficient according to a geometric shape of a microphone array and a set audio collection effective area, or determine the differential beamforming weighting coefficient according to the geometric shape of the microphone array, the set audio collection effective area, and a position of a loudspeaker;

transmit the formed weighting coefficient;

acquire, according to an output signal type required by a current application scenario, a weighting coefficient corresponding to the current application scenario; and

perform differential beamforming processing on an audio input signal using the acquired weighting coefficient.

17. The apparatus according to claim 16, wherein the processor is further configured to execute the instructions to:

determine D(ω

,θ

) and β

according to the geometric shape of the microphone array and the set audio collection effective area;

or

determine D(ω

,θ

) and β

according to the geometric shape of the microphone array, the set audio collection effective area, and the position of the loudspeaker;

determine a super-directional differential beamforming weighting coefficient according to the determined D(ω

,θ

) and β

using a formula h(ω

)=D^H(ω

,θ

)[D(ω

,θ

)D^H(ω

,θ

)]<sup>−

1</sup>β

, wherein the h(ω

) represents a weighting coefficient, the D(ω

,θ

) represents a steering matrix corresponding to the microphone array in any geometric shape, wherein the steering matrix is determined according to a relative delay generated when a sound source arrives at each microphone in the microphone array from different incident angles, wherein the D^H(ω

,θ

) represents a conjugate transpose matrix of D(ω

,θ

), wherein the ω

represents a frequency of an audio signal, wherein the θ

represents an incident angle of the sound source, and wherein the β

represents a response vector when the incident angle is θ

.

18. The apparatus according to claim 17, wherein the processor is further configured to execute the instructions to:

convert the set audio effective area into a pole direction and a null direction according to output signal types required by different application scenarios;

determine D(ω

,θ

) and β

in different application scenarios according to the obtained pole direction and the obtained null direction;

or

convert the set audio effective area into the pole direction and the null direction according to output signal types required by different application scenarios;

convert the position of the loudspeaker into the null direction; and

determine D(ω

,θ

) and β

in different application scenarios according to the obtained pole direction and the obtained null directions, wherein the pole direction is an incident angle that enables a response value of a super-directional differential beam in this direction to be 1, and wherein the null direction is an incident angle that enables the response value of the super-directional differential beam in this direction to be 0.

19. The apparatus according to claim 18, wherein the processor is further configured to execute the instructions to:

set an end-fire direction of the microphone array as the pole direction when the output signal type required by an application scenario is a mono signal type;

set M null directions when the output signal type required by the application scenario is the mono signal type, wherein M≦

N−

1, and wherein N represents a quantity of microphones in the microphone array;

set a 0-degree direction of the microphone array as the pole direction when the output signal type required by the application scenario is a dual-channel signal type;

set a 180-degree direction of the microphone array as the null direction in order to determine the super-directional differential beamforming weighting coefficient corresponding to one channel in dual channels when the output signal type required by the application scenario is the dual-channel signal type;

set the 180-degree direction of the microphone array as the pole direction in order to determine the super-directional differential beamforming weighting coefficient corresponding to the other channel; and

set the 0-degree direction of the microphone array as the null direction in order to determine the super-directional differential beamforming weighting coefficient corresponding to the other channel.