System and method for speech processing using independent component analysis under stability constraints

• US 7,383,178 B2
• Filed: 12/11/2003
• Issued: 06/03/2008
• Est. Priority Date: 12/11/2002
• Status: Active Grant

• Current Assignee: Qualcomm, Inc., Regents of the University of California (University of California)
• Original Assignee: SoftMax Incorporated (Qualcomm, Inc.)
• Inventors: Visser, Erik, Lee, Te-Won
• Primary Examiner(s): Storm; Donald L.
• Application Number:
  US10/537,985
• Publication Number:
  US 20060053002A1
• Time in Patent Office: 1,636 Days
• Field of Search: None
• US Class Current: 704/227
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

A system and method for separating a mixture of audio signal into desired audio signals (430) (e.g., speech) and a noise sign (440) is disclosed. Microphones (310, 320) are positioned to receive the mixed audio signals, and an independent component analysis (ICA) processes (212) the sound mixture using stability constraints. The ICA process (508) uses predefined characteristics of the desired speech signal to identify and isolate a target sound signal (430). Filter coefficients are adapted with a learning rule and filter weight update dynamics are stabilized to assist convergence to a stable separated ICA signal result. The separated signals may be peripherally-processed to further reduce noise effects using post-processing (214) and pre-processing (220, 230) techniques and information. The proposed system is designed and easily adaptable for implementation on DSP units or CPUs in audio communication hardware environments.

Claims

1. A method of separating a desired speech signal in an acoustic environment, comprising:

receiving a plurality of input signals, the input signals being generated responsive to the desired speech signal and other acoustic signals;

processing the received input signals using an independent component analysis (ICA) or blind source separation (BSS) method under stability constraints, wherein the ICA or BSS method modulates the mathematical formulation of mutual information directly or indirectly through approximations; and

separating the received input signals into output channels comprising one or more desired audio output signals and one or more noise output signals.

2. The method according to claim 1, wherein one of the desired audio signals is the desired speech signal.

3. The method according to claim 2, further comprising utilizing characteristic information of the desired speech signal to identify the output channel comprising the separated desired speech signal.

4. The method according to claim 3 wherein the characteristic information is spatial, spectral or temporal information.

5. The method according to claim 1, wherein the ICA method further comprises minimizing or maximizing the mathematical formulation of mutual information directly or indirectly through approximations.

6. The method according to claim 1, wherein the stability constraints comprise pacing the adapting of an ICA filter.

7. The method according to claim 1, wherein the stability constraints comprise scaling the received input signals using an adaptive scaling factor, the adaptive scaling factor being selected to constrain weight adaptation speed.

8. The method according to claim 1, wherein the stability constraints comprise filtering learned filter weights in the time domain and the frequency domain, the filtering selected to avoid introduction of artificial reverberation effects.

9. The method according to claim 1, further comprising applying peripheral pre-processing or post-processing techniques to at least one of the received input signals or at least one of the separated output signals.

10. The method according to claim 1, further comprising pre-processing the received input signals.

11. The method according to claim 10, further comprising improving the conditioning of a mixing scenario applied to the input signals.

12. The method according to claim 1, further comprising applying post-processing techniques to at least one of the output signals using at least one processing signal selected from one or more of the noise signals and one or more of the input signals.

13. The method according to claim 12, wherein the using at least one processing signal consists of using the noise signal.

14. The method according to claim 13 wherein the using the noise signal comprises using the noise signal to estimate the noise spectrum for a noise filter.

15. The method according to claim 1, further comprising:

spacing apart at least a first and a second microphone; and

generating one of the input signals at each respective microphone.

16. The method according to claim 15, wherein the spacing apart at least a first and a second microphone comprises spacing the microphones between about 1 millimeter and about 1 meter apart.

17. The method according to claim 15, wherein the spacing apart at least a first and a second microphone comprises spacing the microphones apart on a telephone receiver, a headset, or a hands-free kit.

18. The method according to claim 1, wherein the ICA or BSS method comprises:

adapting a first adaptive ICA filter connected to a first output signal and to a second input signal by a recursive learning rule involving the application of a nonlinear bounded sign function to one or more noise output signals; and

adapting a second adaptive ICA filter connected to a first input signal and to a second output signal by a recursive learning rule involving the application of a nonlinear bounded sign function to the one or more desired audio output signals,

wherein the first filter and the second filter are repeatedly applied to produce the desired speech signal.

19. The method according to claim 18, further comprising:

spacing apart at least a first and a second microphone;

generating one of the input signals at each respective microphone;

recursively filtering the one or more desired audio output signals by the first adaptive independent component analysis filter to obtain a recursively filtered speech signal;

recursively filtering the one or more noise output signals by the second adaptive independent component analysis filter to obtain a recursively filtered noise signal;

adding the recursively filtered speech signal to the input signal from the second microphone, thereby producing the one or more noise output signals; and

adding the recursively filtered noise signal to the input signal from the first microphone, thereby producing the one or more desired audio output signals.

20. The method according to claim 19, wherein the received input signals are inversely scaled by an adaptive scaling factor computed from a recursive equation as a function of incoming signal energy.

21. The method according to claim 18, further comprising:

stabilizing a recursive learning rule adapting the first adaptive ICA filter by smoothing coefficients of the first adaptive ICA filter in time; and

stabilizing a recursive learning rule adapting the second adaptive ICA filter by smoothing coefficients of the second adaptive ICA filter in time.

22. The method according to claim 18, wherein filter weights of the first adaptive ICA filter are filtered in the frequency domain, and wherein filter weights of the second adaptive ICA filter are filtered in the frequency domain.

23. The method according to claim 18, wherein the ICA method is implemented in a fixed point computing precision environment and wherein the ICA method further comprises:

applying the adaptive ICA filters at every sampling instant;

updating filter coefficients at multiples of the sampling instant; and

adapting filter lengths of variable sizes according to the computational power available.

24. The method according to claim 1, further comprising post processing the desired speech signal comprising voice activity detection and wherein post-processed outputs are not fed back to input signals.

25. The method according to claim 1, further comprising applying spectral subtracting to the one or more desired audio output signals based on the one or more noise signals.

26. The method according to claim 1, further comprising applying Wiener filtering to the one or more desired audio output signals based on the one or more noise signals.

27. The method according to claim 1, further generating a third set of audio input signals at a third microphone, and applying a nonlinear bounded function to incoming signals using a third filter.

28. A system for separating a desired speech signal in an acoustic environment, comprising

a plurality of input channels each receiving one or more acoustic signals, wherein the one or more acoustic signals comprises a speech signal;

at least one independent component analysis (ICA) or blind-source separation (BSS) filter module comprising an ICA or BSS filter that separates the received signals into one or more desired audio signals and one or more noise signals;

a stability constraint, wherein the stability constraint at least partially stabilizes the ICA or BSS filter; and

a plurality of output channels transmitting the separated signals,

wherein the filter modulates the mathematical formulation of mutual information directly or indirectly through approximations.

29. The system according to claim 28, wherein the one or more acoustic signals comprise the one or more desired audio signals.

30. The system according to claim 28, wherein implementing the stability constraint paces adaptation of the ICA or BSS filter.

31. The system according to claim 28, wherein implementing the stability constraint comprises scaling ICA or BSS inputs using an adaptive scaling factor, the adaptive scaling factor selected to constrain adaptation speed.

32. The system according to claim 28, wherein implementing the stability constraint comprises filtering learned filter weights in the time domain and the frequency domain, the filter selected to avoid introduction of artificial reverberation effects.

33. The system according to claim 28, further comprising one or more processing modules comprising at least one filter selected from a pre-processing peripheral filter and a post-processing peripheral filter applied to the one or more acoustic signals and/or the separated signals.

34. The system according to claim 33, wherein the filter is the pre-processing peripheral filter.

35. The system according to claim 33, wherein the filter is the post-processing peripheral filter.

36. The system according to claim 28, further comprising one or more microphones connected to the plurality of input channels.

37. The system according to claim 36, wherein the one or more microphones are two or more microphones, and wherein each of the two or more microphones is spaced between about 1 millimeter and about 1 meter apart.

38. The system according to claim 28, wherein the system is constructed on a hand-held device.

39. The system according to claim 28, wherein the at least one ICA or BSS filter module comprises:

a first adaptive independent component analysis (ICA) filter connected to a first output channel and to a second input channel, the first filter being adapted by a recursive learning rule involving the application of a nonlinear bounded sign function to the one or more noise signals;

a second adaptive independent component analysis filter connected to a first output channel and to a second input channel, the second filter being adapted by a recursive learning rule involving the application of a nonlinear bounded sign function to the desired speech signal;

wherein the first filter and the second filter are repeatedly applied to produce the desired speech signal.

40. The system according to claim 28, wherein the plurality of input channels comprises at least two spaced-apart microphones constructed to receive the acoustic signals, the microphones being an expected distance from a speech source;

wherein the at least one ICA or BSS filter module is coupled to the microphones; and

wherein the at least one ICA or BSS filter module is configured to;

receive sound signals from the two microphones; and

separate the sound signals under the stability constraint into at least one desired speech output signal line and at least one noise output signal line.

41. The system according to claim 40, further comprising a post-processing filter coupled to the noise output signal line and to the desired speech output signal line.

42. The system according to claim 40, wherein the microphones are spaced about 1 millimeter to about 1 meter apart.

43. The system according to claim 42 further comprising a pre-processing module configured to pre-process the acoustic signals received at each microphone.

44. The system according to claim 40, wherein one of the microphones is on a face of a device housing and another of the microphones is on another face of the device housing.

45. The system according to claim 40, wherein the system is integrated into a speech device.

46. The system according to claim 45, wherein the speech device comprises a wireless phone.

47. The system according to claim 45, wherein the speech device comprises a hands-free car kit.

48. The system according to claim 45, wherein the speech device comprises a headset.

49. The system according to claim 45, wherein the speech device comprises a personal data assistant.

50. The system according to claim 45, wherein the speech device comprises a handheld bar-code scanning device.