System and method for speech processing using independent component analysis under stability restraints

• US 20060053002A1
• Filed: 12/11/2003
• Published: 03/09/2006
• Est. Priority Date: 12/11/2002
• Status: Active Grant

• Current Assignee: Qualcomm, Inc., Regents of the University of California (University of California)
• Original Assignee: SoftMax Incorporated (Qualcomm, Inc.)
• Inventors: Visser, Erik, Lee, Te-Won
• Granted Patent: US 7,383,178 B2
• Time in Patent Office: Days
• US Class Current: 704/200
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

A system and method for separating a mixture of audio signal into desired audio signals (430) (e.g., speech) and a noise sign (440) is disclosed. Microphones (310, 320) are positioned to receive the mixed audio signals, and an independent component analysis (ICA) processes (212) the sound mixture using stability constraints. The ICA process (508) uses predefined characteristics of the desired speech signal to identify and isolate a target sound signal (430). Filter coefficients are adapted with a learning rule and filter weight update dynamics are stabilized to assist convergence to a stable separated ICA signal result. The separated signals may be peripherally-processed to further reduce noise effects using post-processing (214) and pre-processing (220, 230) techniques and information. The proposed system is designed and easily adaptable for implementation on DSP units or CPUs in audio communication hardware environments.

Claims

1. A method of separating a desired speech signal in an acoustic environment, comprising:

receiving a plurality of input signals, the input signals being generated responsive to the desired speech signal and other acoustic signals;

processing the received input signals using an independent component analysis (ICA) or blind source separation (BSS) under stability constraints; and

separating the received input signals into at one or more desired audio signals and one or more noise signals.

2. The method according to claim 1, wherein one of the desired audio signals is the desired speech signal.

3. The method according to claim 1 where the ICA or BSS process includes minimizing or maximizing the mathematical formulation of mutual information directly or indirectly through approximations.

4. The method according to claim 1 further comprising the step of stabilizing the ICA process by pacing ICA weight adaptation dynamics.

5. The method according to claim 1 further comprising the step of stabilizing the ICA process by scaling ICA inputs using an adaptive scaling factor to constrain weight adaptation speed.

6. The method according to claim 1 further comprising the step of stabilizing the ICA process by filtering learned filter weights in the time domain and the frequency domain to avoid reverberation effects.

7. The method according to claim 1, wherein peripheral processing techniques are applied to the input and separated signals in varying degrees.

8. The method according to claim 1, further comprising utilizing pre-processing techniques or information to enhance the performance of the separation.

9. The method according to claim 8, further comprising improving the conditioning of a mixing scenario applied to the input signals.

10. The method according to claim 2 further comprising utilizing characteristic information of the desired speech signal to identify the channel containing the separated desired speech signal.

11. The method according to claim 10 wherein the characteristic information is spatial, spectral or temporal information.

12. The method according to claim 1, wherein post-processing techniques are used to improve the quality of the desired signal utilizing the at least one of the noise signals or at least one of the input signals.

13. The method according to claim 12 further including the step of using the separated noise signal to further separate and enhance the desired speech signal.

14. The method according to claim 13 wherein the using step includes using the noise signal to estimate the noise spectrum for a noise filter.

15. The method according to claim 1 further including:

spacing apart at least two microphones; and

generating one of the input signals at each respective microphone.

16. The method according to claim 15 wherein the spacing step includes spacing the microphones between about 1 mm and about 1 m apart.

17. The method according to claim 15 wherein the spacing step includes spacing the microphones apart on a telephone receiver, a headset, or a hands-free kit.

18. The method according to claim 15, wherein the ICA process includes:

a first adaptive independent component analysis (ICA) filter connected to a first output channel and to a second input channel, the first filter being adapted by a recursive learning rule involving the application of a nonlinear bounded sign function to the noise signal channel;

a second adaptive independent component analysis filter connected to a first input channel and to a second output channel, the second filter being adapted by a recursive learning rule involving the application of a nonlinear bounded sign function to the desired speech signal channel;

wherein the first filter and the second filter are repeatedly applied to produce the desired speech signal.

19. The method according to claim 18, wherein (a) the desired speech channel recursively filtered by the first adaptive independent component analysis filter is fed back and added to the input channel from the second microphone, thereby producing the noise signal channel, and (b) the noise signal channel recursively filtered by the second adaptive independent component analysis filter is fed back and added to the input channel from the first microphone, producing the desired speech signal channel.

20. The method according to claim 19, wherein the input channel signals are scaled down by an adaptive scaling factor computed from a recursive equation as a function of the incoming signal energy.

21. The method according to claim 18, wherein the filter weight learning rule for the first adaptive ICA cross filter is stabilized by smoothing the filter coefficients in time, and wherein the filter weight learning rule for the second adaptive ICA cross filter is stabilized by smoothing the filter coefficients in time.

22. The method according to claim 18, wherein the first adaptive ICA cross filter weights are filtered in the frequency domain, and wherein the second adaptive ICA cross filter weights are filtered in the frequency domain.

23. The method according to claim 18, further comprising a post processing module connected to the desired speech signal which applies a single or multi channel speech enhancement module including voice activity detection and wherein the post-processed outputs are not fed back to the input channels.

24. The method according to claim 18 wherein the ICA process is implemented in a fixed point precision environment where the adaptive ICA cross filters are applied at every sampling instant but where filter coefficients are updated at multiples of the sampling instant and filter lengths of variable size are used to fit the computational power available.

25. The method according to claim 18, further comprising post processing the desired speech signal using the noise signal, the post processing module applying spectral subtraction to the desired speech signal based on the noise signal.

26. The method according to claim 18, further comprising post processing the desired speech signal using the noise signal, the post processing module applying Wiener filtering to the desired speech signal based on the noise signal.

27. The method according to claim 18, further comprising receiving a third set of audio input signals from a third channel, and applying a nonlinear bounded function to incoming signals using a third filter.

28. A speech device, comprising:

at least two spaced-apart microphones constructed to receive acoustic sound signals, the microphones being an expected distance from a speech source, and;

an ICA or BSS processor coupled to the microphones,

the processor operating steps comprising;

receiving sound signals from the two microphones,

separating the sound signals under stability constraints into at least one desired speech signal line and at least one noise signal line.

29. The speech device according to claim 28, further comprising a post process filter coupled to the noise line and to the desired speech signal line.

30. The speech device according to claim 28, wherein the microphones are spaced apart about 1 mm to about 1 m.

31. The method according to claim 30 further including pre-processing the acoustic sound signals received at each microphone.

32. The speech device according to claim 28, wherein one of the microphones is on a face of the device housing and the other microphone is on another face of the device housing.

33. The speech device according to claim 28, wherein the speech device is constructed to be a wireless phone.

34. The speech device according to claim 28, wherein the speech device is constructed to be a wireless phone.

35. The speech device according to claim 28, wherein the speech device is constructed to be a hands-free car kit.

36. The speech device according to claim 28, wherein the speech device is constructed to be a headset.

37. The speech device according to claim 28, wherein the speech device is constructed to be a personal data assistant.

38. The speech device according to claim 28, wherein the speech device is constructed to be a handheld bar-code scanning device.

39. A system for separating desired speech signals in an acoustic environment, comprising

a plurality of input channels each receiving one or more acoustic signals;

at least one ICA or BSS filter, wherein the filter separates the received signals under stability constraints into one or more desired audio signals and one or more noise signals; and

a plurality of output channels transmitting the separated signals.

40. The system according to claim 39, wherein the desired audio signal is a speech signal received in the plurality of acoustic signals.

41. The system according to claim 39, wherein the filter modulates the mathematical formulation of mutual information directly or indirectly through approximations.

42. The system according to claim 39, wherein the filter stabilizes the ICA process by pacing ICA weight adaptation dynamics.

43. The system according to claim 39, wherein the filter stabilizes the ICA process by scaling ICA inputs using an adaptive scaling factor to constrain weight adaptation speed.

44. The system according to claim 39, wherein the filter stabilizes the ICA process by filtering learned filter weights in the time domain and the frequency domain to avoid reverberation effects.

45. The system according to claim 39, further comprising one or more peripheral processing filters applied to the input and/or output signals.

46. The system according to claim 45, further comprising one or more pre-processing filters.

47. The system according to claim 45, further comprising one or more post-processing filters.

48. The system according to claim 39, further comprising one or more microphones connected to the input channels.

49. The system according to claim 48, comprising two or more microphones each spaced apart between about 1 mm and about 1 m apart.

50. The system according to claim 39, wherein the system is constructed on a hand-held device.

51. The system according to claim 39, wherein the filter includes:

a first adaptive independent component analysis (ICA) filter connected a first output channel and to a second input channel, the first filter being adapted by a recursive learning rule involving the application of a nonlinear bounded sign function to the noise signal channel;

a second adaptive independent component analysis filter connected to a first output channel and to a second input channel, the second filter being adapted by a recursive learning rule involving the application of a nonlinear bounded sign function to the desired speech signal channel;

wherein the first filter and the second filter are repeatedly applied to produce the desired speech signal.

52. A system for isolating a speech signal, comprising:

a set of signal generators, each signal generator arranged to generate a mixed signal indicative of a mixture of the speech signal and other acoustic signals;

a processor configured to receive each of the mixed signals;

the processor operating a method further comprising;

processing the set of mixed signals using an independent component analysis (ICA) or blind source separation (BSS) under stability constraints; and

separating the mixed signals into the speech signal and at least one noise signal; and

a speech enabled unit receiving the speech signal.

53. The system-according to claim 52, wherein the signal generators are constructed as acoustic transducers.

54. The system according to claim 53 wherein the acoustic transducers are microphones constructed to receive acoustic signals in the human-speech frequency range.