Speech recognizer performance in car and home applications utilizing novel multiple microphone configurations

• US 6,889,189 B2
• Filed: 09/26/2003
• Issued: 05/03/2005
• Est. Priority Date: 09/26/2003
• Status: Active Grant

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Chengalvarayan, Rathinavelu, Rigazio, Luca, Boman, Robert, Hanson, Brian
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US10/672,167
• Publication Number:
  US 20050071159A1
• Time in Patent Office: 585 Days
• Field of Search: 704/270, 704/275, 704/278
• US Class Current: 704/270
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

System speakers are switched to function as sound input transducers to improve recognizer performance and to support recognizer features. A crossbar switch is selectively activated, either manually or under software control, to allow system loudspeakers to function as sound input transducers that supplement the recognition system microphone or microphone array. Using loudspeakers as “microphones” improves speech recognition in noisy environments, thus attaining better recognition performance with little added system cost. The loudspeakers, positioned in physically separate locations also provide spatial information that can be used to determine the location of the person speaking and thereby offer different functionality for different persons. Acoustic models are selected based on environmental and vehicle operating conditions and may be adapted dynamically using ambient information obtained using the loudspeakers as sound input transducers.

Claims

1. A signal processing system to improve speech recognition, comprising:

a loudspeaker system for playback of audio content;

a sound transducer for input of speech utterances from a recognition system user;

a multi-channel signal processing system configured to receive speech utterance input from said sound transducer and from at least one additional audio source, the signal processing system being operative to enhance the recognition quality of the received speech utterance;

a switching system coupled to said signal processing system and to said loudspeaker system, the switching system being operative to selectively couple said loudspeaker system to said signal processing system thereby utilizing said loudspeaker system as said additional audio source.

2. The system of claim 1 wherein said signal processing system is configured to increase discrimination between speech utterances produced by one source from sounds produced by other sources.

3. The system of claim 1 wherein said signal processing system is configured to perform source separation.

4. The system of claim 1 wherein said signal processing system is configured to perform source separation using independent component analysis to separate speech from noise.

5. The system of claim 4 wherein said independent component analysis is performed in the frequency domain.

6. The system of claim 1 further comprising a recognizer coupled to said signal processor.

7. The system of claim 6 further comprising a control system coupled to said recognizer and operative to control a function of an electronic component.

8. The system of claim 7 wherein said electronic component is an audio system or audio/video system coupled to said loudspeaker system.

9. The system of claim 7 wherein said control system includes a memory for storing first data regarding electronic component function in association with second data indicative of at least one recognition system user.

10. The system of claim 9 wherein said control system uses said first and second data to selectively permit control of a function of an electronic component.

11. The system of claim 1 wherein said signal processing system controls said switching system.

12. The system of claim 6 wherein said recognizer controls said switching system.

13. A method of improving speech recognition, comprising:

selectively interrupting playback of audio content through a loudspeaker system while concurrently utilizing said loudspeaker system as a sound input transducer;

receiving an input speech utterance from a recognition system user through a plurality of transducers that include said loudspeaker system; and

processing said received input speech utterance to enhance recognition quality.

14. The method of claim 13 wherein said step of selectively interrupting playback is performed automatically.

15. The method of claim 13 wherein one of said plurality of transducers is a microphone and wherein said step of selectively interrupting playback is performed in response to receipt of a speech utterance via said microphone.

16. The method of claim 13 wherein one of said plurality of transducers is a microphone and wherein said step of selectively interrupting playback is performed in response to speech recognition performed upon a speech utterance received via said microphone.

17. The method of claim 13 further comprising controlling at least one function of an electronic component based on the results of said step of performing recognition.

18. The method of claim 13 further comprising processing said received input speech utterance by performing source separation using separate signals input respectively through said loudspeaker system and through a microphone.

19. The method of claim 18 wherein said source separation is performed using independent component analysis to separate speech and noise.

20. The method of claim 13 further comprising using separate signals input respectively through said loudspeaker system and through a microphone to ascertain spatial information from the input speech utterance.

21. The method of claim 20 further comprising using said spatial information to discriminate between a plurality of concurrent recognition system users.

22. The method of claim 20 further comprising controlling at least one function of an electronic component based on the results of said step of performing recognition and further based on said spatial information.

23. A speech recognition system for incorporation into an automotive vehicle having an audio system with an associated loudspeaker system, comprising:

a sound transducer for input of speech utterances from a recognition system user;

a multi-channel signal processing system configured to receive speech utterance input from said microphone and from at least one additional audio source, the signal processing system being operative to enhance the recognition quality of the received speech utterance;

a switching system coupled to said signal processing system and to said loudspeaker system, the switching system being operative to selectively couple said loudspeaker system to said signal processing system thereby utilizing said loudspeaker system as said additional audio source.

24. A speech recognition system for integrating into a home entertainment system having a loudspeaker system for playback of audio content, comprising:

a sound transducer for input of speech utterances from a recognition system user;

a multi-channel signal processing system configured to receive speech utterance input from said microphone and from at least one additional audio source, the signal processing system being operative to enhance the recognition quality of the received speech utterance;

a switching system coupled to said signal processing system and to said loudspeaker system, the switching system being operative to selectively couple said loudspeaker system to said signal processing system thereby utilizing said loudspeaker system as said additional audio source.

25. A signal processing system to improve speech recognition, comprising:

a loudspeaker system for playback of audio content;

a microphone for input of speech utterances from a recognition system user;

a multi-channel signal processing system configured to receive speech utterance input from said sound transducer and from at least one additional audio source, the signal processing system being operative to enhance the recognition quality of the received speech utterance;

a switching system coupled to said signal processing system and to said loudspeaker system, the switching system being operative to selectively couple said loudspeaker system to said signal processing system thereby utilizing said loudspeaker system as said additional audio source.

26. A speech recognition system, comprising:

a loudspeaker system for playback of audio content;

a sound transducer for input of speech utterances from a recognition system user;

a recognizer configured to receive speech utterance input from said sound transducer and configured to use at least one acoustic model in the recognition of said speech utterance input;

an acoustic model selection system coupled to said recognition system and being operative to selectively control the acoustic model used by said recognition system based on at least one environment parameter;

said acoustic model selection system further using said loudspeaker system to assist in determining the acoustic model used by said recognition system.

27. The recognition system of claim 26 wherein said sound transducer is a microphone.

28. The recognition system of claim 26 wherein said sound transducer is a loudspeaker coupled to receive audio input.

29. The recognition system of claim 26 wherein said acoustic model selection system includes a pool of acoustic models and operates to select a model from said pool for use by the recognizer, based on at least one environment parameter.

30. The recognition system of claim 26 wherein said loudspeaker system is coupled to receive audio input from the environment and the recognition system further includes acoustic model training module to provide at least one acoustic model for use by the recognizer.

31. The recognition system of claim 26 wherein said loudspeaker system is coupled to receive audio input from the environment and the recognition system further includes acoustic model training module to provide a plurality of acoustic models for use by the recognizer for different background noise conditions.

32. The recognition system of claim 26 further comprising a decision logic unit coupled to said model selection system that ascertains at least one vehicle operating parameter and causes the model selection system to change at least one aspect of the acoustic model used by the recognizer based on said operating parameter.

33. The recognition system of claim 26 further comprising a decision logic unit coupled to said model selection system that ascertains at least one vehicle environmental characteristic and causes the model selection system to change at least one aspect of the acoustic model used by the recognizer based on said environmental characteristic.

34. The recognition system of claim 26 further comprising a noise compensation module for performing signal processing prior to recognition, the noise compensation module being configured to selectively perform said signal processing based on said at least one environmental parameter.

35. The recognition system of claim 34 further comprising feature extraction module that processes said speech utterance input and wherein said noise compensation module operates upon a speech information signal derived from said feature extraction module.

36. A speech recognition system, comprising:

a recognizer configured to receive speech utterance input from a sound transducer and configured to use at least one acoustic model in the recognition of said speech utterance input;

an acoustic model selection system coupled to said recognition system and being operative to selectively control the acoustic model used by said recognition system based on at least one sensed operating parameter;

a decision logic unit coupled to said module selection system that ascertains at least one vehicle operating parameter and causes the model selection system to change at least one aspect of the acoustic model used by the recognizer based on said operating parameter.

37. The speech recognition system of claim 36, wherein:

said decision logic unit further ascertains vehicle location and causes the model selection system to change at least one aspect of the acoustic model used by the recognizer based on said location.

38. The speech recognition system of claim 36 wherein said acoustic model selection system is configured to selectively control the acoustic model used by the recognition system during an ongoing recognition session.