Speech communication system for combined voice recognition, hands-free telephony and in-car communication

• US 9,620,146 B2
• Filed: 05/16/2012
• Issued: 04/11/2017
• Est. Priority Date: 05/16/2012
• Status: Active Grant

• Current Assignee: Cerence Inc., Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Buck, Markus, Haulick, Tim, Matheja, Timo
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US14/398,769
• Publication Number:
  US 20150120305A1
• Time in Patent Office: 1,791 Days
• Field of Search: 704275
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04M 1/6075 adapted for handsfree use in a vehicle H04M1/6058 takes precedence; arrangements for holding telephones in a vehicle B60R11/0241
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04R 2227/009 Signal processing in [PA] systems to enhance the speech intelligibility
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407

Abstract

A multi-mode speech communication system is described that has different operating modes for different speech applications. A speech service compartment contains multiple system users, multiple input microphones that develop microphone input signals from the system users to the system, and multiple output loudspeakers that develop loudspeaker output signals from the system to the system users. A signal processing module is in communication with the speech applications and includes an input processing module and an output processing module. The input processing module processes the microphone input signals to produce a set user input signals for each speech application that are limited to currently active system users for that speech application. The output processing module processes application output communications from the speech applications to produce loudspeaker output signals to the system users, wherein for each different speech application, the loudspeaker output signals are directed only to system users currently active in that speech application. The signal processing module dynamically controls the processing of the microphone input signals and the loudspeaker output signals to respond to changes in currently active system users for each application.

Claims

1. A multi-mode speech communication system having a plurality of different operating modes, each operating mode associated with one of a plurality of different speech applications, the system comprising:

a signal processing module in communication with the speech applications and including;

an input processing module that processes microphone input signals received from a plurality of input microphones within a speech service compartment to produce a set of user input signals for each speech application that are limited to currently active system users for that speech application, and

an output processing module that processes application output communications from the speech applications to produce loudspeaker output signals to system users, wherein for each different speech application, the loudspeaker output signals are directed only to output loudspeakers within the speech service compartment which are associated with system users currently active in that speech application;

wherein the signal processing module dynamically controls the processing of the microphone input signals and the loudspeaker output signals to respond to changes in currently active system users for each application, and wherein the signal processing module is capable of operating in a plurality of the different operating modes at a same time, and

wherein the user input signals for each speech application are enhanced to maximize speech from system users of that speech application and to minimize audio sources other than those associated with that speech application, wherein the audio sources other than those associated with that speech application are substantially canceled such that the speech from system users of that speech application is substantially exclusively further processed by the input processing module.

2. A system according to claim 1, wherein the system operates a plurality of different speech applications in parallel.

3. A system according to claim 1, wherein the speech service compartment is the passenger compartment of an automobile.

4. A system according to claim 1, wherein the signal processing module further comprises a control module for dynamically controlling the processing of the microphone input signals and the loudspeaker output signals response to control mechanism inputs from the system users.

5. A computer-implemented method using one or more computer processes for multi-mode speech communication using a plurality of different operating modes, each operating mode associated with one of a plurality of different speech applications, the method comprising:

developing a plurality of microphone input signals received by a plurality of input microphones within a speech service compartment;

processing the microphone input signals with an input processing module to produce a set of user input signals for each speech application that are limited to currently active system users for that speech application; and

processing application output communications from the speech applications with an output processing module to produce a plurality of loudspeaker output signals to a plurality of output loudspeakers within the service compartment, wherein for each different speech application, the loudspeaker output signals are directed only to output loudspeakers associated with system users currently active in that speech application;

wherein the processing of the microphone input signals and the loudspeaker output signals is dynamically controlled to respond to changes in system users currently active in each speech application, and wherein the processing modules are capable of operating in a plurality of the different operating modes at a same time, and

wherein the processing of the microphone input signals comprises enhancing the user input signals for each speech application to maximize speech from system users of that speech application and to minimize audio sources other than those associated with that speech application, wherein the audio sources other than those associated with that speech application are substantially canceled such that the speech from system users of that speech application is substantially exclusively further processed by the input processing module.

6. A method according to claim 5, wherein the speech applications include a hands free telephone application and the operating modes include a mode optimized for the hands free telephone application.

7. A method according to claim 5, wherein the speech applications include an in-car communication system and the operating modes include a mode optimized for the in-car communication system.

8. A method according to claim 5, wherein the speech applications include an automatic speech recognition (ASR) application and the operating modes include a mode optimized for the ASR application.

9. A method according to claim 5, wherein a plurality of different speech applications operate in parallel.

10. A method according to claim 5, wherein the speech service compartment is the passenger compartment of an automobile.

11. A method according to claim 5, wherein dynamically controlling the processing of the microphone input signals and the loudspeaker output signals is performed in response to control mechanism inputs from the system users.

12. A method according to claim 5, wherein the output loudspeakers are located in a plurality of different locations within the service compartment and each of the system users is associated with at least one different loudspeaker.

13. A method according to claim 5, wherein the processing of the loudspeaker output signals includes determining which output speakers the loudspeaker output signals should be directed to.

14. A method according to claim 5, wherein substantially all input microphones and substantially all output loudspeakers are available to system users when the systems users are each active in a same speech application.

15. A method according to claim 5, wherein a number of microphone charnels processed by the input processing module does not have to match a number of currently active speech applications or system users.

16. A method according to claim 5, wherein a number of loudspeaker output channels processed and developed by the output processing module does not have to match a number of received signals from different speech applications or for a total number of system users.

17. A method according to claim 5, wherein enhancement of the user input signals includes performing at least one of noise reduction and echo cancellation on the user input signals.

18. A method according to claim 5, wherein the output processing module enhances signals associated with the application output communications based on the speech application and adaptively mixes the enhanced signals for each currently active speech application to produce the plurality of loudspeaker output signals.

19. A method according to claim 5, wherein the enhanced user input signals are further adaptively mixed to reflect different background noises, different speech signals levels, and exploit diversity effects associated with the user input signals for each speech application.

20. A computer program product encoded in a non-transitory computer-readable medium for multi-mode speech communication using a plurality of different operating modes, each operated mode associated with one of a plurality of different speech applications, the product comprising:

program code for developing a plurality of microphone input signals received by a plurality of input microphones within a speech service compartment;

program code for processing the microphone input signals with an input processing module to produce a set of user input signals for each speech application that are limited to currently active system users for that speech application; and

program code for processing application output communications from the speech applications with an output processing module to produce a plurality of loudspeaker output signals to a plurality of output loudspeakers within the service compartment, wherein for each different speech application, the loudspeaker output signals are directed only to output loudspeakers associated with system users currently active in that speech application;

wherein the processing of the microphone input signals and the loudspeaker output signals is dynamically controlled to respond to changes in system users currently active in each speech application, and wherein the processing modules are capable of operating in a plurality of the different operating modes at a same time, and

wherein the processing of the microphone input signals comprises enhancing the user input signals for each speech application to maximize speech from system users of that speech application and to minimize audio sources other than those associated with that speech are substantially canceled such that the speech from system users of that speech application is substantially exclusively further processed by the input processing module.

21. A product according to claim 20, wherein a plurality of different speech applications operate in parallel.