System and method for detecting the recognizability of input speech signals

• US 7,933,771 B2
• Filed: 03/11/2006
• Issued: 04/26/2011
• Est. Priority Date: 10/04/2005
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Lin, Jeng-Shien, Liao, Yuan-Fu, Chang, Sen-Chia
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): ORTIZ SANCHEZ, MICHAEL
• Application Number:
  US11/372,923
• Publication Number:
  US 20070078652A1
• Time in Patent Office: 1,872 Days
• Field of Search: 704/233, 704/231, 704/275
• US Class Current: 704/233
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08

Abstract

A system and method for detecting the recognizability of input speech signal is provided. It is designed in the pre-stage of speech recognition or a dialog system. The invention detects the user'"'"'s environmental condition and verifies if the input speech signal can be recognized. It mainly comprises an environment parameter generator, a signal recognition verifier, and a strategy response processor. Through the use of the invention in the pre-stage of speech recognition or a dialog system, it can precisely verify the recognizability of the input speech signal and receives the input speech signals of high recognition probability in a noisy environment. This reduces the impact caused by receiving the input speech signals of low recognition probability. This invention thus increases the recognition probability for a recognizer.

Claims

1. A system for detecting recognizability of an input signal, said system being a front stage of a speech recognition device or a dialog device, and comprising:

an environment parameter generator to generate at least one environment parameter from an input signal by using a voice activity detection (VAD) method and a missing feature imputation (MFI) method wherein the MFI method comprises a step of calculating a clean speech spectrum feature parameter, said at least one environment parameter including a confidence index of said system processing said input signal;

a signal recognition verifier to verify whether said input signal is recognizable in accordance with said at least one environment parameter, said signal recognition verifier being trained with environment parameters in advance; and

a strategy response processor;

wherein said confidence index is generated based on a probability distribution of a spectrum parameter of said input signal and the probability distribution of the spectrum parameter of a system model, and said input signal is passed to said speech recognition or dialog device when said input signal is verified as recognizable;

while said strategy response processor is triggered to respond with a plurality of strategies when said input signal is verified as unrecognizable.

2. The system as claimed in claim 1, wherein said at least one environment parameter represents environment conditions or signal quality of said input signal.

3. The system as claimed in claim 1, wherein said at least one environment parameter further includes signal-to-noise ratio (SNR) of said input signal, and a probability of said input signal being speech or any combination of the environment parameters, and said signal recognition verifier is trained with environment parameters in advance using multi-layer perception (MLP) method of pattern classification.

4. The system as claimed in claim 3, wherein said environment parameter generator includes an SNR calculator, a probability calculator, and a confidence measure, for calculating said SNR, said probability of said input signal being speech, and said confidence index of said system processing said input signal, respectively.

5. The system as claimed in claim 4, wherein one of said plurality of strategies is to inform a user of environment conditions and signal quality, and provide said user with corresponding solutions.

6. The system as claimed in claim 5, wherein said environment conditions and signal quality of said input signal include said SNR of said input signal, said probability of said input signal being speech, and said confidence index of said system processing said input signal.

7. The system as claimed in claim 5, wherein said corresponding solutions include ways to improve said environment conditions and signal quality.

8. The system as claimed in claim 7, wherein said ways include raising user'"'"'s voice, changing to a quieter location, raising speech clarity, and giving up recognition process.

9. The system as claimed in claim 8, wherein said user is prompted to raise the voice, change to a quieter location and re-input the signal when said SNR is lower than a threshold.

10. The system as claimed in claim 8, wherein said user is prompted to raise the speech clarity and re-input the signal when said confidence index is lower than a threshold.

11. The system as claimed in claim 8, wherein solutions for said giving up recognition process include transferring said user to a customer service representative.

12. A method for detecting the recognizability of an input signal, said method being implemented in a front stage of a speech recognition or dialog device, and comprising the steps of:

(a) generating at least one environment parameter for said input signal by using a voice activity detection (VAD) method and a missing feature imputation (MFI) method wherein the MFI method comprises a step of calculating a clean speech spectrum feature parameter, said at least one environment parameter including a confidence index of said system processing said input signal;

(b) using said at least one environment parameter to verify whether said input signal is recognizable according to verification training with environment parameters in advance; and

(c) passing said input signal to said speech recognition or dialog device when said input signal is verified as recognizable;

otherwise, triggering a strategy response processor to provide a plurality of strategies when said input signal is verified as unrecognizable;

wherein said confidence index is generated based on a probability distribution of a spectrum parameter of said input signal and the probability distribution of the spectrum parameter of a system model.

13. The method as claimed in claim 12, wherein said at least one environment parameter in said step (a) further includes signal-to-noise ratio (SNR) of said input signal, a probability of said input signal being speech or any combination of the environment parameters, and said verification training is trained with environment parameters in advance using multi-layer perception (MLP) method of pattern classification.

14. The method as claimed in claim 13, wherein generation of said SNR comprises the following steps of:

using said VAD method on a spectrum feature parameter of said input signal to detect a speech part and a non-speech part of said input signal;

using said MFI method to eliminate noise from said speech part to obtain a clean speech signal; and

calculating said SNR of said input signal in accordance with said non-speech part and said clean speech signal.

15. The method as claimed in claim 14, wherein generation of said probability of said input signal being speech comprises the steps of:

using said MFI method to calculate the probability of said SNR being greater than 0 when said clean signal spectrum is at a time t and a band d;

using said MFI method to calculate a probability R(t) of said clean signal spectrum being speech at said time t; and

calculating the average of said R(t) during the total length of said input signal to obtain said probability of said input signal being speech.

16. The method as claimed in claim 12, wherein generation of said confidence index comprises the steps of:

measuring the divergence between said input signal and a known system model distribution on frequency spectrum; and

using a sigmoid function to transform said divergence into a confidence index between 0 and 1.

17. The method as claimed in claim 12, wherein said verification training with environment parameters uses a multi-layer perception method of pattern classification.

18. The method as claimed in claim 13, wherein one of said plurality of strategies of said step (c) is to inform a user of environment conditions and signal quality, and provide said user with corresponding solutions.

19. The method as claimed in claim 18, wherein said environment conditions and signal quality of said input signal include said SNR of said input signal, said probability of said input signal being speech, and said confidence index of said system processing said input signal.

20. The method as claimed in claim 18, wherein said corresponding solutions include ways to improve said environment conditions and signal quality.

21. The method as claimed in claim 20, wherein said ways include raising user'"'"'s voice, changing to a quieter location, raising speech clarity, and giving up recognition process.

22. The method as claimed in claim 21, wherein said user is prompted to raise the voice or change to a quieter location and re-input the signal when said SNR is lower than a threshold.

23. The method as claimed in claim 21, wherein said user is prompted to raise the speech clarity and re-input the signal when said confidence index is lower than a threshold.

24. The method as claimed in claim 21, wherein solutions for said giving up recognition process include transferring said user to a customer service representative.

25. A method for detecting the recognizability of an input signal, said method being implemented in a front stage of a speech recognition or dialog device, and comprising the steps of:

(a) generating at least one environment parameter for said input signal by using a voice activity detection (VAD) method and a missing feature imputation (MFI) method wherein the MFI method comprises a step of calculating a clean speech spectrum feature parameter, said at least one environment parameter including a confidence index of said system processing said input signal;

(b) using said at least one environment parameter to verify whether said input signal is recognizable according to verification training with environment parameters in advance; and

(c) passing said input signal to said speech recognition or dialog device when said input signal is verified as recognizable;

otherwise, triggering a strategy response processor to provide a plurality of strategies when said input signal is verified as unrecognizable;

wherein said confidence index is generated based on a probability distribution of a spectrum parameter of said input signal and the probability distribution of the spectrum parameter of a system model by using the steps of;

measuring the divergence between said input signal and a known system model distribution on frequency spectrum; and

using a sigmoid function to transform said divergence into a confidence index between 0 and 1.