MICROPHONE ARRAY BASED SPEECH RECOGNITION SYSTEM AND TARGET SPEECH EXTRACTING METHOD OF THE SYSTEM

• US 20090150146A1
• Filed: 09/30/2008
• Published: 06/11/2009
• Est. Priority Date: 12/11/2007
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: CHO, Hoon Young, Chung, Hoon, Park, Jeon Gue, Jung, Ho Young, Kang, Byung Ok, Kim, Kap Kee, Jeon, Hyung Bae, Lee, Yun Keun, Lee, Sung Joo, Kang, Jeom Ja
• Granted Patent: US 8,249,867 B2
• Time in Patent Office: Days
• US Class Current: 704/233
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

A microphone-array-based speech recognition system using a blind source separation (BBS) and a target speech extraction method in the system are provided. The speech recognition system performs an independent component analysis (ICA) to separate mixed signals input through a plurality of microphone into sound-source signals, extracts one target speech spoken for speech recognition from the separated sound-source signals by using a Gaussian mixture model (GMM) or a hidden Markov Model (HMM), and automatically recognizes a desired speech from the extracted target speech. Accordingly, it is possible to obtain a high speech recognition rate even in a noise environment.

Claims

1. A microphone-array-based speech recognition system comprising:

a signal separator which separates mixed signals input through a plurality of microphone into sound-source signals by an ICA algorithm;

a target speech extractor which extracts one target speech spoken for speech recognition from the sound-source signals separated by the signal separator; and

a speech recognition unit which recognizes a desired speech from the extracted target speech.

2. The microphone-array-based speech recognition system of claim 1, further comprising an additional information provider which transmits additional information used for extraction of the target speech to the target speech extractor.

3. The microphone-array-based speech recognition system of claim 1, wherein the target speech extractor extracts feature vector sequences from the separated sound-source signals, calculates logarithm likelihood ratios (LLRs) of the extracted feature vector sequences, calculates a maximum value by using the calculated LLRs, compares the maximum value with a predetermined threshold value, and determines the maximum value to be the target speech if the maximum value is larger than the threshold value.

4. The microphone-array-based speech recognition system of claim 3, wherein the target speech extractor determines that the target speech does not exist in the separated sound-source signals if the maximum value is smaller than the threshold value.

5. The microphone-array-based speech recognition system of claim 2, wherein, in a case where the additional information is provided, the target speech extractor performs a hypothesis test for the separated sound-source signals by using a Gaussian mixture model (GMM) to determine a sound-source signal having the highest reliability to be the target speech.

6. The microphone-array-based speech recognition system of claim 5, wherein the additional information is gender information, speech-music information, speech-noise information, or speaker'"'"'s identification information.

7. The microphone-array-based speech recognition system of claim 1, wherein, in a case where the additional information for the target speech is not provided, the target speech extractor calculates an LLR-based reliability by using a hidden Markov model (HMM) as a speech-recognition acoustic model.

8. A target speech extraction method in a microphone-array-based speech recognition system, comprising:

separating mixed signals input through a plurality of microphone into sound-source signals by an ICA;

extracting one target speech spoken for speech recognition from the separated sound-source signals; and

recognizing a desired speech from the extracted target speech.

9. The target speech extraction method of claim 8, wherein the extracting of the target speech comprises:

extracting feature vector sequence Xⁱ from the separated sound-source signals;

calculating LLRs (logarithm likelihood ratios) LLR_i of the extracted feature vector sequence;

calculating a maximum value k by using the calculated LLRs;

comparing the maximum value with a predetermined threshold value; and

determining the maximum value k to be the target speech if the maximum value k is larger than the threshold value.

10. The target speech extraction method of claim 9, further comprises determining that the target speech does not exist in the separated sound-source signals if the maximum value is smaller than the threshold value.

11. The target speech extraction method of claim 9, further comprising, in a case where additional information representing that the target speech is a female speech is provided, calculating the LLRs LLR_i expressed by Equation 3 by applying pre-generated male and female GMMs (Gaussian mixture models) λ

_Male and λ

_Female to the extracted feature vector sequence Xⁱ.

$\begin{array}{cc}{\mathrm{LLR}}_i=\log <br><br>\frac{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Female}}\right)}{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Male}}\right)}& \left[\mathrm{Equation}<br><br><br><br>3\right]\end{array}$

12. The target speech extraction method of claim 9, further comprising, in a case where additional information representing that the target speech is a male speech is provided, calculating the LLRs LLR_i expressed by Equation 5 by applying pre-generated male and female GMMs λ

_Male and λ

_Female to the extracted feature vector sequence Xⁱ.

$\begin{array}{cc}{\mathrm{LLR}}_i=\log <br><br>\frac{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Male}}\right)}{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Female}}\right)}& \left[\mathrm{Equation}<br><br><br><br>5\right]\end{array}$

13. The target speech extraction method of claim 9, further comprising, in a case where additional information representing speech-music information is provided, calculating the LLRs LLR_i expressed by Equation 6 by applying GMMs λ

_Speech and λ

_Music generated based on the speech-music information to the extracted feature vector sequence Xⁱ.

$\begin{array}{cc}{\mathrm{LLR}}_i=\log <br><br>\frac{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Speech}}\right)}{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Music}}\right)}& \left[\mathrm{Equation}<br><br><br><br>6\right]\end{array}$

14. The target speech extraction method of claim 9, 10 further comprising, in a case where additional information representing speech-noise information is provided, calculating the LLRs LLR_i expressed by Equation 7 by applying GMMs λ

_Speech and λ

_Noise generated based on the speech-noise information to the extracted feature vector sequence Xⁱ.

$\begin{array}{cc}{\mathrm{LLR}}_i=\log <br><br>\frac{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Speech}}\right)}{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Noise}}\right)}& \left[\mathrm{Equation}<br><br><br><br>7\right]\end{array}$

15. The target speech extraction method of claim 9, further comprising, in a case where additional information representing that the target speech is a speech of a specific speaker is provided, calculating the LLRs LLR_i expressed by Equation 8 by applying pre-generated GMMs λ

_Individual and λ

_Others to the extracted feature vector sequence Xⁱ.

$\begin{array}{cc}{\mathrm{LLR}}_i=\log <br><br>\frac{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Individual}}\right)}{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Others}}\right)}& \left[\mathrm{Equation}<br><br><br><br>8\right]\end{array}$

16. The target speech extraction method of claim 9, further comprising, in a case where additional information representing that the target speech is a speech of a specific property A(Property_A) is provided, calculating the LLRs LLR_i expressed by Equation 10 by applying pre-generated GMMs λ

<sub>property—

A</sub> and λ

<sub>property—

B</sub> to the extracted feature vector sequence Xⁱ.

$\begin{array}{cc}{\mathrm{LLR}}_i=\log <br><br>\frac{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Property\_A}}\right)}{P<br><br>\left(X^i|{\mathrm{\lambda <br><br>}}_{\mathrm{Property\_B}}\right)}& \left[\mathrm{Equation}<br><br><br><br>10\right]\end{array}$

17. The target speech extraction method of claim 8, wherein the extracting of the target speech comprises:

in a case where the additional information for the target speech is not provided, performing primary speech recognition for the separated sound-source signals by using an HMM (hidden Markov model) as a speech-recognition acoustic model;

calculating a closest HMM and a state column thereof for a sequence of the words obtained through the speech recognition;

calculating LLRs (logarithm likelihood ratios) LLR_i by using the HMMs;

calculating a maximum value k by using the calculated LLRS;

comparing the maximum value with a predetermined threshold value; and

determining the maximum value k to be the target speech if the maximum value k is larger than the threshold value.

18. The target speech extraction method of claim 8, wherein the LLR_i is calculated by using the following Equation 9.

$\begin{array}{c}\begin{array}{c}{\mathrm{LLR}}_i=\end{array}\end{array}$

log 

P 

( X i | W ) P 

( X i | W _ ) = 

log 

∏

t 



P 

( x t i | s j m ) ∏

t 



P 

( x t i | s _ j m ) = 

∑

t 



log 



P 

( x t i | s j m ) 

P 

( x t i | s _ j m ) ≅



∑

t 



log 



P 

( x t i | s j m ) - 

log [ 1 K - 1 

∑

k = 1 



k ≠

m K 



exp 

( γ





log 



P 

[ x t i | s j k ] ) ] 1 / γ

] [ Equation 



9 ]

wherein, w denotes a sequence of the words obtained from the speech recognition unit 130, w denotes a complement of the w, s_j^m denotes a complement of the s^m_j, K is the number of the states included in all the HMMs corresponding to w, γ

is a constant term which can be determined experimentally.