APPARATUS AND METHOD FOR SPEECH RECOGNITION BASED ON SOUND SOURCE SEPARATION AND SOUND SOURCE IDENTIFICATION

• US 20100070274A1
• Filed: 07/07/2009
• Published: 03/18/2010
• Est. Priority Date: 12/09/2008
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: CHO, Hoon-Young, Hwang, Kyuwoong, Park, Sang Kyu, Kim, Seung Hi, Lee, Ilbin, Park, Jun, Lee, Yunkeun, Jeon, Hyung-Bae
• Application Number:
  US12/498,544
• Publication Number:
  US 20100070274A1
• Time in Patent Office: Days
• US Class Current: 704/233
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0272 Voice signal separating

Abstract

An apparatus for a speech recognition based on source separation and identification includes: a sound source separator for separating mixed signals, which are input to two or more microphones, into sound source signals by using independent component analysis (ICA), and estimating direction information of the separated sound source signals; and a speech recognizer for calculating normalized log likelihood probabilities of the separated sound source signals. The apparatus further includes a speech signal identifier identifying a sound source corresponding to a user'"'"'s speech signal by using both of the estimated direction information and the reliability information based on the normalized log likelihood probabilities.

Claims

1. An apparatus for a speech recognition based on source separation and identification, comprising:

a sound source separator for separating mixed signals, which are input to two or more microphones, into sound source signals by using independent component analysis (ICA), and estimating direction information of the separated sound source signals;

a speech recognizer for calculating normalized log likelihood probabilities of the separated sound source signals; and

a speech signal identifier identifying a sound source corresponding to a user'"'"'s speech signal by using the estimated direction information and reliability information based on the normalized log likelihood probabilities.

2. The apparatus of claim 1, wherein, under an assumption that noise sources are not moveable, the speech signal identifier estimates reference direction information of noise sources by using the estimated direction information and the obtained reliability.

3. The apparatus of claim 1, wherein the reference direction information of noise sources is updated with direction information of the noise sources output from the speech signal identifier.

4. The apparatus of claim 1,

wherein the sound source separator converts the mixed signals in a time domain into a frequency domain through Fast Fourier Transform;

computes an unmixing matrix by repeatedly executing a learning rule of an ICA algorithm; and

obtains separated signals in the time domain by converting separated signals in a frequency domain through Inverse Fourier Transform, the separated signals in the frequency domain being calculated by using the unmixing matrix.

5. The apparatus of claim 4, wherein the direction information of the separated source signals is determined in the sound source separator by obtaining two frequency response matrices from the unmixing matrix, deriving direction information of each of the separated source signals at a given frequency by using a ratio between the two frequency response matrices, and by averaging values of the latter direction information over the entire frequencies or an interval having a highly reliable value in the entire frequency band.

6. The apparatus of claim 1, wherein the speech recognizer calculates feature vectors for the sound source signals separated by the sound source separator at regular intervals, and calculates the normalized log likelihood probabilities by using the calculated feature vectors and a search network employing a hidden Markov model.

7. The apparatus of claim 1, wherein the speech recognizer determines, when a normalized log likelihood probability l_k is highest among the calculated normalized log likelihood probabilities, the kth separated sound source signal as the user'"'"'s speech signal.

8. The apparatus of claim 6, wherein the user speech signal identifier calculate, as reliability information for determining a first sound source of the highest normalized log likelihood probability l_k as the user'"'"'s speech signal, the reliability defined by a difference between the highest normalized log likelihood probability l_k and a second highest normalized log likelihood probability.

9. The apparatus of claim 8, wherein, when the reliability is greater than a preset threshold value, the user speech signal identifier outputs recognized one or more words of the first sound source corresponding to the reliability as a user'"'"'s speech, otherwise, sound source identification is performed by using respective direction information of the first sound source of the highest normalized log likelihood probability l_k and a second sound source of the second highest normalized log likelihood probability

10. The apparatus of claim 9, wherein the user speech signal identifier finds, when the reliability is less than or equal to the threshold value, a first reference value among reference direction information of noise sources closest to the direction information of the first sound source, calculates a first difference between the direction information of the first sound source and the found first reference value, finds a second reference value among the reference direction information of the noise sources closest to the direction information of the second sound source, and calculates a second difference between the direction information of the second sound source and the found second reference value;

and

determines the first sound source and second sound source as the user speech and a noise source, respectively, when the first difference is greater than the second difference, and determines the second sound source and first sound source as the user speech and a noise source, respectively, when the first difference is less than the second difference.

11. The apparatus of claim 9, wherein the user speech signal identifier includes a reference DOA storage storing, when the reliability is greater than the threshold value, the direction information of noise sources excluding the sound source corresponding to the reliability to the reference DOA update unit.

12. The apparatus of claim 11, wherein the reference DOA update unit compares the direction information of each noise source with existing reference direction information to find one of the reference direction information closest to the direction information, and updates the found reference direction information with a calculation result of the direction information and the found reference direction information.

13. A method for speech recognition based on source separation and source identification, comprising:

separating mixed signals, which are input to two or more microphones, into source signals by using independent component analysis (ICA), and estimating direction information (direction of arrival, DOA) of the separated sound source signals;

calculating normalized log likelihood probabilities of the separated sound source signals by normalizing the log likelihood values of separated source signals; and

identifying a sound source corresponding to a user'"'"'s speech signal using the estimated direction information and reliability information based on the normalized log likelihood probabilities.

14. The method of claim 13, wherein, under an assumption that noise sources are not movable, said identifying the sound source includes estimating reference direction information of noise sources by using the estimated direction information and the reliability.

15. The method of claim 14, wherein said separating the mixed signals includes:

converting the mixed signal in a time domain into a frequency domain through Fast Fourier Transform, computing unmixing matrix by repeatedly executing a learning rule of an ICA algorithm, and obtaining the separated source signals in the time domain by converting separated signals in a frequency domain through Inverse Fourier Transform, the separated signals in the frequency domain being calculated by using the unmixing matrix; and

obtaining two frequency response matrices from the unmixing matrix, and deriving direction information of each of the separated source signals at a given frequency by using a ratio between the two frequency response matrices, and determining the direction information of the separated source signals by averaging values of the direction information over the entire frequencies or an interval having a highly reliably value in the entire frequency band.

16. The method of claim 13, wherein calculating the normalized log likelihood probabilities comprises:

calculating feature vectors for the sound source signals separated by the sound source separator at regular intervals; and

calculating normalized log likelihood probabilities by using the calculated feature vectors and a search network employing a set of hidden Markov models.

17. The method of claim 13, wherein said calculating the normalized log likelihood probabilities includes determining, when a normalized log likelihood probability l_k is highest among the calculated normalized log likelihood probabilities, the kth separated sound source signal as the user'"'"'s speech signal.

18. The method of claim 13, wherein said identifying the sound source includes calculating, as reliability information for determining a first sound source of the highest normalized log likelihood probability l_k as the user'"'"'s speech signal, the reliability defined by a difference between the highest normalized log likelihood probability l_k and a second highest normalized log likelihood probability.

19. The method of claim 18, wherein said identifying the sound source includes:

when the reliability is greater than a preset threshold value, outputting recognized one or more words of the first sound source corresponding to the reliability as a user'"'"'s speech,

otherwise, performing source identification by using respective direction information of the first sound source of the highest normalized log likelihood probability l_k and a second sound source with the second highest normalized log likelihood probability.

20. The method of claim 19, wherein said identifying the sound source includes:

finding, when the reliability is less than or equal to the threshold value, a first reference value among reference direction information of noise sources closest to the direction information of the first sound source, and calculating a first difference between the direction information of the first sound source and the found first reference value, a second reference value among reference direction information of noise sources closest to the direction information of the second sound source, and calculating a second difference between the direction information of the second sound source and the found second reference value; and

determining the first sound source and second sound source as the user speech and a noise source, respectively, when the first difference is greater than the second difference, and determining the second sound source and first sound source as the user speech and a noise source, respectively, when the first difference is less than the second difference.