Speech recognition apparatus, speech recognition apparatus and program thereof

• US 7,720,679 B2
• Filed: 09/24/2008
• Issued: 05/18/2010
• Est. Priority Date: 03/14/2002
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Takiguchi, Tetsuya, Ichikawa, Osamu, Nishimura, Masafumi
• Primary Examiner(s): Chawan; Vijay B
• Application Number:
  US12/236,588
• Publication Number:
  US 20090076815A1
• Time in Patent Office: 601 Days
• Field of Search: 704/231, 704/268, 704/261, 704/233, 704/226, 704/208, 704/234, 381/357, 381/61, 381/92, 381/96, 381/122
• US Class Current: 704/233
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/028 using properties of sound source

Abstract

Provided is a method for canceling background noise of a sound source other than a target direction sound source in order to realize highly accurate speech recognition, and a system using the same. In terms of directional characteristics of a microphone array, due to a capability of approximating a power distribution of each angle of each of possible various sound source directions by use of a sum of coefficient multiples of a base form angle power distribution of a target sound source measured beforehand by base form angle by using a base form sound, and power distribution of a non-directional background sound by base form, only a component of the target sound source direction is extracted at a noise suppression part. In addition, when the target sound source direction is unknown, at a sound source localization part, a distribution for minimizing the approximate residual is selected from base form angle power distributions of various sound source directions to assume a target sound source direction. Further, maximum likelihood estimation is executed by using voice data of the component of the sound source direction passed through these processes, and a voice model obtained by predetermined modeling of the voice data, and speech recognition is carried out based on an obtained assumption value.

Claims

1. A speech recognition apparatus comprising:

a microphone array comprising at least 3 microphones for measuring a profile of a base form sound from possible various sound source directions and a profile of a nondirectional background sound prior to recording a voice;

wherein each microphone measures a delay and a sum of peak power for each of a plurality of angles from a horizontal axis and from a vertical axis in response to a sound source located at a plurality of locations about said microphone array;

a database for storing said profile of said base form sound from said possible various sound source directions and said profile of said nondirectional background sound measured prior to said recording of said voice;

a sound source localization part for comparing a profile of the voice recorded by the microphone array with the profile of the base form sound from said possible various sound source directions and said profile of said nondirectional background sounds measured prior to said recording of said voice and stored in the database to estimate a sound source direction of the recorded voice; and

a speech recognition part for executing speech recognition of voice data of a component of the sound source direction estimated by the sound source localization part.

2. A speech recognition apparatus according to claim 1, wherein the sound source localization part compares profile obtained by combining the profile of the base form sound arriving from each possible sound location and background sound with profile of the recorded voice, and estimates a sound source location of the best-matched combination as a sound source location of the recorded voice based on a result of the comparison.

3. A speech recognition apparatus according to claim 1, further comprising:

a target location for said microphone array, where a voice and noise are recorded;

a noise suppressor, receiving a voice signal and a noise signal recorded at said target location by said microphone array.

4. A speech recognition apparatus according to claim 3, said noise suppressor comprising:

an array of delay and sum units, each delay and sum unit introducing a different delay from a range of negative and positive delays into said recording of said voice and said noise signal and producing a sum of peak power for said voice signal associated with each of said plurality of angles from said horizontal axis and with each of said plurality of angles from said vertical axis.

5. A speech recognition apparatus according to claim 4, wherein said voice signal associated with an angle of said horizontal axis and an angle of said vertical axis, corresponding to said target location, produces a maximal in-phase sum of peak power signal associated with said target location.

6. A speech recognition apparatus according to claim 5, said noise suppressor comprises an array of Fourier transform units, each Fourier transform unit corresponding to one of said array of delay and sum units and converting said voice signal from said one of said array of delay and sum units to a voice power distribution for each of a plurality of frequency bands correspondingly associated with each of said plurality of angles from said horizontal axis and from said vertical axis.

7. A speech recognition apparatus according to claim 6, said noise suppressor comprising an array of second profile fitting units, each said second profile fitting unit approximately decomposing said voice power distribution for each of said plurality of frequency bands, received from each Fourier transform units, providing a number of second profiles corresponding to said plurality of frequency bands, and selecting one of said second profiles based on correlating each of said voice power distributions that are approximately decomposed to each of said plurality of first directional sound source profiles, stored in said first directional sound source profile database, to one direction corresponding to said voice recorded at said target location.

8. A speech recognition apparatus according to claim 7, wherein said approximately decomposing comprises evaluating a directional target voice profile that equals a weighted sum of a first directional sound source profile for said white noise source in said one direction of said target location and a non-directional noise profile.

9. A speech recognition apparatus according to claim 8, wherein a weight coefficient of said first directional sound source profile and a weight coefficient for said non-directional noise profile are obtained by minimizing an evaluative function.

10. A speech recognition method according to claim 9, wherein a power of only a voice signal, without noise components, is determined for each of said plurality of frequency bands, based on said weight coefficient of said first directional sound source profile and said weight coefficient for said non-directional noise profile.

11. A speech recognition method for recognizing a voice inputted through a microphone array comprising at least 3 microphones by controlling a computer, comprising:

a voice inputting step of recording a voice by using the microphone array, and storing voice data in a memory;

wherein each microphone measures a delay and a sum of peak power for each of a plurality of angles from a horizontal axis and from a vertical axis in response to a white noise source located at a plurality of locations about said microphone array;

a sound source localization step of estimating a sound source direction of the recorded voice based on the voice data stored in the memory, and storing a result of the estimation in a memory;

a noise suppression step of decomposing the recorded voice into a component of a sound of the estimated sound source location, and a component of a nondirectional background sound based on the result of the estimation stored in the memory and information regarding premeasured profile of a predetermined voice, and storing voice data in which the component of the background sound from the recorded voice is canceled into a memory; and

a speech recognition step of recognizing the recorded voice based on the voice data in which the component of the background sound is canceled stored in the memory.

12. A speech recognition method according to claim 11, wherein the noise suppression step includes a step of further decomposing and canceling a component of a noise arriving from a specific direction from the recorded voice if the noise is estimated to arrive from the specific direction.

13. A speech recognition method according the claim 11, further comprising inputting a voice signal, recorded from said target location, and a noise signal from said recording into a noise suppressor for noise suppressing, said noise suppressing comprising:

introducing different a delay, from a range of negative and positive delays, into said recording of said voice signal and said noise signal by an array of delay and sum units, each said delay producing a sum of peak power for said voice signal associated with each of said plurality of angles from said horizontal axis and with each of said plurality of angles from said vertical axis.

14. A speech recognition method according the claim 13, wherein said voice signal associated with an angle of said horizontal axis and an angle of said vertical axis, corresponding to said target location, produces a maximal in-phase sum of peak power signal associated with said target location.

15. A speech recognition method according the claim 14, said noise suppressing comprising performing Fourier transforms by an array of Fourier transform units on signals received from said array of delay and sum units, each Fourier transform unit corresponding to one of said array of delay and sum units and converting said voice signal from said one of said array of delay and sum units to a voice power distribution for each of a plurality of frequency bands correspondingly associated with each of said plurality of angles from said horizontal axis and from said vertical axis.

16. A speech recognition method according the claim 15, said noise suppressing comprising approximately decomposing said voice power distributions, received from each of said Fourier transform units for each one of said plurality of frequency bands, by an array of second profile fitting units, each said second profile fitting unit providing a number of second profiles corresponding to said plurality of frequency bands and selecting one of said second profiles based on correlating each of said voice power distributions that are approximately decomposed to each of said plurality of first directional sound source profiles, stored in said first directional sound source profile database, to one direction corresponding to said voice recorded at said target location.

17. A speech recognition method according the claim 16, wherein said approximately decomposing comprises evaluating a directional target voice profile that equals a weighted sum of a first directional sound source profile for said white noise source in said one direction of said target location and a non-directional noise profile.

18. A speech recognition method according the claim 17, wherein a weight coefficient of said first directional sound source profile and a weight coefficient for said non-directional noise profile are obtained by minimizing an evaluative function.

19. A speech recognition method according the claim 18, wherein a power of only a voice signal, without noise, is determined for each said plurality of frequency bands, based on said weight coefficient of said first directional sound source profile and said weight coefficient for said non-directional noise profile.

20. A speech recognition method for recognizing a voice by use of a microphone array comprising at least 3 microphones by controlling a computer, comprising:

a voice inputting step of recording a voice by using the microphone array, and storing voice data in a memory,

wherein each microphone measures a delay and a sum of peak power for each of a plurality of angles from a horizontal axis and from a vertical axis in response to a white noise source located at a plurality of locations about said microphone array;

a sound source localization step of obtaining profile for various voice input directions by combining profiles of base form and nondirectional background sounds from a premeasured specific sound source direction, comparing the obtained profile with profile of the recorded voice obtained from the voice data stored in the memory to estimate a sound source direction of the recorded voice, and storing a result of the estimation in a memory;

a noise suppression step of extracting and storing voice data of the component of the estimated sound source direction of the recorded voice based on the estimation result of the sound source direction stored in the memory, and the voice data; and

a speech recognition step of recognizing the recorded voice based on voice data in which the component of the background sound is canceled stored in the memory.

21. A computer-readable medium encoded with a computer program for recognizing a voice by using a microphone array comprising at least 3 microphones by controlling a computer, making the computer execute:

a voice inputting process of recording a voice by using the microphone array, and storing voice data in a memory;

wherein each microphone measures a delay and a sum of peak power for each of a plurality of angles from a horizontal axis and from a vertical axis in response to a white noise source located at a plurality of locations about said microphone array;

a sound source localization process of estimating a sound source direction of the recorded voice based on the voice data stored in the memory, and storing a result of the estimation in a memory;

a noise suppression process of decomposing the recorded voice into a component of a sound of the estimated sound source direction and a component of a nondirectional background sound based on the result of the estimation stored in the memory and information regarding premeasured profile of a predetermined voice, and storing voice data in which the component of the background sound is canceled from the recorded voice in a memory; and

a speech recognition process of recognizing the recorded voice based on the voice data the component of the background sound is canceled stored in the memory.