Sound source separation device, speech recognition device, mobile telephone, sound source separation method, and program

• US 8,112,272 B2
• Filed: 08/11/2006
• Issued: 02/07/2012
• Est. Priority Date: 08/11/2005
• Status: Active Grant

• Current Assignee: Asahi Kasei Kabushiki Kaisha
• Original Assignee: Asahi Kasei Kabushiki Kaisha
• Inventors: Nagahama, Katsumasa, Matsui, Shinya
• Primary Examiner(s): Azad, Abul K
• Application Number:
  US11/990,200
• Publication Number:
  US 20090055170A1
• Time in Patent Office: 2,006 Days
• Field of Search: 704/205, 704/226, 704/233, 381/10, 381/86, 381/91, 381/92, 381/389
• US Class Current: 704/226
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles

Abstract

A sound source signal from a target sound source is allowed to be separated from a mixed sound which consists of sound source signals emitted from a plurality of sound sources without being affected by uneven sensitivity of microphone elements. A beamformer section 3 of a source separation device 1 performs beamforming processing for attenuating sound source signals arriving from directions symmetrical with respect to a perpendicular line to a straight line connecting two microphones 10 and 11 respectively by multiplying output signals from the microphones 10 and 11 after spectrum analysis by weighted coefficients which are complex conjugate to each other. Power computation sections 40 and 41 compute power spectrum information, and target sound spectrum extraction sections 50 and 51 extract spectrum information of a target sound source based on a difference between the power spectrum information.

Claims

1. A sound source separation device for separating a sound source signal of a target sound source from a mixed sound which includes sound source signals emitted from a plurality of sound sources using at least two microphones arranged separately from each other comprising:

beamforming means for

performing a first beamforming processing to attenuate a sound source signal arriving from a predetermined direction by performing computations using first coefficients on an output signal of said microphones, and

performing a second beamforming processing to attenuate a sound source signal arriving from a direction symmetrical to said predetermined direction with respect to a perpendicular line to a straight line connecting the two microphones by performing computations using second coefficients which are complex conjugate of said first coefficients in a frequency domain on the output signal of said microphone;

power computation means for computing power spectrum information with respect to each of sound source signals obtained by said beamforming means; and

target sound spectrum extraction means for extracting spectrum information of a target sound source based on a difference between the power spectrum information calculated by said power computation means.

2. The sound source separation device according to claim 1, wherein said beamforming means applies said first beamforming processing and said second beamforming processing to each of a combination of any two microphones among three microphones which arranged separately from each other and another combination of two microphones among said three microphones.

3. The sound source separation device according to claim 1 or 2, further comprising directional characteristics control means for applying a delay to an output signal of a microphone.

4. The sound source separation device according to claim 3, wherein said directional characteristics control means virtually generates output signals of three microphones by applying a delay to an output signal of at least one microphone among two microphones.

5. The sound source separation device according to claim 3, further comprising arrival direction estimating means for estimating a arrival direction of said sound source signal, wherein said directional characteristics control means applies a delay to an output signal of the microphone such that two sound sources virtually locate symmetrically with respect to a perpendicular line to a straight line connecting the two microphones based on an arrival direction estimated by said arrival direction estimating means.

6. The sound source separation device according to claim 1 or 2, further comprising spectral subtraction means for performing spectral subtraction processing on the power spectrum information extracted by said target sound spectrum extraction means.

7. The sound source separation device according to claim 1 or 2, further comprising stationary noise reduction means for performing processing to reduce noise before the processing by said beamforming means is performed.

8. A speech recognition device comprising speech recognition means for performing speech recognition of a sound source signal separated by the sound source separation device according to claim 1 or 2.

9. The speech recognition device according to claim 8, further comprising recognition vocabulary list storage means for storing a driver seat side recognition vocabulary list which is a list of candidates of vocabulary spoken from a driver'"'"'s seat side of a vehicle and a passenger'"'"'s seat side recognition vocabulary list which is a list of candidates of vocabulary spoken from a passenger'"'"'s seat side, wherein said speech recognition means performs speech recognition processing of a sound source signal separated by said sound source separation device based on the driver'"'"'s seat side recognition vocabulary list and the passenger'"'"'s seat side recognition vocabulary list stored in said recognition vocabulary list storage means.

10. The speech recognition device according to claim 8, further comprising:

state transition means for managing a current state of a vehicle;

valid vocabulary list storage means for storing a valid vocabulary list of the passenger'"'"'s seat side and the driver'"'"'s seat side depending on a state of the vehicle; and

control means for determining whether a vocabulary item recognized by said speech recognition means is valid or not based on the current state of the vehicle managed by said state transition means and the vocabulary list stored in said valid vocabulary list storage means, and controlling depending on the determination result.

11. A mobile phone comprising the sound source separation device according to claim 1 or 2.

12. A sound source separation method comprising:

a sound source signal receiving step of inputting sound source signals emitted from a plurality of sound sources to at least two microphones arranged separately from each other;

a beamforming processing step of performing a first beamforming processing and a second beamforming processing to attenuate sound source signals arriving from predetermined directions symmetrical with respect to a perpendicular line to a straight line connecting two microphones respectively by performing computations using two weighted coefficients which are complex conjugate to each other in a frequency domain on an output signal of said microphone respectively;

a power computation step of computing power spectrum information with respect to each of sound source signals obtained in said beamforming processing step; and

a target sound spectrum extracting step of extracting spectrum information of a target sound source based on a difference between the power spectrum information calculated in said power computation step.

13. A tangible, non-transitory, computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a sound source separation method, comprising:

acquiring an output signal which includes sound source signals emitted from a plurality of sound sources are mixed from at least two microphones arranged separately from each other;

performing a first beamforming processing and a second beamforming processing to attenuate the sound source signals arriving from predetermined directions symmetrical with respect to a perpendicular line to a straight line connecting the two microphones respectively by performing computations using two weighted coefficients which are complex conjugate to each other in a frequency domain on the acquired output signal;

computing power spectrum information with respect to each of the sound source signals; and

extracting spectrum information of a target sound source based on a difference between the computed power spectrum information.