Acoustic localization of a speaker

• US 9,338,549 B2
• Filed: 05/23/2012
• Issued: 05/10/2016
• Est. Priority Date: 04/17/2007
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Haulick, Tim, Schmidt, Gerhard Uwe, Buck, Markus, Wolff, Tobias
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): ODUNUKWE, UBACHUKWU A
• Application Number:
  US13/478,941
• Publication Number:
  US 20120294118A1
• Time in Patent Office: 1,448 Days
• Field of Search: 367/127, 367/118, 367/119, 367/99, 367/124, 381/92, 381/66, 381/300, 381/97, 381/111, 381/22, 381/71.11, 381/77, 381/58, 381/111.66, 381/56, 381/1, 381/17, 381/18, 704/233, 73/405.A, 379/406
• US Class Current: 1/1
• CPC Class Codes: G01S 13/003 Bistatic radar systems; Multistatic radar systems
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 15/00 Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
• CPC Class Codes: G01S 15/003 Bistatic sonar systems; Multistatic sonar systems
• CPC Class Codes: G01S 15/06 Systems determining the position data of a target
• CPC Class Codes: G01S 15/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S15/46
• CPC Class Codes: G01S 15/876 Combination of several spaced transmitters or receivers of known location for determining the position of a transponder or a reflector G01S15/872 takes precedence
• CPC Class Codes: G01S 3/00 Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received position-fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00
• CPC Class Codes: G01S 3/043 Receivers
• CPC Class Codes: G01S 3/802 Systems for determining direction or deviation from predetermined direction
• CPC Class Codes: G01S 3/808 using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G01S 5/0273 using multipath or indirect path propagation signals in position determination
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G01S 5/30 Determining absolute distances from a plurality of spaced points of known location
• CPC Class Codes: G01S 7/292 Extracting wanted echo-signals
• CPC Class Codes: G01S 7/52003 Techniques for enhancing spatial resolution of targets G01S7/52046 takes precedence
• CPC Class Codes: G01S 7/523 Details of pulse systems short-range imaging G01S7/52017; methods or devices for transmitting, conducting or directing sound G10K11/18
• CPC Class Codes: G10K 11/34 using electrical steering of transducer arrays, e.g. beam steering constructional aspects B06B1/0607, B06B1/085
• CPC Class Codes: G10K 15/08 Arrangements for producing a reverberation or echo sound modifying acoustic properties to change reverberation time G10K11/002
• CPC Class Codes: H04B 7/086 : using weights depending on external parameters, e.g. direction of arrival [DOA], predetermined weights or beamforming
• CPC Class Codes: H04M 1/6075 : adapted for handsfree use in a vehicle H04M1/6058 takes precedence; arrangements for holding telephones in a vehicle B60R11/0241
• CPC Class Codes: H04R 1/40 : by combining a number of identical transducers specially adapted for hearing aids H04R25/405
• CPC Class Codes: H04R 1/406 : microphones
• CPC Class Codes: H04R 2201/401 : 2D or 3D arrays of transducers
• CPC Class Codes: H04R 2201/403 : Linear arrays of transducers
• CPC Class Codes: H04R 2203/12 : Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
• CPC Class Codes: H04R 2410/01 : Noise reduction using microphones having different directional characteristics
• CPC Class Codes: H04R 2430/20 : Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/11 : Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 29/00 : Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308
• CPC Class Codes: H04R 29/001 : for loudspeakers H04R29/007 takes precedence
• CPC Class Codes: H04R 3/005 : for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 7/00 : Indicating arrangements; Control arrangements, e.g. balance control
• CPC Class Codes: H04S 7/30 : Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/301 : Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/302 : Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence
• CPC Class Codes: H04S 7/303 : Tracking of listener position or orientation
• CPC Class Codes: H04S 7/305 : Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence

Abstract

A system locates a speaker in a room containing a loudspeaker and a microphone array. The loudspeaker transmits a sound that is partly reflected by a speaker. The microphone array detects the reflected sound and converts the sound into a microphone array, the speaker'"'"'s distance from the microphone array, or both, based on the characteristics of the microphone signals.

Claims

1. A communication system adapted for a localization of a speaker in a room comprising:

at least one loudspeaker configured to output sound such that the sound is at least partly reflected by the speaker;

a microphone array configured to;

detect the sound output by the at least one loudspeaker and reflected by the speaker, the sound output by the at least one loudspeaker in a direct path to the microphone array, and the sound output by the at least one loudspeaker and reflected by a surface other than the speaker; and

obtain microphone signals for each microphone comprising the microphone array;

a beamformer configured to obtain at least one beamformed signal from the microphone signals;

a filter configured to estimate, using the at least one beamformed signal, an impulse response of a system comprising the at least one loudspeaker, the room, and the microphone array;

wherein the impulse response is determined based on the at least one loudspeaker, the room, and the microphone array; and

a processor programmed to determine the speaker'"'"'s direction towards and/or distance from the microphone array based on the estimated impulse responses.

2. The communication system according to claim 1, further comprising:

a loudspeaker array, including the at least one loudspeaker, configured to sequentially output sound in a number of predetermined directions; and

a controller configured to control the loudspeaker array to sequentially output the sound in a respective one of the number of predetermined directions and to simultaneously steer the microphone array to the respective one of the number of predetermined directions to obtain a beamformed signal for each of the predetermined directions, and where the filter is configured to estimate an impulse response of a system comprising the loudspeaker array, the room, and the microphone array for the obtained beamformed signals.

3. The communication system according to claim 1, where the processor is further configured to:

determine energy responses from each of the estimated impulse responses; and

generate a direction-distance diagram based on the determined energy responses; and

determine the speaker'"'"'s direction towards or distance from the microphone array based on the generated direction-distance diagram by determining a local maxima in the direction-distance diagram.

4. A communication system adapted for a localization of a speaker in a room comprising:

at least one loudspeaker configured to output sound such that the sound is at least partly reflected by the speaker;

a microphone array configured to detect the sound output by the at least one loudspeaker and reflected by the speaker, the sound output by the at least one loudspeaker in a direct path to the microphone array, and the sound output by the at least one loudspeaker and reflected by a surface other than the speaker; and

to

obtain microphone signals for each of a plurality of microphones comprising the microphone array;

a filter configured to estimate impulse responses of a system comprising the at least one loudspeaker, the room, and the microphone array for at least some of the microphone signals;

wherein the impulse response is determined based on the at least one loudspeaker, the room, and the microphone array;

a beamformer configured to beamform the estimated impulse responses to obtain at least one beamformed impulse response; and

a processor programmed to determine the speaker'"'"'s direction towards Of and distance from the microphone array based on the at least one beamformed impulse response.

5. The communication system according to claim 4, where the processor is further programmed to:

determine energy responses from each of the beamformed impulse responses;

generate a direction-distance diagram based on the determined energy responses; and

determine the speaker'"'"'s direction towards or distance from the microphone array based on the generated direction-distance diagram by determining a local maxima in the direction-distance diagram.

6. The communication system of claim 4, where the at least one loudspeaker, the microphone array, the filter, the beamformer, and the processor comprise a hands-free set.

7. The communication system of claim 4, where the at least one loudspeaker, the microphone array, the filter, the beamformer, and the processor comprise an audio or video conference system.

8. The communication system of claim 4, where the at least one loudspeaker, the microphone array, the filter, the beamformer, and the processor comprise a speech controller.

9. The communication system of claim 4, where the at least one loudspeaker, the microphone array, the filter, the beamformer, and the processor comprise a speech recognition device.

10. A method localization of a speaker in a room, comprising:

receiving microphone signals for each microphone in a microphone array, wherein the microphone array detects sound output by at least one loudspeaker and at least partly reflected by the speaker, sound output by the at least one loudspeaker in a direct path to the microphone array, and sound output by the at least one loudspeaker and reflected by a surface other than the speaker; and

obtaining at least one beamformed signal from the microphone signals;

estimating, using the at least one beamformed signal, an impulse response of a system comprising the at least one loudspeaker, the room, and the microphone array, wherein the impulse response is determined based on the at least one loudspeaker, the room, and the microphone array; and

determining, using a computer processor, the speaker'"'"'s direction towards and/or distance from the microphone array based on the estimated impulse responses.

11. The method according to claim 10, further comprising:

processing sound from a loudspeaker array, including the at least one loudspeaker, that is sequentially output in a number of predetermined directions; and

controlling the loudspeaker array to sequentially output the sound in a respective one of the number of predetermined directions and to simultaneously steer the microphone array to the respective one of the number of predetermined directions to obtain a beamformed signal for each of the predetermined directions, and where the filter is configured to estimate an impulse response of a system comprising the loudspeaker array, the room, and the microphone array for the obtained beamformed signals.

12. The method according to claim 11, further comprising:

determining energy responses from each of the estimated impulse responses; and

generating a direction-distance diagram based on the determined energy responses; and

determining the speaker'"'"'s direction towards or distance from the microphone array based on the generated direction-distance diagram by determining a local maxima in the direction-distance diagram.