Arrangement and method for audio source tracking

• US 7,586,513 B2
• Filed: 04/07/2004
• Issued: 09/08/2009
• Est. Priority Date: 05/08/2003
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Tandberg Telecom AS (Cisco Systems, Inc.)
• Inventors: Marton, Trygve Frederik, Johansen, Tom-Ivar, Muren, Petter
• Primary Examiner(s): Chin; Vivian
• Assistant Examiner(s): Tran; Con P
• Application Number:
  US10/819,108
• Publication Number:
  US 20050008169A1
• Time in Patent Office: 1,980 Days
• Field of Search: 381/92, 381/122, 348/14.08, 348/14.09, 348/14.01, 348/169, 348/240.99, 348/211.4, 348/211.99, 348/211.9, 348/211.11, 348/211.12, 348/14.1
• US Class Current: 348/14.01
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

An arrangement and a method of localizing active speakers in a video conference includes a localization device that locates at least one microphone relative to the camera, while the at least one microphone in turn localizes the relative positions of audio sources. As the microphones are usually positioned close to the audio source in a video conference, the ratio of the distance between microphones relative to the distance between table microphone and audio source is reduced. Thus, the microphones are able to determine the positions of the audio sources with a higher resolution than if placed close to the camera. When the respective positions of the microphones relative to the camera and the audio source are known, the position of the audio source relative to the camera is then determined by means of geometrical calculations.

Claims

1. A method of localizing a position of an audio source relative to a camera, comprising steps of:

determining a position of the audio source relative to at least one of a microphone and array of microphone elements;

geometrically deriving, by a control unit, a second distance and direction between the camera and one of the at least one of the microphone and array of microphone elements; and

geometrically deriving, by the control unit through a mathematical analysis, a first distance and direction between the camera and the audio source, for a system wherein a distance between the at least one of the microphone and array of microphone elements and the camera is greater than a distance between the audio source and the at least one of the microphone and array of microphone elements, from the position of the audio source relative to one of the at least one the microphone and array of microphone elements and said second distance and direction,

wherein said step of geometrically deriving said second distance or direction between the camera and one of the at least one of the microphone and array of microphone elements, further comprises;

transmitting a sound signal generated by a machine from a known or detectable position relative to the camera;

respectively receiving the sound signal in at least two of the microphones and arrays of microphone elements; and

processing the received sound signals for calculating said second distance or direction between the camera and one of the at least one of the microphone and array of microphone elements,

wherein the at least one of the microphone and array of microphone elements is remote from the camera.

2. The method according to claim 1, wherein determining a position of the audio source relative to the at least one of the microphone and array of microphone elements includes:

detecting a respective time difference of receiving audio signals from the audio source to the at least one of the microphone and array of microphone elements for at least one of a pair of microphones, a pair of arrays of microphone elements, and a pair of a microphone and an array of microphone elements.

3. The method according to claim 1, wherein said second distance or direction between the camera and one of the at least one of the microphone and array of microphone elements is fixed.

4. The method according to claim 1, wherein said step of determining the position of the audio source relative to the at least one of the microphone and array of microphone elements further comprises:

calculating a first time difference for receiving audio signals from the audio source to the respective microphones and arrays of microphone elements of each possible pair of microphones and arrays of microphone elements for each point in a predefined set of points;

measuring a second time difference for receiving audio signals from the audio source to the respective microphones and arrays of microphone elements of each possible pair of microphones and arrays of microphone elements;

calculating values of an error function for each point of the predefined set of points by summarizing the squared difference between the corresponding first and second time differences for each possible pair of microphones and arrays of microphone elements; and

selecting a point associated with a least value of the error function as the position of the audio source.

5. The method according to claim 1, further comprising:

executing a look up with said first distance or direction in a table associating various distances and directions with corresponding camera zooming quantities and orientations, respectively; and

zooming or orienting the camera according to a result of said look up.

6. The method of claim 1, wherein a location of the camera is unfixed with respect to the control unit.

7. A method of localizing a position of an audio source relative to a camera, comprising steps of:

determining a position of the audio source relative to at least one of a microphone and array of microphone e1ements;

geometrically deriving, by a control unit, a second distance and direction between the camera and one of the at least one of the microphone and array of microphone elements; and

geometrically deriving, by the control unit through a mathematical analysis, a first distance and direction between the camera and the audio source, for a system wherein a distance between the at least one of the microphone and array of microphone elements and the camera is greater than a distance between the audio source and the at least one of the microphone and array of microphone elements, from the position of the audio source relative to one of the at least one the microphone and array of microphone elements and said second distance and direction,

wherein said step of geometrically deriving said second distance and direction between the camera and one of the at least one of the microphone and array of microphone elements, further comprises;

providing said one of the at least one of the microphone and array of microphone elements with a recognizable pattern;

identifying the recognizable pattern within the view captured by the camera; and

determining said second distance and direction based on a size or a position of the recognizable pattern within the view.

8. The method of claim 7, wherein the recognizable pattern includes two or more controllable light sources, and the method includes a step of using the camera to locate the two or more controllable light sources.

9. A system comprising:

an apparatus configured to determine a position of an audio source relative to a camera that determines a position of the audio source relative to at least one of a microphone and an array of microphone elements, said apparatus including;

a control unit configured to

determine a position of the audio source relative to at least one of a microphone and array of microphone elements,

geometrically derive a second distance and direction between the camera and one of the at least one of the microphone and array of microphone elements, and

geometrically derive, through use of a mathematical analysis, a first distance and direction between the camera and the audio source, for a system wherein a distance between the at least one of the microphone and array of microphone elements and the camera is greater than a distance between the audio source and the at least one of the microphone and array of microphone elements, from the position of the audio source relative to one of the at least one the microphone and array of microphone elements and said second distance and direction;

a sound signal transmitter device positioned in a known or detectable position relative to said camera and configured to transmit a sound signal; and

a sound signal transceiver that receives the sound signal in at least two of the microphones and arrays of microphone elements and transmits the received sound signals to the control unit, which is configured to process the received sound signals to calculate said second distance or direction between the camera and one of the at least one of the microphone and array of microphone elements,

wherein the at least one of the microphone and array of microphone elements is remote from the camera.

10. The system according to claim 9, wherein the control unit is further configured to detect a respective time difference of receiving audio signals from the audio source to the at least one of the microphone and array of microphone elements for at least one of a pair of microphones, a pair of arrays of microphone elements, and a pair of a microphone and an array of microphone elements.

11. The system according to claim 10, wherein the control unit is further configured to calculate a first time difference for receiving audio signals from the audio source to the respective microphones and arrays of microphone elements of each possible pair of microphones and arrays of microphone elements for each point in a predefined set of points, measure a second time difference for receiving audio signals from the audio source to the respective microphones and arrays of microphone elements of each possible pair of microphones and arrays of microphone elements, calculate values of an error function for each point of the predefined set of points by summarizing the squared difference between the corresponding first and second time differences for each possible pair of microphones and arrays of microphone elements, and select a point associated with a least value of the error function as the position of the audio source.

12. The system according to claim 9, further comprising:

the at least one of the microphone and array of microphone elements; and

a video camera,

wherein said second distance or direction between the camera and one of the at least one of the microphone and array of microphone elements is fixed.

13. The system according to claim 9, where the control unit is configured to access a look up table associating various distances and directions with corresponding camera zooming quantities and orientations, respectively, and the control unit is configured to zoom or orient the camera according to a zooming quantity or orientation associated with said first distance or direction.

14. The system of claim 9, wherein a location of the camera is unfixed with respect to the control unit.

15. A system comprising:

an apparatus configured to determine a position of an audio source relative to a camera that determines a position of the audio source relative to at least one of a microphone and an array of microphone elements, said apparatus including;

a control unit configured to

determine a position of the audio source relative to at least one of a microphone and array of microphone elements,

geometrically derive a second distance and direction between the camera and one of the at least one of the microphone and array of microphone elements, and

geometrically derive, through use of a mathematical analysis, a first distance and direction between the camera and the audio source, for a system wherein a distance between the at least one of the microphone and array of microphone elements and the camera is greater than a distance between the audio source and the at least one of the microphone and array of microphone elements, from the position of the audio source relative to one of the at least one the microphone and array of microphone elements and said second distance and direction;

the at least one of the microphone and array of microphone elements; and

the camera,

wherein said one of the at least one of the microphone and array of microphone elements is provided with a recognizable pattern, the camera or the control unit is configured to identify the recognizable pattern within the view captured by the camera, and the control unit is configured to determine said second distance or direction based on a size or a position of the recognizable pattern within the view.

16. The system of claim 15, wherein the recognizable pattern includes two or more controllable light sources, and the control unit is configured to use the camera to locate the two or more controllable light sources.