Stereo microphone processing for teleconferencing

• US 20060013416A1
• Filed: 06/30/2004
• Published: 01/19/2006
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Polycom Incorporated (HP Inc.)
• Inventors: Chu, Peter, Truong, Kwan K., Pocino, Michael A.
• Granted Patent: US 8,687,820 B2
• Time in Patent Office: Days
• US Class Current: 381/119
• CPC Class Codes: H04M 3/56 Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities video conference systems H04N7/15
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Methods and devices for improving the intelligibility of audio in a teleconferencing unit. Multiple microphones and multiple audio channels are used, in which only the best microphones are selected to represent each audio channel. Multiple microphones signals may be mixed according to microphones'"'"' positions in a room to form a single signal to represent one audio channel. The audio signal may be further processed to effectuate other features.

Claims

1. A method for processing multi-channel audio signals in a teleconference, the method comprising:

receiving microphone signals from a plurality of microphones for a plurality channels;

arbitrating the microphone signals for each channel to develop one or more selected microphone signals;

forming an audio signal for each channel from the one or more selected microphone signals; and

sending the audio signal for each channel to the far end.

2. The method in claim 1,

wherein the step of arbitrating the microphone signals for each channel includes:

comparing a microphone signal strength to a strength threshold;

selecting the microphone signal if its signal strength is greater than the strength threshold; and

discarding the microphone signal if its signal strength is less than the strength threshold.

3. The method in claim 2,

wherein selecting the microphone signal if its signal strength is greater than the strength threshold is to assign a microphone signal gain substantially greater than zero;

and

wherein discarding the microphone signal if its signal strength is less than the strength threshold is to assign a microphone signal gain substantially equal to zero.

4. The method in claim 1,

wherein the step of arbitrating the microphone signals for each channel includes:

selecting the microphone signal with the greatest signal strength; and

discarding the remaining microphone signals.

5. The method in claim 4,

wherein selecting the microphone signal with the greatest signal strength is to assign a microphone signal gain of 1 to the selected microphone signal;

and

wherein discarding the remaining microphone signals is to assign a microphone signal gain of 0 to the remaining microphone signals.

6. The method in claim 4,

wherein selecting the microphone signal with the greatest signal strength is to assign a microphone signal gain substantially equal to 1 to the selected microphone signal;

and

wherein discarding the remaining microphone signals is to assign a microphone signal gain substantially equal to 0 to the remaining microphone signals.

7. The method in claim 1, further comprising:

accumulating microphone signals for one time frame; and

splitting each microphone signals in the time frame into a plurality of frequency-band-limited sub-bands,

wherein arbitrating the microphone signals for each channel includes steps of;

for each sub-band, comparing the sub-band signals from all microphones;

giving a vote to the microphone having the greatest signal energy;

adding all votes for all sub-bands for each microphone; and

selecting a microphone signal whose number of votes exceeds a vote threshold for the time frame.

8. The method in claim 7,

wherein arbitrating the microphone signals for each channel further includes steps of:

if more than one microphones are selected whose number of votes exceeds a vote threshold for the time frame, keeping one microphone signal whose number of votes is the largest; and

discarding the other selected microphone signals.

9. The method in claim 7,

wherein arbitrating the microphone signals for each channel further includes a step of:

selecting the microphones that were selected for a previous time frame, if no microphone has votes exceeding the threshold for the time frame.

10. The method in claim 7,

wherein splitting each microphone signal in the time frame into a plurality of frequency-band-limited sub-bands is to splitting each microphone signal into at least 50 subbands, and the vote threshold is 16.

11. The method in claim 1, further comprising:

panning a selected microphone signal for one audio channel into the other audio channels based on the selected microphone'"'"'s position.

12. The method in claim 11, further comprising:

panning every selected microphone signal for every audio channel based on the microphone'"'"'s position.

13. The method in claim 1, further comprising:

showing images of conference participants on a video display; and

adjusting selected microphone signal levels to match the audio image position of a talker with the video image position of the talker as shown on the video display.

14. The method in claim 1, further comprising performing Automatic Gain Control.

15. The method in claim 1, further comprising performing noise suppression.

16. The method in claim 1, further comprising retrieving profiles of the selected microphones and processing the audio signals according to the retrieved profile of the selected microphones.

17. The method in claim 16, wherein the profile of each microphone includes at least the following information of the microphone:

the microphone position and the noise spectrum.

18. A teleconference unit comprising:

a network interface;

a processing module coupled to the network interface;

an audio input interface coupled to the processing module for receiving audio input signals; and

an audio output interface coupled to the processing module for sending out audio output signals to loudspeakers,

wherein the processing module is operable to;

receive a plurality of microphone signals from the audio input interface, each of which is assigned to one of a plurality of audio channels;

arbitrate the microphone signals for each audio channel to develop one or more selected microphone signals;

form an audio signal for each channel from the selected microphone signals;

encode audio channels into one or more audio signals;

send the audio signals to a far end teleconference unit via the network interface;

receive far end audio signals from the far end teleconference unit; and

decode the far end audio signals into far end audio channels; and

reproduce the far end audio channels through the loudspeakers via audio output interface.

19. The teleconference unit in claim 18, further comprising:

a plurality of microphones coupled to the audio input interface; and

a plurality of loudspeakers coupled to the audio output interface,

20. The conference unit in claim 18,

wherein the processing module is operable to arbitrate the microphone signals for each audio channel includes operability to,

compare the microphone signal strength to a strength threshold;

select the microphone signal if its signal strength is greater than the strength threshold; and

discard the microphone signal if its signal strength is less than the strength threshold.

21. The conference unit in claim 20,

wherein to select the microphone signal if its signal strength is greater than the strength threshold is to assign a microphone signal gain substantially greater than zero;

and

wherein to discard the microphone signal if its signal strength is less than the strength threshold is to assign a microphone signal gain substantially equal to zero.

22. The conference unit in claim 18,

wherein the processing module is operable to arbitrate the microphone signals for each audio channel includes operability to,

select the microphone signal with the greatest signal strength;

and

discard the remaining microphone signals.

23. The conference unit in claim 22,

wherein to select the microphone signal with the greatest signal strength is to assign a microphone signal gain of 1 to the selected microphone signal;

and

wherein to discard the remaining microphone signals is to assign a microphone signal gain of 0 to the remaining microphone signals.

24. The conference unit in claim 22,

wherein to select the microphone signal with the greatest signal strength is to assign a microphone signal gain substantially equal to 1 to the selected microphone signal;

and

wherein to discard the remaining microphone signals is to assign a microphone signal gain substantially equal to 0 to the remaining microphone signals.

25. The conference unit in claim 18,

wherein the processing module is further operable to,

accumulate microphone signals for one time frame;

and

split each microphone signals in the time frame into a plurality of frequency-band-limited sub-bands, and

wherein to arbitrate the microphone signals for each channel includes;

for each sub-band, to compare the sub-band signals from all microphones;

to give a vote to the microphone having the greatest signal energy;

to add all votes for all sub-bands for each microphone; and

to select a microphone signal whose number of votes exceeds a vote threshold for the time frame.

26. The conference unit in claim 25,

wherein to arbitrate the microphone signals for each channel includes:

if more than one microphones are selected whose number of votes exceeds a vote threshold for the time frame, to keep one microphone signal whose number of votes is the largest; and

to discard the other selected microphone signals.

27. The conference unit in claim 25,

wherein to arbitrate the microphone signals for each channel includes:

to select the microphones that were selected for a previous time frame,

if no microphone has votes exceeding the threshold for the time frame.

28. The conference unit in claim 25,

wherein to split each microphone signal in the time frame into a plurality of frequency-band-limited sub-bands is to split each microphone signal into at least 50 subbands, and the vote threshold is 16.

29. The conference unit in claim 18,

wherein the processing module is further operable to pan a selected microphone signal for one audio channel into the other audio channels based on the selected microphone'"'"'s position.

30. The conference unit in claim 29,

wherein the processing module is further operable to pan every microphone signal for every audio channels based on the microphone'"'"'s position.

31. The conference unit in claim 18, wherein the processing module is further operable to perform Automatic Gain Control.

32. The conference unit in claim 18, wherein the processing module is further operable to perform noise suppression.

33. The conference unit in claim 18, wherein the processing module is further operable to retrieve profiles of the selected microphones and process the audio signals according to the retrieved profile of the selected microphones.

34. The conference unit in claim 33, wherein the profile of each microphone includes at least the following information of the microphone:

the microphone position and the noise spectrum.

35. The teleconference unit in claim 18, further comprising:

a video input interface coupled to the processing module for receiving video signals; and

a video output interface coupled to the processing module for sending out video signals to a video display.

36. The teleconference unit in claim 35,

wherein the processing module is further operable to adjust selected microphone signals to match the audio image position of a talker with the video image position of the talker as shown on a video display.

37. A teleconference system comprising:

a first teleconference unit at a near end includes,

a first network interface,

a first processing module coupled to the first network interface,

a first audio input interface coupled to the first processing module for receiving audio input signals, and

a first audio output interface coupled to the first processing module for sending out audio output signals to loudspeakers; and

a second teleconference unit at a far end coupled to the first teleconference unit through a network, includes,

a second network interface coupled to the first network interface through the network,

a second processing module coupled to the second network interface,

a second audio input interface coupled to the second processing module for receiving audio input signals, and

a second audio output interface coupled to the second processing module for sending out audio output signals to loudspeakers;

wherein the first processing module is operable to;

receive a plurality of microphone signals from the first audio input interface, each of which is assigned to one of a plurality of audio channels,

arbitrate the microphone signals for each audio channel to develop one or more selected microphone signals,

form an audio signal for each channel from the selected microphone signals,

encode audio channels into one or more audio signals,

send the audio signals to the second teleconference unit via the first network interface,

receive far end audio signals from the second teleconference unit,

decode the far end audio signals into far end audio channels, and

reproduce the far end audio channels through the loudspeakers via the first audio output interface; and

wherein the second processing module is operable to;

receive a plurality of microphone signals from the second audio input interface, each of which is assigned to one of a plurality of audio channels,

arbitrate the microphone signals for each audio channel to develop one or more selected microphone signals,

form an audio signal for each channel from the selected microphone signals,

encode audio channels into one or more audio signals,

send the audio signals to the first teleconference unit via the second network interface,

receive near end audio signals from the first teleconference unit,

decode the near end audio signals into near end audio channels, and

reproduce the near end audio channels through the loudspeakers via the second audio output interface.

38. The teleconference system in claim 37,

wherein the first processing module is operable to arbitrate the microphone signals for each audio channel includes operability to,

compare the microphone signal strength to a strength threshold;

select the microphone signal if its signal strength is greater than the strength threshold; and

discard the microphone signal if its signal strength is less than the strength threshold.

39. The teleconference system in claim 38,

wherein to select the microphone signal if its signal strength is greater than the strength threshold is to assign a microphone signal gain substantially greater than zero;

and

wherein to discard the microphone signal if its signal strength is less than the strength threshold is to assign a microphone signal gain substantially equal to zero.

40. The teleconference system in claim 37,

wherein the first processing module is operable to arbitrate the microphone signals for each audio channel includes operability to,

select the microphone signal with the greatest signal strength;

and

discard the remaining microphone signals.

41. The teleconference system in claim 40,

wherein to select the microphone signal with the greatest signal strength is to assign a microphone signal gain of 1 to the selected microphone signal;

and

wherein to discard the remaining microphone signals is to assign a microphone signal gain of 0 to the remaining microphone signals.

42. The teleconference system in claim 40,

wherein to select the microphone signal with the greatest signal strength is to assign a microphone signal gain substantially equal to 1 to the selected microphone signal;

and

wherein to discard the remaining microphone signals is to assign a microphone signal gain substantially equal to 0 to the remaining microphone signals.

43. The conference system in claim 37,

wherein the first processing module is further operable to,

accumulate microphone signals for one time frame;

and

split each microphone signals in the time frame into a plurality of frequency-band-limited sub-bands, and

wherein to arbitrate the microphone signals for each channel includes;

for each sub-band, to compare the sub-band signals from all microphones;

to give a vote to the microphone having the greatest signal energy;

to add all votes for all sub-bands for each microphone; and

to select a microphone signal whose number of votes exceeds a vote threshold for the time frame.

44. The conference system in claim 43:

wherein to arbitrate the microphone signals for each channel includes;

if more than one microphones are selected whose number of votes exceeds a vote threshold for the time frame, to keep the one microphone signal whose number of votes is the largest; and

to discard the other selected microphone signals.

45. The conference system in claim 44:

wherein to arbitrate the microphone signals for each channel includes;

to select the microphones that were selected for a previous time frame, if no microphone has votes exceeding the threshold for the time frame.