AUDIO PROCESSING IN A MULTI-PARTICIPANT CONFERENCE

• US 20110074914A1
• Filed: 11/29/2010
• Published: 03/31/2011
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Hyeonkuk Jeong, Ryan Salsbury
• Original Assignee: Hyeonkuk Jeong, Ryan Salsbury
• Inventors: Jeong, Hyeonkuk, Salsbury, Ryan
• Granted Patent: US 8,456,508 B2
• Time in Patent Office: Days
• US Class Current: 348/14.08
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Some embodiments provide an architecture for establishing multi-participant audio conferences over a computer network. This architecture has a central distributor that receives audio signals from one or more participants. The central distributor mixes the received signals and transmits them back to participants. In some embodiments, the central distributor eliminates echo by removing each participant'"'"'s audio signal from the mixed signal that the central distributor sends to the particular participant.

Claims

1-25. -25. (canceled)

26. A non-transitory computer readable medium of a device of a first participant of a multi-participant conference, the computer readable medium storing a computer program which when executed by at least one processor of the device distributes audio content in a multi-participant conference, the computer program comprising sets of instructions for:

receiving audio signals from at least second and third participants of said conference;

generating data representative of strengths of the received audio signals;

generating mixed audio signals from the received audio signals;

to each particular mixed audio signal, appending a set of generated strength data for the audio signals that are mixed to produce the particular mixed audio signal; and

transmitting said mixed audio signals with the strength data to the second and third participants of the conference.

27. The non-transitory computer readable medium of claim 26, wherein the computer program further comprises a set of instructions for determining a strength of each received audio signal, the set of instructions for determining said strength comprising a set of instructions for calculating the strength of each received audio signal as a root mean square (RMS) power of said received audio signal.

28. The non-transitory computer readable medium of claim 26, wherein the set of instructions for generating a mixed audio signal for a particular participant comprises a set of instructions for removing the audio signal of the particular participant.

29. The non-transitory computer readable medium of claim 26, wherein the appended strength data is for use by devices of the second and third participants for panning the mixed audio signal across audio loudspeakers.

30. The non-transitory computer readable medium of claim 26, wherein the set of instructions for generating the mixed audio signals comprises a set of instructions for generating a first mixed audio signal for the second participant and generating a second, different mixed audio signal for the third participant.

31. The non-transitory computer readable medium of claim 26, wherein the mixed audio signals are transmitted using real time protocol (RTP) packets comprising the strength data for each of the audio signals in the mixed audio signals.

32. The non-transitory computer readable medium of claim 26, wherein the computer program further comprises a set of instructions for locally outputting the mixed audio signals at the device of the first participant.

33. A method of distributing audio content in a multi-participant conference, the method comprising:

at a device of a first participant of the conference;

receiving audio signals from at least second and third participants of the conference;

generating a single mixed audio signal comprising an audio signal locally captured at the device of the first participant and the received audio signals;

transmitting the mixed audio signal to the second and third participants; and

removing the audio signal of the device from the mixed audio signal in order to locally output the mixed audio signal with the audio signal of the device removed.

34. The method of claim 33 further comprising generating strength data that represents strengths of (i) the audio signal captured by the device of the first participant and (ii) each of the received audio signals.

35. The method of claim 34, wherein the strength data comprises the strength of each of the audio signals as a root mean square (RMS) of the audio signals.

36. The method of claim 33, wherein the single mixed audio signal is generated by the device using the audio signals of the first, second, and third participants.

37. The method of claim 33 further comprising locally outputting the mixed audio signal with the audio signal of the device removed.

38. The method of claim 34, wherein the strength data is for panning the mixed audio signal across audio loudspeakers of the first, second, and third devices.

39. The method of claim 33, wherein the device generates all mixed audio signals during the conference.

40. A non-transitory computer readable medium of a first device of a first participant of a multi-participant conference, the computer readable medium storing a computer program which when executed by at least one processor of the device receives audio content in a multi-participant conference, the computer program comprising sets of instructions for:

receiving a mixed audio signal from a second device of a second participant of said conference, the mixed audio signal comprising (i) an audio signal sent from the first device to the second device, (ii) audio captured by the second device, and (iii) an audio signal sent from a third device to the second device;

removing the audio signal of the first device from the mixed audio signal; and

outputting the mixed audio signal with the audio signal of the first device removed.

41. The non-transitory computer readable medium of claim 40, wherein the computer program further comprises a set of instructions for sending the audio signal of the first device to the second device of the second participant.

42. The non-transitory computer readable medium of claim 40, wherein the computer program further comprises a set of instructions for receiving strength data that represents strengths of audio signals received at the second device of the second participant.

43. The non-transitory computer readable medium of claim 42, wherein the set of instructions for outputting the mixed audio signal comprises a set of instructions for using the strength data is to pan the mixed audio signal across audio loudspeakers of the first device.

44. The non-transitory computer readable medium of claim 42, wherein the strength data is generated by the second device by calculating a strength of each audio signal of the mixed audio signal as a root mean square (RMS) power of the audio signal.

45. The non-transitory computer readable medium of claim 40, wherein the second device generates all mixed audio signals during the conference.