Spatially correlated audio in multipoint videoconferencing

• US 20070064094A1
• Filed: 09/07/2005
• Published: 03/22/2007
• Est. Priority Date: 09/07/2005
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Polycom Incorporated (Plantronics, Inc.)
• Inventors: Potekhin, Sergey, Shani, Sharon, Knaz, Eran
• Granted Patent: US 7,612,793 B2
• Time in Patent Office: Days
• US Class Current: 348/14.80
• CPC Class Codes: H04M 2201/50 Telephonic communication in combination with video communication
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

The disclosed method provides audio location perception to an endpoint in a multipoint videoconference by providing a plurality of audio streams to the endpoint, wherein each of the audio streams corresponds to one of the loudspeakers at the endpoint. The audio streams are differentiated so as to emphasize broadcasting of the audio streams through one or more loudspeakers closest to a position of a speaking endpoint in a videoconference layout that is displayed at the endpoint. For example, the audio broadcast at a loudspeaker that is at a far-side of the screen might be attenuated or time delayed compared to audio broadcast at a loudspeaker that is located at a near-side of the display. The disclosure also provides a multipoint control unit (MCU) that processes audio signals from two or more endpoints according to the positions in a layout of the endpoints and then transmits processed audio streams to the endpoints in a way that allows endpoints to broadcast spatially correlated audio.

Claims

1. A method for controlling a first endpoint in a multipoint videoconference, the first endpoint comprising a plurality of loudspeakers spatially arranged with respect to a screen, comprising:

receiving audio and video image signals from a plurality of endpoints;

assessing from the audio signals which of the plurality of endpoints comprises a speaking endpoint;

generating a video layout for the first endpoint, the layout positioning video images from one or more of the plurality of endpoints at different positions in the layout; and

generating a plurality of audio streams for the first endpoint, each one of the plurality of audio streams corresponding to one of the plurality of loudspeakers,

wherein the audio streams are differentiated so as to generate a perception that the audio stream emanates from a position in the layout corresponding to a video image from the speaking endpoint.

2. The method of claim 1, wherein the speaking endpoint is not displayed on the screen and wherein the audio streams are differentiated as if the video image from the speaking endpoint is located at the center of the layout.

3. The method of claim 1, wherein the audio streams are differentiated by volume such that an audio stream that is broadcast through a loudspeaker that is further from the position of the video image from the speaking endpoint in the layout is attenuated compared an audio stream that is broadcast through a loudspeaker that is closer to the position of the video image from the speaking endpoint in the layout.

4. The method of claim 1, wherein the audio streams are differentiated such that an audio stream that is broadcast through a loudspeaker that is further from the position of the video image from the speaking endpoint in the layout is delayed compared an audio stream that is broadcast through a loudspeaker that is closer to the position of the video image from the speaking endpoint in the layout.

5. The method of claim 1, wherein the differentiating is adjustable by a user of the first endpoint.

6. The method of claim 1, generated plurality of audio streams is multiplexed.

7. The method of claim 1, wherein the position of the video image from the speaking endpoint in the video layout changes during the course of a videoconference.

8. A method for providing audio location perception to a first endpoint in a multipoint video conference, the endpoint comprising a plurality of loudspeakers, the method comprising:

providing a plurality of audio streams to the first endpoint, each one of the plurality of audio streams corresponding to one of the plurality of loudspeakers,

wherein the audio streams are differentiated so as to emphasize broadcasting of the audio stream through one or more loudspeakers closest to a position of a speaking endpoint in a videoconference layout.

9. The method of claim 8, further comprising, prior to providing the plurality of audio streams to the first endpoint:

receiving one or more audio signals from the speaking endpoint;

obtaining information regarding the position the speaking endpoint in the videoconference layout;

processing the one or more audio signals to form the audio streams in audio channels each associated with one of the loudspeakers of the first endpoint according to the information regarding the position in the layout of the speaking endpoint.

10. The method of claim 8, wherein the speaking endpoint is not displayed in the layout and wherein the audio signals are processed as if the speaking endpoint is located at the center of the display.

11. The method of claim 8, wherein the audio streams are differentiated by volume such that an audio stream that is broadcast through a loudspeaker that is further from the position of the speaking endpoint in the layout is attenuated compared an audio stream that is broadcast through a loudspeaker that is closer to the position of the speaking endpoint in the layout.

12. The method of claim 8, wherein the audio streams are differentiated such that an audio stream that is broadcast through a loudspeaker that is further from the position of the speaking endpoint in the layout is delayed compared an audio stream that is broadcast through a loudspeaker that is closer to the position of the speaking endpoint in the layout.

13. The method of claim 8, wherein the differentiating is adjustable by a user of the first endpoint.

14. The method of claim 8, wherein the provided plurality of audio streams is multiplexed.

15. The method of claim 8, wherein the position of the speaking endpoint in the videoconference layout changes during the course of a videoconference.

16. The method of claim 8, further comprising adjusting the loudspeakers of the first endpoint.

17. The method of claim 8, further comprising providing an interactive voice response (IRV) session for adjusting the speakers of the first endpoint.