Non-causal speaker selection for conference multicast

• US 6,963,353 B1
• Filed: 05/14/2003
• Issued: 11/08/2005
• Est. Priority Date: 05/14/2003
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Firestone, Scott
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US10/439,147
• Time in Patent Office: 909 Days
• Field of Search: 348 1401- 1409, 348/14.1, 348/14.11, 348/14.12, 348/14.13, 379/202.01, 709/204, 370/260, 370/261, 345/753
• US Class Current: 348/14.09
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method for non-causal speaker selection is provided. In accordance with a particular embodiment of the present invention the method includes receiving a plurality of video streams at a multipoint control unit, each of the plurality of video streams being associated with a respective endpoint of a multipoint conference. A plurality of audio streams may also be received at the multipoint control unit, and each audio stream may be associated with a respective one of the video streams. The audio streams are buffered in respective audio buffers, and the video streams are buffered in respective video buffers. First video data is copied from the video buffers to obtain a low latency video stream for distribution to active conference participants. In a particular embodiment, second video data may be copied from the video buffers to obtain a high latency video stream for distribution to passive conference participants, the high latency video streams being delayed in time with respect to the low latency video stream.

Claims

1. A method for non-causal speaker selection, comprising:

receiving a plurality of video streams at a multipoint control unit, each of the plurality of video streams being associated with a respective endpoint of a multipoint conference;

receiving a plurality of audio streams at the multipoint control unit, each audio stream being associated with a respective one of the video streams;

buffering the audio streams in respective audio buffers;

buffering the video streams in respective video buffers;

copying first video data from the video buffers to obtain a low latency video stream for distribution to active conference participants; and

copying second video data from the video buffers to obtain a high latency video stream for distribution to passive conference participants, the high latency video stream being delayed in time with respect to the low latency video stream.

2. The method for non-causal speaker selection of claim 1, further comprising:

copying first audio data from the audio buffers to obtain a low latency audio stream that corresponds to the low latency video stream;

copying second audio data from the audio buffers to obtain a high latency audio stream that corresponds to the high latency video stream; and

combining the first and second audio data with the first and second video data, respectively, for combined distribution to the active conference participants and the passive conference participants, respectively.

3. The method for non-causal speaker selection of claim 2, further comprising;

using the first audio data to detect a new speaker; and

switching the second video data based, at least in part, upon detection of the new speaker.

4. The method for non-causal speaker selection of claim 2, further comprising:

removing the second video data from the video buffers at approximately a time at which the second video data is copied to obtain the high latency video streams.

5. The method for non-causal speaker selection of claim 2, wherein the low latency video stream comprises a first low latency video stream, and further comprising:

copying third video data from the video buffers to obtain a second low latency video stream for distribution to a current speaker.

6. The method for non-causal speaker selection of claim 5, further comprising:

copying third audio data from the audio buffers to obtain a second low latency audio stream that corresponds to the second low latency video stream; and

combining the third audio data with the third video data, respectively, for distribution to the current speaker.

7. The method for non-causal speaker selection of claim 1, further comprising calculating current and previous speakers for a voice-activated switched multipoint conference by copying audio information from middle portions of each audio buffer in order to reduce hysteresis delay.

8. The method for non-causal speaker selection of claim 1, wherein each audio buffer includes a write pointer position, and further comprising:

selecting a common buffer position of each audio buffer from which to copy audio data, using a point in time which corresponds to an oldest write pointer location among all audio buffers.

9. The method for non-causal speaker selection of claim 1, wherein an amount of video data that is buffered in the video buffers is dynamically varied in accordance with an amount of memory available to a voice-activated switch module of an MCU.

10. The method for non-causal speaker selection of claim 1, wherein each video buffer includes a respective read pointer, and further comprising:

increasing an amount of video by moving ones of the read pointers that are associated with passive conference participants only.

11. The method for non-causal speaker selection of claim 1, wherein a video delay associated with the high latency video stream is greater than a hysteresis switching delay of the MCU.

12. A multipoint control unit, comprising:

an interface being operable to receive a plurality of video streams at a multipoint control unit, each of the plurality of video streams being associated with a respective endpoint of a multipoint conference;

the interface being further operable to receive a plurality of audio streams at the multipoint control unit, each audio stream being associated with a respective one of the video streams;

memory being operable to buffer the audio streams in respective audio buffers;

the memory being further operable to buffer the video streams in respective video buffers;

a processor being operable to copy first video data from the video buffers to obtain a low latency video stream for distribution to active conference participants;

the processor being further operable to copy second video data from the video buffers to obtain a high latency video stream for distribution to passive conference viewers, the high latency video stream being delayed in time with respect to the low latency video stream.

13. The multipoint control unit of claim 12, wherein the processor is further operable to:

copy first audio data from the audio buffers to obtain a low latency audio stream that corresponds to the low latency video stream;

copy second audio data from the audio buffers to obtain a high latency audio stream that corresponds to the high latency video stream;

combine the first and second audio data with the first and second video data, respectively, for combined distribution to the active conference participants and the passive conference participants, respectively.

14. The multipoint control unit of claim 13, wherein the processor is further operable to use the first audio data to detect a new speaker, and switch the second video data based upon detection of the new speaker.

15. The multipoint control unit of claim 12, wherein the processor is further operable to remove the second video data from the video buffers at approximately a time at which the second video data is copied to obtain the high latency video stream.

16. The multipoint control unit of claim 12, wherein the low latency video stream comprises a first low latency video stream, and wherein the processor is further operable to copy third video data from the video buffers to obtain a second low latency video stream for distribution to a current speaker.

17. The multipoint control unit of claim 16, wherein the processor is further operable to:

copy third audio data from the audio buffers to obtain a second low latency audio stream that corresponds to the second low latency video stream;

combine the third audio data with the third video data, respectively, for distribution to the current speaker.

18. The multipoint control unit of claim 12, wherein the processor is further operable to determine current and previous speakers for a voice-activated switched multipoint conference using audio information copied from middle portions of each audio buffer in order to reduce hysteresis delay.

19. The multipoint control unit of claim 12, wherein each audio buffer includes a write pointer position, and wherein the processor is further operable to select a common buffer position of each audio buffer from which to copy audio data, using a point in time which corresponds to an oldest write pointer location among all audio buffers.

20. The multipoint control unit of claim 12, wherein an amount of video data that is buffered in the video buffers is dynamically varied in accordance with an amount of memory available to a voice-activated switch module of an MCU.

21. The multipoint control unit of claim 12, wherein each video buffer includes a respective read pointer, and wherein the processor is further operable to increase an amount of video by moving ones of the read pointers that are associated with passive conference participants only.

22. The multipoint control unit of claim 12, wherein a video delay associated with the high latency video stream is greater than a hysteresis switching delay of the MCU.

23. Logic encoded in media for non-causal speaker selection, the logic being operable to perform the following steps:

receive a plurality of video streams at a multipoint control unit, each of the plurality of video streams being associated with a respective endpoint of a multipoint conference;

receive a plurality of audio streams at the multipoint control unit, each audio stream being associated with a respective one of the video streams;

buffer the audio streams in respective audio buffers;

buffer the video streams in respective video buffers;

copy first video data from the video buffers to obtain a low latency video stream for distribution to active conference participants; and

copy second video data from the video buffers to obtain a high latency video stream for distribution to passive conference participants, the high latency video stream being delayed in time with respect to the low latency video stream.

24. The logic encoded in media of claim 23, wherein the logic is further operable to:

copy first audio data from the audio buffers to obtain a low latency audio stream that corresponds to the low latency video stream;

copy second audio data from the audio buffers to obtain a high latency audio stream that corresponds to the high latency video stream; and

combine the first and second audio data with the first and second video data, respectively, for distribution to the active conference participants and the passive conference participants, respectively.

25. The logic encoded in media of claim 24, wherein the logic is further operable to:

use the first audio data to detect a new speaker; and

switch the second video data based upon detection of the new speaker.

26. The logic encoded in media of claim 24, wherein the logic is further operable to remove the second video data from the video buffers at approximately a time at which the second video data is copied to obtain the high latency video streams.

27. The logic encoded in media of claim 24, wherein the low latency video stream comprises a first low latency video stream, and wherein the logic is further operable to copy third video data from the video buffers to obtain a second low latency video stream for distribution to a current speaker.

28. The logic encoded in media of claim 27, wherein the logic is further operable to:

copy third audio data from the audio buffers to obtain a second low latency audio stream that corresponds to the second low latency video stream; and

combine the third audio data with the third video data, respectively, for distribution to the current speaker.

29. The logic encoded in media of claim 23, wherein the logic is further operable to calculate current and previous speakers for a voice-activated switched multipoint conference, by copying audio information from middle portions of each audio buffer, in order to reduce hysteresis delay.

30. A system for non-causal speaker selection, comprising:

means for receiving a plurality of video streams at a multipoint control unit, each of the plurality of video streams being associated with a respective endpoint of a multipoint conference;

means for receiving a plurality of audio streams at the multipoint control unit, each audio stream being associated with a respective one of the video streams;

means for buffering the audio streams in respective audio buffers;

means for buffering the video streams in respective video buffers;

means for copying first video data from the video buffers to obtain a low latency video stream for distribution to active conference participants; and

means for copying second video data from the video buffers to obtain a high latency video stream for distribution to passive conference participants, the high latency video steam being delayed in time with respect to the low latency video stream.

31. The system of claim 30, further comprising:

means for copying first audio data from the audio buffers to obtain a low latency audio stream that corresponds to the low latency video streams;

means for copying second audio data from the audio buffer to obtain a high latency audio stream that corresponds to the high latency video stream; and

means for combining the first and second audio data with the first and second video data, respectively, for combined distribution to the active conference participants and the passive conference participants, respectively.

32. The system of claim 31, further comprising:

means for using the first audio data to detect a new speaker; and

means for switching the second video data based upon detection of the new speaker.

33. The system of claim 31, further comprising means for removing the second video data from the video buffers at approximately a time at which the second video data is copied to obtain the high latency video streams.

34. The system of claim 31, wherein the low latency video stream comprises a first low latency video stream, and further comprising means for copying third video data from the video buffers to obtain a second low latency video stream for distribution to a current speaker.

35. The system of claim 34, further comprising:

means for copying third audio data from the audio buffers to obtain a second low latency audio stream that corresponds to the second low latency video stream; and

means for combining the third audio data with the third video data, respectively, for distribution to the current speaker.

36. The system of claim 30, further comprising means for calculating current and previous speakers for a voice activated switched multipoint conference by copying audio information from middle portions of each audio buffer in order to reduce hysteresis delay.