SYSTEM AND METHOD FOR A CONFERENCE SERVER ARCHITECTURE FOR LOW DELAY AND DISTRIBUTED CONFERENCING APPLICATIONS

• US 20080158339A1
• Filed: 01/17/2008
• Published: 07/03/2008
• Est. Priority Date: 07/20/2005
• Status: Active Grant

• Current Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Original Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Inventors: Eleftheriadis, Alexandros, Shapiro, Ofer, Civanlar, Reha
• Granted Patent: US 7,593,032 B2
• Time in Patent Office: Days
• US Class Current: 348/14.09
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

Systems and methods for conducting a multi-endpoint video signal conference are provided. Conferencing endpoints are linked by pairs of a reliable and a less reliable communication channel. Conference video signals are scaleable coded in base layer and enhancement layers format. Video signal base layers, which correspond to a minimum picture quality, are communicated over reliable channels. The video signal enhancements layers may be communicated over the less reliable channels. A conference server mediates the switching of video layer information from transmitting endpoints to receiving endpoints without any intermediate coding or re-coding operations. The video conference can be integrated with an audio conference using either scalable coded audio signals or non-scaleable coded audio signals.

Claims

1. A multi-endpoint video signal conferencing system, wherein video signals are scalably coded into layers including a base layer and one or more enhancement layers, the conferencing system comprising:

a scalable video coding server (SVCS) linked to at least one receiving and at least one transmitting endpoint by at least one communication channel each,

wherein at least one of the communication channels linking the SVCS with each of the endpoints offers improved quality of service; and

wherein the SVCS is configured to selectively forward a video signal layer received from a transmitting endpoint over its at least one linking communication channel to a receiving endpoint over its at least one linking communication channel.

2. The conferencing system of claim 1 wherein the SVCS is configured to forward the video signal layer received from the transmitting endpoint to the receiving endpoint using a dynamic switching matrix.

3. The conferencing system of claim 1 wherein the SVCS is configured to forward a video signal received from a transmitting endpoint to a receiving endpoint without decoding and/or re-coding the video signal.

4. The conferencing system of claim 1 wherein the SVCS is configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to at least one endpoint linked through the SVCS by selectively multiplexing and forwarding video signal layers to the linked endpoints.

5. The conferencing system of claim 4 wherein the SVCS is configured to selectively multiplex and forward video signal layers having different signal characteristics to a receiving endpoint, the different signal characteristics including at least one of different resolution, bit rate, quality, and frame rate characteristics.

6. The conferencing system of claim 4 wherein the SVCS is further configured to respond to bandwidth conditions by at least one of:

statistically multiplexing video signals from a plurality of transmitting endpoints; and

synchronizing the transmission of video signals from a plurality of transmitting endpoints to stagger larger-than-average video frames-in the multiplexed video signal.

7. The conferencing system of claim 4 wherein the SVCS is configured to forward enhancement signal layers to receiving endpoints in priority according to a conferencing system priority policy, which assigns priority to receiving endpoints.

8. The conferencing system of claim 1, wherein the SVCS is further configured to process audio signals in addition to processing video signals.

9. The conferencing system of claim 1, further comprising a plurality of linked SVCS.

10. The conferencing system of claim 9 wherein the linked SVCS are disposed over heterogeneous communication network domains.

11. The conferencing system of claim 1, wherein the SVCS is further configured to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

12. A multi-endpoint audio signal conferencing system, wherein audio signals are coded in components such that multiple qualities can be derived from the bitstream in the coded domain, the conferencing system comprising:

a scaleable audio coding server (SACS) linked to at least one receiving and at least one transmitting endpoint in an audio conference by at least one communication channel each, and

wherein the SACS is configured to selectively forward an audio signal component received from a transmitting endpoint over its at least one linking communication channel to a receiving endpoint over its at least one linking communication channel.

13. The conferencing system of claim 12, wherein the audio signal is scalably coded into layered components comprising a base layer and one or more enhancement layers.

14. The conferencing system of claim 12, wherein at least one of the communication channels linking the SVCS with each of the endpoints offers improved quality of service.

15. The conferencing system of claim 12, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators.

16. The conferencing system of claim 15, wherein the SACS is further configured to.

forward all quality components of the strongest received audio signal to all participants except the one it was transmitted from;

forward less than all quality components of a number of the less strong received audio signals to all participants except the one they were transmitted from; and

forward no quality components of the remaining less strong received audio signals.

17. The conferencing system of claim 12, wherein the audio signals received from transmitting endpoints are forwarded to the receiving endpoints by the SACS and mixed at the receiving endpoints.

18. The conferencing system of claim 12, wherein the SACS is configured to cache audio components received from transmitting endpoints and to forward the cached components to a specific receiver when said receiver needs to commence decoding of the audio signal from a specific transmitting endpoint at a specific quality level, and if the quality components were not present in in previous audio frames received by the receiver.

19. The conferencing system of claim 12, wherein the communication channels are packet-based, and wherein the SACS is configured to aggregate a number of audio packets intended for a specific receiver into one combined packet to save on packet header overhead, and to then forward the one combined packet to the receiver to effect rate control.

20. The conferencing system of claim 13, wherein the SACS is configured to forward only base layer information received from a transmitting endpoint when a signal strength indicator exceeds a first threshold, and to forward enhancement layer information received from the transmitting endpoint only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

21. The conferencing system of claim 13, wherein the endpoint is configured to transmit only base layer information when a signal strength indicator exceeds a first threshold, and to transmit enhancement layer information only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

22. The conferencing system of claim 12 wherein the SACS is configured to forward the audio signal component received from the transmitting endpoint to the receiving endpoint using a dynamic switching matrix.

23. The conferencing system of claim 12 wherein the SACS is configured to forward an audio signal received from a transmitting endpoint to a receiving endpoint without decoding and/or re-coding the audio signal.

24. The conferencing system of claim 12 wherein the SACS is configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to at least one endpoint linked through the SACS by selectively multiplexing and forwarding audio signal components to the linked endpoints.

25. The conferencing system of claim 24 wherein the SACS is configured to selectively multiplex and forward audio signal components having different signal characteristics to a receiving endpoint, the different signal characteristics including at least one of different sampling rate, bit rate, quality, and number of audio signal channels.

26. The conferencing system of claim 24 wherein the SACS is configured to forward enhancement signal layers to receiving endpoints in priority according to a conferencing system priority policy, which assigns priority to receiving endpoints.

27. The conferencing system of claim 12 wherein the SACS is further configured to process video signal in addition to processing audio signals.

28. The conferencing system of claim 12, further comprising a plurality of linked SACS.

29. The conferencing system of claim 28 wherein the linked SACS are disposed over heterogeneous communication network domains.

30. The conferencing system of claim 12 wherein the SACS is further configured to provide at least one of session border control, media proxy, firewall, and network address translation functions.

31. The conferencing system of claim 12, wherein the multiple quality components of the coded audio signals are each comprised of independently decodable encodings of the said audio signals.

32. The conferencing system of claim 12, wherein the SACS is configured to forward only one quality component received from a transmitting endpoint when a signal strength indicator exceeds a first threshold, and to forward additional quality components received from the transmitting endpoint only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

33. The conferencing system of claim 12, wherein the endpoint is configured to transmit only one quality component when a signal strength indicator exceeds a first threshold, and to transmit additional quality components only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

34. The conferencing system of claim 12, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators that are computed at the SACS.

35. The conferencing system of claim 12, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators that are computed at the transmitting endpoints.

36. A method for multi-endpoint video signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints with at least one communication channel linking an endpoint having a superior quality of service compared to other channels, the method comprising:

obtaining a scalable video coded video signal, wherein the video signal is coded in a layered format including at least one layer from a base layer and an enhancement layer;

selecting at least one layer of the coded video signal; and

forwarding information in the selected layer to the endpoint over the communication channel having superior quality of service.

37. The method of claim 36, wherein forwarding information in the selected layer to the endpoint over the communication channel having superior quality of service comprises using a dynamic switching matrix to switch the information to the endpoint.

38. The method of claim 36, wherein forwarding information in the selected layer to the endpoint over the communication channel having superior quality of service comprises forwarding the information to the endpoint without decoding and/or recoding the video signal.

39. The method of claim 36, further comprising selectively multiplexing and forwarding video signal layers to the conferencing endpoints, thereby providing at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the conferencing endpoints.

40. The method of claim 39, wherein selectively multiplexing and forwarding video signal layers to the conferencing endpoints comprises selectively multiplexing and forwarding video signal layers having different signal characteristics to a receiving endpoint, the different signal characteristics including at least one of different resolution, bit rate, quality, and frame rate characteristics.

41. The method of claim 39 further comprising responding to network bandwidth conditions by at least one of:

statistically multiplexing video signals from a plurality of transmitting endpoints; and

synchronizing the transmission of video signals from a plurality of transmitting endpoints to stagger larger-than-average video frames in the multiplexed video signal.

42. The method of claim 39, further comprising forwarding enhancement signal layers to receiving endpoints in priority according to a priority policy, which assigns priority to receiving endpoints.

43. The method of claim 36, further comprising forwarding audio signals in addition to forwarding video signals.

44. The method of claim 36, further comprising using at least one linked SVCS to mediate forwarding of information to the endpoints.

45. The method of claim 36, further comprising using a plurality of linked SVCS that are disposed over heterogeneous communication network domains to mediate forwarding of information to the endpoints.

46. The method of claim 36, further comprising using the SVCS to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

73. Computer readable media comprising a set of instructions to perform the steps recited in at least one of claims 36-70 and 72.

47. A method for multi-endpoint audio signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints (i.e., the transmitting and receiving endpoints), the method comprising:

obtaining audio signals that are coded in component bitstreams such that multiple qualities can be derived from a bitstream in the coded domain; and

selectively forwarding audio signal components received from transmitting endpoints over their respective linking communication channels to receiving endpoints over their respective linking communication channels.

48. The method of claim 47, wherein the audio signal is scalably coded into layered components comprising a base layer and at least one enhancement layer.

49. The method of claim 47, wherein the network has at least one communication channel linking an endpoint that has a superior quality of service compared to other network channels, the method further comprising selectively forwarding audio signal components to the endpoint over the at least one communication channel that has a superior quality of service.

50. The method of claim 47, wherein obtaining audio signals comprises obtaining audio signals that include signal strength indicators.

51. The method of claim 50, further comprising:

receiving audio signals including signal strength indicators from transmitting endpoints;

forwarding all quality components of the strongest received audio signal to all endpoints except the one it was transmitted from;

forwarding less than all quality components of a number of the less strong received audio signals to all endpoints except the one they were transmitted from; and

forwarding no quality components of the remaining less strong received audio signals to the endpoints.

52. The method of claim 47, further comprising using a SACS to forward the audio signals received from transmitting endpoints to the receiving endpoints, and mixing the audio signals at the receiving endpoints.

53. The method of claim 47, further comprising caching audio signal components received from transmitting endpoints and forwarding cached audio components to a specific receiving endpoint which is commencing decoding of the audio signal from a specific transmitting endpoint at a specific quality level, and which previously was not receiving the audio components of that quality level.

54. The method of claim 47, wherein the network communication channels linking the conferencing end points are packet-based, the method further comprising:

aggregating a number of audio packets intended for a specific receiving endpoint into one combined packet to save on packet header overhead; and

then, forwarding the one combined packet to the specific receiving endpoint to effect rate control.

55. The method of claim 48, further comprising,

from a transmitting endpoint, forwarding only base layer information when a signal strength indicator exceeds a first threshold;

and

from the transmitting endpoint, forwarding enhancement layer information from the transmitting endpoint only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

56. The method of claim 48, further comprising:

forwarding only base layer information received from a transmitting endpoint when a signal strength indicator exceeds a first threshold; and

forwarding enhancement layer information received from the transmitting endpoint only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

57. The method of claim 47, wherein selectively forwarding audio signal components received from transmitting endpoints over their respective linking communication channels to receiving endpoints over their respective linking communication channels comprises using a SACS having a dynamic switching matrix to mediate the receiving and forwarding of audio signal components.

58. The method of claim 47 wherein selectively forwarding an audio signal received from a transmitting endpoint to a receiving endpoint comprises forwarding without decoding and/or re-coding the audio signal.

59. The method of claim 47 wherein selectively forwarding an audio signal received from a transmitting endpoint to a receiving endpoint comprises selectively multiplexing and forwarding audio signal components, whereby the receiving endpoint is provided at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features.

60. The method of claim 59 wherein selectively forwarding an audio signal received from a transmitting endpoint to a receiving endpoint comprises selectively multiplexing and forwarding audio signal components having different signal characteristics to a receiving endpoint, the different signal characteristics including at least one of different sampling rate, bit rate, quality, and number of audio signal channels.

61. The method of claim 59 wherein selectively forwarding an audio signal received from a transmitting endpoint to a receiving endpoint comprises forwarding enhancement signal layers to receiving endpoints in priority according to a conferencing system priority policy, which assigns priority to receiving endpoints.

62. The method of claim 47, further comprising receiving and forwarding video signal in addition to processing audio signals.

63. The method of claim 47, further comprising using at least one SACS to mediate the forwarding of audio signal components.

64. The conferencing system of claim 63, further comprising using a plurality of linked SACS that are disposed over heterogeneous communication network domains to mediate forwarding of audio signal components to the endpoints.

65. The method of claim 47, further comprising:

using a plurality of linked SACS to mediate the forwarding of audio signal components; and

further using the linked SACS to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

66. The method of claim 47, wherein the multiple quality components of the coded audio signal are each comprised of independent encodings of the said audio signals.

67. The method of claim 47, further comprising,

from a transmitting endpoint, transmitting only one quality component when a signal strength indicator exceeds a first threshold;

and

from the transmitting endpoint, transmitting additional quality components from the transmitting endpoint only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable

68. The method of claim 47, further comprising:

forwarding only one quality component received from a transmitting endpoint when a signal strength indicator exceeds a first threshold; and

forwarding additional quality components received from the transmitting endpoint only when the signal strength indicator exceeds a second threshold, whereby clipping of speaker cut-in talk spurt can be minimized or made less noticeable.

69. The method of claim 47, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators that are computed at the SACS.

70. The method of claim 47, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators that are computed at the transmitting endpoints.

71. A multi-endpoint video signal conferencing system, wherein video signals are scalably coded into layers including a base layer and one or more enhancement layers, the conferencing system comprising:

a scalable video coding server (SVCS) linked to at least one receiving and at least one transmitting endpoint by corresponding communication channels,

wherein the SVCS is configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the at least one receiving endpoint by selectively multiplexing and forwarding to the at least one receiving endpoint video signal layers received from the at least one transmitting endpoint.

72. A method for multi-endpoint video signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints, the method comprising:

obtaining a scalable video coded video signal, wherein the video signal is coded in a layered format including at least one layer from a base layer and an enhancement layer;

selecting at least one layer of the coded video signal; and

selectively multiplexing and forwarding video signal layers to the conferencing endpoints over the linking communication channels, thereby providing at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the conferencing endpoints.