System and method for a conference server architecture for low delay and distributed conferencing applications

• US 8,279,260 B2
• Filed: 08/11/2009
• Issued: 10/02/2012
• Est. Priority Date: 07/20/2005
• Status: Active Grant

• Current Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Original Assignee: Vidyo Incorporated (Enghouse Systems Limited)
• Inventors: Civanlar, Reha, Eleftheriadis, Alexandros, Shapiro, Ofer
• Primary Examiner(s): Woo, Stella
• Application Number:
  US12/539,501
• Publication Number:
  US 20090295905A1
• Time in Patent Office: 1,148 Days
• Field of Search: 348/14.08, 348/14.09, 348/14.01, 348/14.12
• US Class Current: 348/14.09
• CPC Class Codes: H04L 12/4604 LAN interconnection over a backbone network, e.g. Internet, Frame Relay
• CPC Class Codes: H04L 45/22 Alternate routing
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2441 relying on flow classification, e.g. using integrated services [IntServ]
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/724 at intermediate nodes, e.g. resource reservation protocol [RSVP]
• CPC Class Codes: H04L 47/801 Real time traffic
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/5683 Storage of data provided by user terminals, i.e. reverse caching
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/6405 Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/6408 Unicasting
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

Systems and methods for conducting a multi-endpoint video signal conference are provided. Conferencing endpoints are linked by pairs of a reliable and a less reliable communication channel. Conference video signals are scaleable coded in base layer and enhancement layers format. Video signal base layers, which correspond to a minimum picture quality, are communicated over reliable channels. The video signal enhancements layers may be communicated over the less reliable channels. A conference server mediates the switching of video layer information from transmitting endpoints to receiving endpoints without any intermediate coding or re-coding operations. The video conference can be integrated with an audio conference using either scalable coded audio signals or non-scaleable coded audio signals.

Claims

1. A multi-endpoint video signal conferencing system for communicating video signals to at least one receiving endpoint over at least one communication channel, wherein the video signals are scalably coded into layers including a base layer and one or more enhancement layers, the conferencing system comprising:

a scalable video coding server (SVCS) adapted to be linked to the at least one receiving endpoint by the at least one communication channel,

wherein the at least one communication channel offers improved quality of service; and

wherein the SVCS is configured to receive and selectively forward a video signal layer to the at least one receiving endpoint over the at least one communication channel.

2. The conferencing system of claim 1 wherein the SVCS is further configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the at least one receiving endpoint by selectively multiplexing and forwarding video signal layers thereto.

3. The conferencing system of claim 2 wherein the SVCS is further configured to selectively multiplex and forward video signal layers having two or more signal characteristics to the at least one receivingendpoint, the signal characteristics including different resolution, bit rate, quality, and frame rate characteristics.

4. The conferencing system of claim 1, wherein the SVCS is further configured to process audio signals in addition to processing video signals.

5. The conferencing system of claim 1, wherein the SVCS is further configured to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

6. The conferencing system of claim 1, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to proactively repeat transmission of at least the base layer packets.

7. The conferencing system of claim 1, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to transmit at least the base layer packets with forward error correction.

8. The conferencing system of claim 1, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to cache at least the base layer packets and to retransmit cached packets upon request.

9. A multi-endpoint video signal conferencing system for communicating video signals with at least one transmitting endpoint over at least one communication channel, wherein video signals are scalably coded into layers including a base layer and one or more enhancement layers, the conferencing system comprising:

a scalable video coding server (SVCS) adapted to be linked to the at least one transmitting endpoint by the at least one communication channel and to receive one or more video signal layers therefrom,

wherein the at least one communication channel offers improved quality of service; and

wherein the SVCS is configured to selectively forward the one or more video signal layers received from the transmitting endpoint over the at least one communication channel.

10. The conferencing system of claim 9 wherein the SVCS is further configured to provide at least one of continuous presence, personalized layout, rate matching, error localization, and random entry featuresfor the at least one transmitting endpoint by selectively multiplexing and forwarding video signal layers received therefrom.

11. The conferencing system of claim 9, wherein the SVCS is further configured to process audio signals in addition to processing video signals.

12. The conferencing system of claim 9, wherein the SVCS is further configured to provide at least one of session network border control, media proxy, firewall, and network address translation functions.

13. The conferencing system of claim 9, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to proactively repeat transmission of at least the base layer.

14. The conferencing system of claim 9, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to receive packets with forward error correction.

15. The conferencing system of claim 9, wherein the improved quality of service of the communication channel comprises the SVCS being furtherconfigured to receive retransmitted cached packets upon request.

16. A multi-endpoint audio signal conferencing system for communicating audio signals to at least one receiving endpoint over at least one communication channel, wherein the audio signals are coded in components such that multiple qualities can be derived from the bitstream in the coded domain, the conferencing system comprising:

a scaleable audio coding server (SACS) adapted to be linked to the at least one receiving endpoint in an audio conference by the at least one communication channel,

wherein the SACS is configured to receive and selectively forward an audio signal component of the audio signals to the at least one receiving endpoint over the at least one communication channel.

17. The conferencing system of claim 16, wherein the audio signal is scalably coded into layered components comprising a base layer and one or more enhancement layers.

18. The conferencing system of claim 16, wherein the at least one communication channel offers improved quality of service.

19. The conferencing system of claim 18, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to proactively repeat transmission of the at least one audio signal component.

20. The conferencing system of claim 18, wherein the improved quality of service of the communication comprises the SVCS being furtherconfigured to transmit the at least one audio signal component with forward error correction.

21. The conferencing system of claim 18, wherein the improved quality of service of the communication channel comprises the SVCS being further configured to cache data for the at least one audio signal component and to retransmit cached data upon request.

22. The conferencing system of claim 16, wherein the audio signals are associated with signal strength indicators.

23. The conferencing system of claim 22, wherein the SACS is further configured to:

forward all quality components of a strongest received audio signal to all participants except an originating participant

forward less than all quality components of a number of less strong received audio signals, if any, to all participants except an originating participant; and

forward no quality components of any remaining less strong received audio signals.

24. The conferencing system of claim 16, wherein the audio signals are forwarded to the at least one receiving endpoint by the SACS for mixing.

25. The conferencing system of claim 16, wherein the SACS is further configured to cache received audio components and to forward the cached components to a first receiver when the receiver needs to commence decoding of the audio signal at a specific quality level if such quality level was not previously received.

26. The conferencing system of claim 16, wherein the at least one communication channel comprises a packet-based channel, and wherein the SACS is further configured to aggregate a number of audio packets intended for a receiver into a combined packet, and to forward the combined packet to the receiver.

27. The conferencing system of claim 17, wherein the audio signals are associated with one or more signal strength indicators, and wherein the SACS is further configured to forward only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to forward base layer and enhancement layer information when one of the one or more signal strength indicators exceeds a second threshold.

28. The conferencing system of claim 16 wherein the SACS is further configured to selectively multiplex and forward audio signal components having two or more signal characteristics to the at least one receiving endpoint, the characteristics including different sampling rate, bit rate, quality, and number of audio signal channels.

29. The conferencing system of claim 16 wherein the SACS is further configured to forward enhancement signal layers to receiving endpoints in priority according to a conferencing system priority policy, which assigns priority to receiving endpoints.

30. The conferencing system of claim 16 wherein the SACS is further configured to process video signals.

31. The conferencing system of claim 16 wherein the SACS is further configured to provide at least one of a session border control, media proxy, firewall, and network address translation functions.

32. The conferencing system of claim 16, wherein the multiple quality components of the coded audio signals comprise independently decodable encodings of said audio signals.

33. The conferencing system of claim 16, wherein the audio signals are associated with one or more signal strength indicators, and wherein the SACS is further configured to forward one quality component when one of the one or more signal strength indicators exceeds a first threshold, and to forward additional quality components when one of the one or more signal strength indicators exceeds a second threshold.

34. The conferencing system of claim 16, wherein the audio signals are associated with one or more signal strength indicators that are computed at the SACS.

35. A multi-endpoint audio signal conferencing system for communicating audio signals with at least one transmitting endpoint over at least one communication channel, wherein the audio signals are coded in components such that multiple qualities can be derived from the bitstream in the coded domain, the conferencing system comprising:

a scaleable audio coding server (SACS) adapted to be linked to at least one transmitting endpoint in an audio conference by the at least one communication channel,

wherein the SACS is configured to receive and selectively forward an audio signal component of the audio signals received from the at least one transmitting endpoint overthe at least one communication channel.

36. The conferencing system of claim 35, wherein the audio signal is scalably coded into layered components comprising a base layer and one or more enhancement layers.

37. The conferencing system of claim 35, wherein at the least one communication channel offers improved quality of service.

38. The conferencing system of claim 37, wherein the improved quality of service of the communication channel comprises the SACS being further configured to receive a proactively repeated transmission of at least one audio signal component from the at least one transmitting endpoint.

39. The conferencing system of claim 37, wherein the improved quality of service of the communication channel comprises the SACS being further configured to receive transmit audio signal components with forward error correction.

40. The conferencing system of claim 37, wherein the improved quality of service of the communication comprises the SACS being further configured to receive retransmited cached data upon request.

41. The conferencing system of claim 35, wherein the audio signals are associated with one or more signal strength indicators.

42. The conferencing system of claim 41, wherein the SACS is further configured to:

forward all quality components of a strongest received audio signal to all participants except an originating participant;

forward less than all quality components of a number of less strong received audio signals, if any, to all participants except the originating participant; and

forward no quality components of any remaining less strong received audio signals.

43. The conferencing system of claim 35, wherein the at least one communication channel comprises a packet-based channel and wherein the SACS is further configured to aggregate a number of audio packets into one combined packet, and to forward the combined packet.

44. The conferencing system of claim 36, wherein the received audio signals are associated with one or more signal strength indicators, and wherein the SACS is further configured to forward only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to forward base layer and enhancement layer information when one of the one or more signal strength indicators exceeds a second threshold.

45. The conferencing system of claim 36, wherein the audio signals are associated with one or more signal strength indicators, and wherein the one or more transmitting endpoints are configured to transmit only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to transmit base layer and enhancement layer information only when one of the one or more signal strength indicators exceeds a second threshold.

46. The conferencing system of claim 35 wherein the SACS is configured to selectively multiplex and forward audio signal components having two or more signal characteristics, the characteristics including at least one of different sampling rate, bit rate, quality, and number of audio signal channels.

47. The conferencing system of claim 35 wherein the SACS is further configured to process video signal in addition to processing audio signals.

48. The conferencing system of claim 35 wherein the SACS is further configured to provide at least one of a session border control, media proxy, firewall, and network address translation functions.

49. The conferencing system of claim 35, wherein the multiple quality components of the coded audio signals each comprise independently decodable encodings of the said audio signals.

50. The conferencing system of claim 35, wherein the audio signals received from transmitting endpoints are associated with signal strength indicators, and wherein the SACS is further configured to forward only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to forward base layer and enhancement layer information when one of the one or more signal strength indicators exceeds a second threshold.

51. The conferencing system of claim 35, wherein the audio signals are associated with signal one or more strength indicators, and wherein the at least one transmitting endpoint is configured to transmit only base layer information when one of the one or more signal strength indicators exceeds a first threshold, and to transmit base layer and enhancement layer information only when one of the one or more signal strength indicators exceeds a second threshold.

52. The conferencing system of claim 35, wherein the audio signals received from the at least one transmitting endpoint is associated with one or more signal strength indicators that are computed at the SACS.

53. The conferencing system of claim 35, wherein the audio signals received from the at least one transmitting endpoint is are associated with signal strength indicators that are computed at the transmitting endpoints.

54. An apparatus for multi-endpoint video signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints with at least one communication channel linking an endpoint having a superior quality of service compared to other channels, said apparatus comprising:

one or more computer-readable storage media; and

software embodied in the one or more computer-readable storage media that is operable when executed to;

obtain a scalable video coded video signal, wherein the video signal is coded in a layered format including a base layer and one or more enhancement layers

select at least one layer of the coded video signal; and

forward information in the selected layer to the endpoint over the communication channel having superior quality of service.

55. An apparatus for multi-endpoint audio signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints (i.e., the transmitting and receiving endpoints), the apparatus comprising:

one or more computer-readable storage media; and

software embodied in the one or more computer-readable storage media that is operable when executed to;

obtain audio signals that are coded in component bitstreams such that multiple qualities can be derived from a bitstream in the coded domain; and

selectively forward audio signal components received from transmitting endpoints over their respective linking communication channels to receiving endpoints over their respective linking communication channels.

56. A apparatus for multi-endpoint video signal conferencing over an electronic communications network, the network having communication channels linking the conferencing endpoints, the apparatus comprising:

one or more computer-readable storage media; and

software embodied in the one or more computer-readable storage media that is operable when executed to;

obtain a scalable video coded video signal, wherein the video signal is coded in a layered format including at least a base layer and one or more enhancement layers;

select at least one layer of the coded video signal; and

selectively multiplex and forward video signal layers to the conferencing endpoints over the linking communication channels, thereby providing at least one of continuous presence, personalized layout, rate matching, error localization, and random entry features to the conferencing endpoints.