Resource-adaptive video encoder sharing in multipoint control unit

• US 9,210,381 B2
• Filed: 06/13/2014
• Issued: 12/08/2015
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Dialogic, Inc. (Enghouse Systems Limited)
• Original Assignee: Dialogic, Inc. (Enghouse Systems Limited)
• Inventors: Yang, Kyeong Ho, Tun, Myo, Kolakowski, Lawrence
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US14/304,144
• Publication Number:
  US 20140375756A1
• Time in Patent Office: 543 Days
• Field of Search: 348/14.09, 370/248, 370/260, 370/262, 370/263, 370/352, 375/240.03, 375/240.16, 375/240.25, 379/202.01, 709/201, 709/204, 709/224, 709/227, 725/92, 382/232
• US Class Current: 1/1
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

Systems and methods of video conferencing, in which existing video encoders associated with a multipoint control unit (MCU) can be shared among conference participant devices, depending upon the availability of processor and/or memory resources for the MCU. In making decisions regarding the sharing of video encoders, the systems and methods take into account various parameters associated with potential conference participant devices and the total cost of the processor/memory resources for the MCU, to determine whether a video encoder of the MCU can be shared with, or allocated to, the respective conference participant devices. By determining whether to share or allocate a video encoder of the MCU, depending upon the available processor/memory resources for the MCU, the MCU can be made to operate with increased stability. The systems and methods can be applied to any application that sends the same content to multiple receivers, including announcement and video streaming applications.

Claims

1. In a multimedia communications system, a method of handling packet errors associated with multimedia content received at a respective one of a plurality of multimedia participant devices from a multimedia communications server, the multimedia communications server including at least one video encoder, the plurality of multimedia participant devices being communicably coupleable to the multimedia communications server over at least one network, the method comprising:

receiving, at the multimedia communications server, at least one report packet from a respective multimedia participant device, the at least one report packet containing packet error feedback information, the plurality of multimedia participant devices including the respective multimedia participant device participating in a multimedia session and sharing operations of the same video encoder;

determining, at the multimedia communications server, whether the packet error feedback information includes one or more of a request for an intra-coded frame and an estimated available bandwidth of the respective multimedia participant device;

in the event the packet error feedback information includes the request for the intra-coded frame, determining a first cost of sending the intra-coded frame to the plurality of multimedia participant devices participating in the multimedia session and sharing the operations of the same video encoder;

in the event the packet error feedback information includes the estimated available bandwidth of the respective multimedia participant device, determining a second cost of adjusting an average bitrate employed in the multimedia session; and

based at least upon one or more of the first cost and the second cost, performing, by the multimedia communications server, one or more of sending the intra-coded frame to the plurality of multimedia participant devices, and adjusting the average bitrate employed in the multimedia session.

2. The method of claim 1 wherein the determining of the first cost of sending the intra-coded frame includes obtaining a cost function of a plurality of cost factors, the plurality of cost factors including a number of additional bits required to encode a current frame using intra-coding as opposed to using inter-coding, and a total number of the plurality of multimedia participant devices sharing the operations of the same video encoder.

3. The method of claim 2 wherein the determining of the first cost of sending the intra-coded frame includes weighting the first cost by a number of inter-coded frames scheduled to be sent to a receiving side of the multimedia session before sending a next scheduled intra-coded frame.

4. The method of claim 3 further comprising:

based at least upon the first cost, performing, by the multimedia communications server, one of (1) sending the intra-coded frame substantially immediately, (2) ignoring the request for the intra-coded frame, and (3) implementing an additional video encoder or designating another existing video encoder for use by the respective multimedia participant device.

5. The method of claim 4 wherein the at least one video encoder employs a predetermined quantization parameter for encoding the multimedia content, and wherein the selecting between (1) the sending of the intra-coded frame substantially immediately, and (2) the implementing of the additional video encoder or the designating of another existing video encoder for use by the respective multimedia participant device, is based at least upon an average value of the quantization parameter.

6. The method of claim 1 wherein the determining of the second cost of adjusting the average bitrate employed in the multimedia session includes obtaining one or more of (1) a current bitrate for the at least one video encoder, (2) the estimated available bitrate for the respective multimedia participant device, (3) a maximum available bitrate among available bitrates for the plurality of multimedia participant devices, and (4) a minimum available bitrate among the available bitrates for the plurality of multimedia participant devices.

7. The method of claim 6 further comprising:

setting, by the multimedia communications server, the current bitrate for the at least one video encoder to be equal to the estimated available bitrate for the respective multimedia participant device if (1) the estimated available bitrate calls for an increase in the current bitrate, and (2) the estimated available bitrate is lower than the minimum available bitrate.

8. The method of claim 6 further comprising:

setting, by the multimedia communications server, the current bitrate for the at least one video encoder to be equal to the minimum available bitrate among the available bitrates for the plurality of multimedia participant devices if (1) the estimated available bitrate calls for an increase in the current bitrate, and (2) the estimated available bitrate is lower than the maximum available bitrate multiplied by a first predetermined factor.

9. The method of claim 8 further comprising:

setting, by the multimedia communications server, the current bitrate for the at least one video encoder to be equal to the minimum available bitrate among the available bitrates for the plurality of multimedia participant devices if (1) the estimated available bitrate calls for an increase in the current bitrate, (2) the estimated available bitrate is higher than or equal to the maximum available bitrate multiplied by the first predetermined factor, and (3) an acceptable quality of experience for multimedia transmissions can be achieved with the current bitrate set to be equal to the minimum available bitrate.

10. The method of claim 9 further comprising:

implementing, by the multimedia communications server, an additional video encoder, or designating, by the multimedia communications server, another existing video encoder for use by the respective multimedia participant device if (1) the estimated available bitrate calls for an increase in the current bitrate, (2) the estimated available bitrate is higher than or equal to the maximum available bitrate multiplied by the first predetermined factor, and (3) the acceptable quality of experience for multimedia transmissions cannot be achieved with the current bitrate set to be equal to the minimum available bitrate.

11. The method of claim 6 further comprising:

setting, by the multimedia communications server, the current bitrate for the at least one video encoder to be equal to the estimated available bitrate for the respective multimedia participant device if (1) the estimated available bitrate calls for a decrease in the current bitrate, and (2) the estimated available bitrate is higher than the current bitrate multiplied by a second predetermined factor.

12. The method of claim 11 further comprising:

setting, by the multimedia communications server, the current bitrate for the at least one video encoder to be equal to the estimated available bitrate for the respective multimedia participant device if (1) the estimated available bitrate calls for a decrease in the current bitrate, (2) the estimated available bitrate is lower than or equal to the current bitrate multiplied by the second predetermined factor, and (3) an acceptable quality of experience for multimedia transmissions can be achieved with the current bitrate set to be equal to the estimated available bitrate.

13. A multimedia communications server for use in a multimedia communications system, the multimedia communications system including a plurality of multimedia participant devices communicably coupleable to the multimedia communications server over at least one network, the multimedia communications server comprising:

at least one video encoder; and

a controller including at least one processor operative to execute at least one program out of at least one memory;

to receive at least one report packet from a respective multimedia participant device, the at least one report packet containing packet error feedback information, the plurality of multimedia participant devices participating in a multimedia session and sharing operations of the same video encoder;

to determine whether the packet error feedback information includes one or more of a request for an intra-coded frame and an estimated available bandwidth of the respective multimedia participant device;

in the event the packet error feedback information includes the request for the intra-coded frame, to determine a first cost of sending the intra-coded frame to the plurality of multimedia participant devices participating in the multimedia session and sharing the operations of the same video encoder;

in the event the packet error feedback information includes the estimated available bandwidth of the respective multimedia participant device, to determine a second cost of adjusting an average bitrate employed in the multimedia session; and

based at least upon one or more of the first cost and the second cost, to perform one or more of sending the intra-coded frame to the plurality of multimedia participant devices and adjusting the average bitrate employed in the multimedia session.

14. The system of claim 13 wherein the at least one processor is further operative to execute the at least one program out of the at least one memory to determine the first cost of sending the intra-coded frame using a cost function of a plurality of cost factors, the plurality of cost factors including a number of additional bits required to encode a current frame using intra-coding as opposed to using inter-coding, and a total number of the plurality of multimedia participant devices sharing the operations of the at least one video encoder.