Methods and systems for interfacing heterogeneous endpoints and web-based media sources in a video conference

• US 9,300,705 B2
• Filed: 03/17/2014
• Issued: 03/29/2016
• Est. Priority Date: 05/11/2011
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Blue Jeans Network, Inc. (Verizon Communications Inc.)
• Inventors: Periyannan, Alagu, Mohire, Prasad R., Grupenhoff, Michael, Kulkarni, Swaroop, Sridharan, Akshay Kumar, Prakash, Nithin A., Weber, Emmanuel, Raghavan, Anand, Wong, Kelly
• Primary Examiner(s): Patel, Hemant
• Application Number:
  US14/217,275
• Publication Number:
  US 20140313278A1
• Time in Patent Office: 743 Days
• Field of Search: 348 1401- 1416, 370259-271, 370351-357, 379/202.01
• US Class Current: 1/1
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

Described herein are methods and systems for interfacing heterogeneous endpoints and web-based media sources in a video conference. Interfacing heterogeneous endpoints can be challenging due to audio/video (A/V) streams being encoded in a format that is proprietary to a media provider of the A/V stream (e.g., format that is proprietary to Skype). To address such a challenge, a client application from the media provider may be installed at a virtual machine, and may be used to decode A/V streams that are encoded in a format proprietary to the media provider. Further, to display content from a web-based media source (e.g., YouTube) in a video conference, a browser may be installed at a virtual machine, and may be used to virtually render a media signal from the web-based media source, such virtually rendered media signal subsequently being captured and presented in a video conference.

Claims

1. A method for processing audio/video (A/V) streams of a video conference at one or more servers, the method comprising:

for each of a plurality of user endpoints of the video conference;

receiving an A/V stream, the A/V stream encoded in a format that is proprietary to a media provider of the A/V stream; and

transcoding the A/V stream by (i) decoding the A/V stream using a client application from the media provider, the client application executing at a virtual machine instantiated on the one or more servers, and (ii) encoding the decoded A/V stream into a format that is compatible with a media mixer,

wherein each of the virtual machines instantiated on the one or more servers only executes a single client application, and

wherein load prediction is employed to minimize a number of virtual machines that are instantiated on the one or more servers at any time instance, the load prediction comprising;

predicting a first number of client applications that will be needed at a first time instance to decode the A/V streams received at the one or more servers;

before the first time instance, (i) deploying the first number of virtual machines on the one or more servers, and (ii) installing the first number of client applications, one on each of the deployed virtual machines;

at the first time instance, determining whether an additional number of client applications are needed to decode the A/V streams, the additional number of client applications in excess of the first number of client applications; and

if an additional number of client applications are needed, (i) deploying a corresponding number of additional virtual machines on the one or more servers, and (ii) installing the additional number of client applications, one on each of the additional virtual machines.

2. The method of claim 1, further comprising:

composing, at the media mixer, a plurality of the transcoded A/V streams into a composite A/V stream; and

encoding the composite A/V stream into a format that is compatible with one of the user endpoints.

3. The method of claim 1, wherein decoding, at the virtual machine, the A/V stream using the client from the media provider comprises:

virtually playing the A/V stream using the client from the media provider;

capturing the audio stream of the played A/V stream; and

capturing the video stream of the played A/V stream.

4. The method of claim 1, wherein for each of the plurality of user endpoints of the video conference, the virtual machine receives the A/V stream from a client application executing at a user device, wherein the client application executing at the virtual machine is identical to the client application executing at the user device, wherein the user device comprises one or more of a desktop computer, a tablet computing device and a smartphone.

5. The method of claim 1, wherein for each of the plurality of user endpoints of the video conference, the virtual machine receives the A/V stream from a client application executing at a user device, wherein the client application executing at the virtual machine is compatible with the client application executing at the user device, wherein the user device comprises one or more of a desktop computer, a tablet computing device and a smartphone.

6. The method of claim 1, wherein for each of the plurality of user endpoints of the video conference, the media provider comprises one of Skype, WebEx, FaceTime and Blue Jeans Network.

7. The method of claim 1, wherein for each of the plurality of user endpoints of the video conference, the client application from the media provider comprises one of a Skype client application, a WebEx client application, a FaceTime client application, and a Blue Jeans client application.

8. The method of claim 1, wherein predicting the first number of client applications that will be needed at the first time instance comprises applying a filter to a time series of a historical number of clients that were installed on the virtual machines of the one or more servers.

9. A method for processing audio/video (A/V) streams of a video conference at one or more servers, the method comprising:

for each of a plurality of user endpoints of the video conference;

receiving an A/V stream;

determining whether the A/V stream is in a format that is compatible with a media mixer;

if the A/V stream is in a format compatible with the media mixer, not transcoding the A/V stream;

otherwise, if the A/V stream is not in a format compatible with the media mixer, transcoding the A/V stream, the transcoding comprising;

determining a media provider associated with the A/V stream;

determining whether an application software development kit (SDK) from the media provider is available at the one or more servers;

if the application SDK from the media provider is available, decoding the A/V stream using the application SDK from the media provider, otherwise, if the application SDK from the media provider is not available, decoding the A/V stream using a client application from the media provider, the client application executing at a virtual machine instantiated on the one or more servers; and

encoding the decoded A/V stream into a format compatible with the media mixer,

wherein each of the virtual machines instantiated on the one or more servers only executes a single client application;

composing, at the media mixer, a plurality of ones of the transcoded A/V streams and the not transcoded A/V streams into one composite A/V stream; and

encoding the composite A/V stream into a format that is compatible with one of the user endpoints,

wherein load prediction is employed to minimize a number of virtual machines that are instantiated on the one or more servers at any time instance, the load prediction comprising;

predicting a first number of client applications that will be needed at a first time instance to decode the A/V streams received at the one or more servers;

before the first time instance, (i) deploying the first number of virtual machines on the one or more servers, and (ii) installing the first number of client applications, one on each of the deployed virtual machines;

at the first time instance, determining whether an additional number of client applications are needed to decode the A/V streams, the additional number of client applications in excess of the first number of client applications; and

if an additional number of client applications are needed, (i) deploying a corresponding number of additional virtual machines on the one or more servers, and (ii) installing the additional number of client applications, one on each of the additional virtual machines.

10. A method for incorporating one or more web-based media sources into a video conference, the method comprising:

for each of the one or more web-based media sources;

receiving a media signal from the web-based media source;

virtually rendering the media signal using a browser executing on a virtual machine instantiated on one or more servers;

capturing one or more of an audio stream, a video stream and an image associated with the virtually rendered media signal; and

encoding one or more of the captured audio stream, the captured video stream and the captured image into a format compatible with a media mixer,

wherein each of the virtual machines instantiated on the one or more servers only executes a single browser;

composing, at the media mixer, one or more of the encoded audio streams, the encoded video stream and the encoded images with one or more audio and video (A/V) streams associated with user endpoints of the video conference to form a composite A/V stream; and

encoding the composite A/V stream into a format that is compatible with one of the user endpoints,

wherein load prediction is employed to minimize a number of virtual machines that are instantiated on the one or more servers at any time instance, the load prediction comprising;

predicting a first number of browsers that will be needed at a first time instance to virtually render the media signals received at the one or more servers;

before the first time instance, (i) deploying the first number of virtual machines on the one or more servers, and (ii) installing the first number of browsers, one on each of the deployed virtual machines;

at the first time instance, determining whether an additional number of browsers are needed to decode the A/V streams, the additional number of browsers in excess of the first number of browsers; and

if an additional number of browsers is needed, (i) deploying a corresponding number of additional virtual machines on the one or more servers, and (ii) installing the additional number of browsers, one on each of the additional virtual machines.