Video multicast engine

• US 9,386,273 B1
• Filed: 07/25/2014
• Issued: 07/05/2016
• Est. Priority Date: 06/27/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ellner, Lars Henrik
• Primary Examiner(s): Smith, Creighton
• Application Number:
  US14/341,051
• Time in Patent Office: 711 Days
• Field of Search: 348/14.1, 370/260
• US Class Current: 1/1
• CPC Class Codes: H04L 12/1818 Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04M 3/563 User guidance or feature selection
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Introduction of one or more multicast engines to a video conferencing system permits an improvement of overall network bandwidth used for a video conference. Video streams are received at a multicast engine from each of at least two participants in the video conference. The multicast engine transmits only the first of the two video streams based on a determination that only the first video stream is a candidate for a conference video stream to be transmitted to at least some of a plurality of conference participants from the server station. The conference video stream is received at the multicast engine from the server station, and the stream is transmitted from the multicast engine to each of the participants in the video conference that are connected to the multicast engine.

Claims

1. An apparatus, comprising:

a multicast engine, including;

a first memory; and

a first processor configured to execute instructions stored in the first memory to;

receive a plurality of video streams including at least a first video stream and a second video stream respectively supplied from each of at least two participants in a video conference that are connected to an input of the multicast engine;

transmit the first video stream to a server station while not transmitting the second video stream to the server station based on a determination that the first video stream and not the second video stream is a candidate for a conference video stream to be transmitted to at least some of a plurality of conference participants from the server station;

receive at least one video stream forming the conference video stream from the server station; and

transmit the conference video stream to each of the at least two participants in the video conference that are connected to the multicast engine.

2. The apparatus of claim 1, further comprising:

the first processor configured to execute instructions stored in the first memory to determine which of the at least two participants connected to the input of the multicast engine is a candidate for the conference video stream.

3. The apparatus of claim 1, further comprising:

the first processor configured to execute instructions stored in the first memory to determine which of the at least two participants connected to the input of the multicast engine is a candidate for the conference video stream by performing a mixing algorithm that selects the first video stream as the candidate for the conference video stream to be transmitted to the at least some of the plurality of conference participants from the server station.

4. The apparatus of claim 3, further comprising:

the mixing algorithm being a same mixing algorithm used by the server station to determine which video streams of the plurality of conference participants to include in the conference video stream.

5. The apparatus of claim 3, further comprising:

the server station being a relay station; and

the mixing algorithm being a same mixing algorithm used by the server station to determine which video streams of the plurality of conference participants to include in the conference video stream.

6. The apparatus of claim 1, further comprising:

the first processor configured to execute instructions stored in the first memory to mix the conference video stream before transmitting the conference video stream.

7. The apparatus of claim 1, further comprising:

the first processor configured to execute instructions stored in the first memory to

receive a signal from the server station indicating that the first video stream is the candidate for the conference video stream.

8. The apparatus of claim 1, further comprising:

a distance between each of the at least two participants in the video conference and the multicast engine being less than a distance between the multicast engine and the server station.

9. The apparatus of claim 1, further comprising:

the multicast engine being one of a plurality of multicast engines, each of the plurality of multicast engines connected to the server station and to different ones of the plurality of participants to form respective clusters; and

all but at least one of the plurality of multicast engines is less distant from its connected participants than it is from the server station.

10. The apparatus of claim 1, further comprising:

the server station including a second memory and a second processor configured to execute instructions stored in the second memory to;

perform a mixing algorithm that determines which video streams of the plurality of participants to include in the conference video stream; and

transmit a signal to the multicast engine indicating those video streams supplied from each of the at least two participants to include in the conference video stream.

11. The apparatus of claim 1, further comprising:

the first processor configured to execute instructions stored in the first memory to;

transmit the first video stream and the second video stream, upon the multicast engine establishing a connection to the server station, to the server station; and

end transmission of the second video stream to the server station when a result of the determination indicates that the second video stream is not a candidate for the video conference stream.

12. The apparatus of claim 1, further comprising:

the first processor configured to execute instructions stored in the first memory to receive the at least one video stream forming the conference video stream by receiving the conference video stream;

an encoding format of the conference video stream received at the multicast engine corresponds to that of a highest quality video stream of the at least two participants connected to the multicast engine; and

the first processor configured to execute instructions stored in the first memory to transcode the conference video stream to an encoding format required by a respective connected client before transmitting the conference video stream to each of the at least two participants in the video conference that are connected to the multicast engine.

13. A method, comprising:

receiving, at a first processor of a multicast engine, at least a first video stream and a second video stream respectively supplied from each of at least two participants in a video conference that are connected to an input of the multicast engine;

of the first video stream and the second video stream, transmitting only the first video stream to a server station from the multicast engine based on a determination that only the first video stream is a candidate for a conference video stream to be transmitted to at least some of a plurality of conference participants from the server station, the server station including a second memory and a second processor configured to execute instructions stored in the second memory;

receiving at least one video stream forming the conference video stream from the server station; and

transmitting the conference video stream to each of the at least two participants in the video conference that are connected to the multicast engine.

14. The method of claim 13, further comprising:

performing a mixing algorithm, using the second processor of the server station, that determines which video streams of the plurality of conference participants to include in the conference video stream for a plurality of mix cycles; and

transmitting a signal to the multicast engine that the first video stream is to be included in the conference video stream in a new cycle based on the mixing algorithm when the first video stream was not mixed to form the conference video stream in a previous cycle.

15. The method of claim 14, further comprising:

an input to the mixing algorithm being a voice activity detection signal.

16. The method of claim 14, further comprising:

the signal being a NACK signal.

17. The method of claim 16, further comprising:

transmitting, to the multicast engine from the server station, identifiers of the video streams of those of the plurality of conference participants to be included in the conference video stream in a first cycle after the new cycle based on the mixing algorithm; and

ending transmission, by the multicast engine, of any video stream of the at least two participants that is not included in the identifiers.

18. The method of claim 17, further comprising:

the identifiers belonging to a contributing source list; and

a connected participant supplying the first video stream being in the contributing source list when a synchronization source identifier of the connected participant is included in the contributing source list.

19. The method of claim 13, further comprising:

estimating an input into a mixing algorithm that determines which video streams of the at least two participants connected to the multicast engine are candidates for the conference video stream.

20. The method of claim 19, further comprising:

estimating the input by biasing the mixing algorithm to select at least one connected participant of the at least two participants.