Multimedia conference broadcast system

• US 9,955,119 B2
• Filed: 08/08/2016
• Issued: 04/24/2018
• Est. Priority Date: 01/27/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Leske, Matthew J., Groves, Loren, Schmelcher, Tristan, Milligan, Timothy S.
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US15/231,678
• Publication Number:
  US 20160353061A1
• Time in Patent Office: 624 Days
• Field of Search: 348 1401, 348 1403, 348 1407, 348 1408, 348 1412
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831

Abstract

A multimedia conference broadcast system may include one or more processors and a memory. The one or more processors may receive video streams and audio streams from participant devices. The one or more processors may transmit, to each of the participant devices, the video streams and the audio streams corresponding to the other devices. The one or more processors may receive a request to broadcast the video streams and the audio streams from one of the participant devices. The one or more processors may generate a composite multimedia stream based on the video streams and the audio streams received from the participant devices. The one or more processors may broadcast the composite multimedia stream to viewer devices that are separate from the participant devices.

Claims

1. A computer-implemented method, comprising:

initiating a connection to a multimedia conference, the multimedia conference being hosted with a plurality of participant devices, the plurality of participant devices including one or more communicating participant devices and one or more non-communicating participant devices;

receiving one or more data streams from at least one participant device of the plurality of participant devices through the initiated connection;

integrating the received data streams into a composite stream;

encoding the composite stream into a broadcast format based on the received data streams, the broadcast format including one or more first data streams in a first broadcast quality for the one or more communicating participant devices and one or more second data streams in a second broadcast quality different than the first broadcast quality for the one or more non-communicating participant devices; and

providing the encoded composite stream for broadcast to a plurality of endpoint devices, the plurality of endpoint devices being non-participants of the multimedia conference, the composite stream being broadcast independent of the initiated connection to the multimedia conference.

2. The computer-implemented method of claim 1, wherein receiving the one or more data streams comprises:

receiving one or more low quality data streams from the one or more non-communicating participant devices; and

receiving one or more high quality data streams from the one or more communicating participant devices.

3. The computer-implemented method of claim 2, wherein the one or more first data streams comprise the one or more high quality data streams, and wherein the one or more second data streams comprise the one or more low quality data streams.

4. The computer-implemented method of claim 1, wherein encoding the composite stream comprises:

encoding different versions of the composite stream based on bandwidth capabilities of the plurality of endpoint devices, wherein each of the different versions of the composite stream corresponds to a different broadcast quality.

5. The computer-implemented method of claim 4, wherein each of the different versions of the composite stream has a different layout based on display capabilities of the plurality of endpoint devices, wherein at least one of the different versions of the composite stream includes a first layout that includes a data stream corresponding to one of the plurality of participant devices, and wherein at least one of the different versions of the composite stream includes a second layout that includes data steams corresponding to each of the plurality of participant devices.

6. The computer-implemented method of claim 4, wherein each of the different versions of the composite stream includes a primary data stream and a secondary data stream, wherein the primary data stream corresponds to a different one of the plurality of participant devices and the secondary data stream corresponds to remaining ones of the plurality of participant devices for each of the different versions of the composite stream.

7. The computer-implemented method of claim 6, further comprising:

providing a control user interface to the plurality of endpoint devices; and

receiving a user selection via the control user interface from at least one of the plurality of endpoint devices, the user selection indicating which version among the different versions of the composite stream is to be provided for display at the at least one of the plurality of endpoint devices.

8. The computer-implemented method of claim 1, wherein the composite stream is formed independent of a data stream from the plurality of endpoint devices.

9. The computer-implemented method of claim 1, wherein integrating the received data streams comprises:

decoding the received data streams into a raw format; and

combining the decoded data streams in the raw format into the composite stream.

10. A system, comprising:

one or more processors; and

a memory including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

initiating a connection to a multimedia conference with a plurality of first endpoint devices, the plurality of first endpoint devices including one or more communicating participant devices and one or more non-communicating participant devices;

receiving one or more data streams from at least one endpoint device of the plurality of first endpoint devices through the initiated connection;

generating a composite stream based on the received data streams;

encoding the composite stream into a broadcast format based on the received data streams, the broadcast format including one ore more first data streams in a first broadcast quality for the one or more communicating participant devices and one or more second data streams in a second broadcast quality different than the first broadcast quality for the one or more non-communicating participant devices; and

providing the encoded composite stream for broadcast to a plurality of second endpoint devices, the plurality of second endpoint devices being non-participants of the multimedia conference, the composite stream being broadcast independent of the initiated connection to the multimedia conference.

11. The system of claim 10, wherein the one or more first data streams comprise one or more high quality data streams, and wherein the one or more second data streams comprise one or more low quality data streams.

12. The system of claim 10, wherein the instructions further cause the one or more processors to:

encode different versions of the composite stream based on bandwidth capabilities of the plurality of second endpoint devices, wherein each of the different versions of the composite stream corresponds to a different broadcast quality.

13. The system of claim 12, wherein each of the different versions of the composite stream has a different layout based on display capabilities of the plurality of second endpoint devices, wherein at least one of the different versions of the composite stream includes a first layout that includes a data stream corresponding to one of the plurality of second endpoint devices, and wherein at least one of the different versions of the composite stream includes a second layout that includes data steams corresponding to each of the plurality of second endpoint devices.

14. The system of claim 12;

wherein each of the different versions of the composite stream includes a primary data stream and a secondary data stream, wherein the primary data stream corresponds to a different one of the plurality of first endpoint devices and the secondary data stream corresponds to a remaining ones of the plurality of first endpoint devices for each of the different versions of the composite stream.

15. The system of claim 14, wherein the instructions further cause the one or more processors to:

provide a control user interface to the plurality of second endpoint devices; and

receive a user selection via the control user interface from at least one of the plurality of second endpoint devices, the user selection indicating which version among the different versions of the composite stream is to be provided for display at the at least one of the plurality of second endpoint devices.

16. The system of claim 10, wherein the composite stream is formed independent of a data stream from the plurality of second endpoint devices.

17. A non-transitory machine-readable medium embodying instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving one or more data streams from at least one device of a plurality of first endpoint devices through a connection to a multimedia conference, the plurality of first endpoint devices including one or more communicating participant devices and one or more non-communicating participant devices;

combining the received data streams into a composite stream;

encoding the composite stream into a broadcast format based on the received data streams, the broadcast format including one or more first data streams in a first broadcast quality for the one or more communicating participant devices and one or more second data streams in a second broadcast quality different than the first broadcast quality for the one or more non-communicating participant devices; and

providing the encoded composite stream for broadcast to a plurality of second endpoint devices, the plurality of second endpoint devices being non-participants of the multimedia conference, the encoded composite stream being broadcast independent of the connection to the multimedia conference.