Key frame distribution in video conferencing

• US 8,917,309 B1
• Filed: 03/08/2012
• Issued: 12/23/2014
• Est. Priority Date: 03/08/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Westin, Patrik
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): Ndje, Theodore
• Application Number:
  US13/415,267
• Time in Patent Office: 1,020 Days
• Field of Search: 348/14.01, 348/14.09, 348/14.13
• US Class Current: 348/14.08
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A system, apparatus, and method for relaying video information that is part of a multi-party video communication session having multiple endpoints. The server can receive multiple video information streams from multiple endpoints and re-transmit some or all of the video information streams to the endpoints with one or more of the video information steams identified as having active status and being decoded at high resolution at the endpoints, while transmitting key frames from video information streams not having active status. Upon switching active speaker status to a new video information stream, the endpoints, already having a key frame from the switched-to video information, can switch to decoding the new video information stream having active status at high resolution without delay.

Claims

1. A method for relaying video information that is part of a multi-party video communication session having multiple endpoints, the method comprising:

receiving, at a server, first encoded video information that has originated at a first endpoint of the multi-party video communication session;

receiving, at the server, second encoded video information that has originated at a second endpoint of the multi-party video communication session, the second encoded video information including at least one intra-coded frame;

transmitting, to at least a third endpoint of the multi-party video communication session, the received first encoded video information while the first endpoint has an active status;

processing, at the server, the second encoded video information to extract therefrom the at least one intra-coded frame while the second endpoint does not have the active status; and

transmitting, to the third endpoint, the extracted at least one intra-coded frame while the first endpoint has the active status and the second endpoint does not have the active status.

2. The method of claim 1, further comprising:

transmitting, to at least the third endpoint, the second encoded video information while the first endpoint has an active status and the second endpoint does not have an active status.

3. The method of claim 2 wherein the first encoded video information is transmitted to the third endpoint in a manner that permits the third endpoint to render the first encoded video information at a first resolution.

4. The method of claim 3 wherein the second encoded video information is transmitted to the third endpoint in a manner that permits the third endpoint to render the second encoded video information at a second resolution that is lower than the first resolution.

5. The method of claim 1, further comprising:

identifying a window of excess bandwidth during transmitting of the first encoded video information; and

wherein transmitting the at least one intra-coded frame occurs at least in part in the identified window.

6. The method of claim 1, further comprising:

selecting the at least one intra-coded frame for transmission from a plurality of candidate intra-coded frame based on a determination of the likelihood that the second endpoint will be a next endpoint to transition to an active status.

7. The method of claim 1, further comprising:

identifying a transition of the second endpoint to an active status; and

transmitting, to at least the third endpoint of the multi-party video communication session, the second encoded video information after identifying that the second endpoint has an active status;

wherein the second encoded video information is configured to permit the third endpoint to render at least a portion of the second encoded video information using the at least one intra-coded frame transmitted while the second endpoint did not have an active status.

8. An apparatus for relaying video information that is part of a multi-party video communication session having multiple endpoints, the apparatus comprising:

a server including a memory and a processor configured to execute instructions stored in the memory to;

receive first encoded video information that has originated at a first endpoint of the multi-party video communication session;

receive second encoded video information that has originated at a second endpoint of the multi-party video communication session, the second encoded video information including at least one intra-coded frame;

transmit, to at least a third endpoint of the multi-party video communication session, the first encoded video information while the first endpoint has an active status;

extract at least one intra-coded frame from the second encoded video information; and

transmit the extracted at least one intra-coded frame to the third endpoint while the first endpoint has the active status and the second endpoint does not have an active status.

9. The apparatus of claim 8 wherein the first encoded video information is transmitted to the third endpoint in a manner that permits the third endpoint to render the first encoded video information at a first resolution.

10. The apparatus of claim 9 wherein the second encoded video information is transmitted to the third endpoint in a manner that permits the third endpoint to render the second encoded video information at a second resolution that is lower than the first resolution.

11. The apparatus of claim 8 wherein the processor is further configured to:

identify a transition of the second endpoint to the active status; and

transmit, to at least the third endpoint, the received second encoded video information in a manner that permits the third endpoint to render the second encoded video information at the first resolution;

wherein the received second encoded video information is configured to permit the third endpoint to render at least a portion of the second encoded video information using the at least one intra-coded frame transmitted while the second endpoint did not have the active status.

12. The apparatus of claim 8 wherein the processor is further configured to:

identify a window of excess bandwidth during transmitting of the received first encoded video information;

wherein transmitting the encoded portion of the at least one intra-coded frame occurs at least in part in the identified window.

13. The apparatus of claim 8 wherein the processor is further configured to:

select the at least one intra-coded frame for transmission from a plurality of candidate intra-coded frame based on a determination of the likelihood that the second endpoint will be the next endpoint to transition to an active status.

14. The apparatus of claim 8 wherein the processor is further configured to:

extract, from the second encoded video information, at least one intra-coded frame without decoding the second encoded video information.

15. A method for relaying video information that is part of a multi-party video communication session having multiple endpoints, the method comprising:

receiving, at a third endpoint of the multi-party video communication session, first encoded video information that has originated at a first endpoint of the multi-party video communication session;

receiving second encoded video information that has originated at a second endpoint of the multi-party video communication session at the third endpoint while the first endpoint has an active status and the second endpoint does not have an active status;

receiving at least one intra-coded frame for the second encoded video information at the third endpoint while the first endpoint has the active status and the second endpoint does not have the active status, the at least one intra-coded frame received separate from the second encoded video information; and

rendering the second encoded video information using the at least one intra-coded frame when the second endpoint transitions to have the active status.

16. The method of claim 15 wherein the first encoded video information is received by the third endpoint in a manner that permits the third endpoint to render the first encoded video information at a first resolution.

17. The method of claim 16 wherein the second encoded video information is received by the third endpoint in a manner that permits the third endpoint to render the second encoded video information at a second resolution that is lower than the first resolution when the second endpoint does not have the active status.

18. The method of claim 15, further comprising:

receiving, at the third endpoint, an indication of a transition of the second endpoint to the active status; and

receiving, at the third endpoint, the second encoded video information in a manner that permits the third endpoint to render the second encoded video information at the first resolution.

19. The method of claim 15 wherein receiving the at least one intra-coded frame occurs at least in part in a window of excess bandwidth between the endpoint and a server transmitting the at least intra-coded frame and the second encoded video information to the third endpoint.

20. The method of claim 15, further comprising:

rendering the second encoded video information at a first resolution while the second endpoint does not have the active status; and

wherein rendering the second encoded video information using the at least intra-coded frame when the second endpoint has the active status comprises rendering the second encoded video information at a second resolution higher than the first resolution.