Single-stream transmission method for multi-user video conferencing

• US 9,609,275 B2
• Filed: 07/08/2015
• Issued: 03/28/2017
• Est. Priority Date: 07/08/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kuusela, Aki, Tikkala, Aki, Rautio, Ville-Mikko
• Primary Examiner(s): Nguyen, Joseph J
• Assistant Examiner(s): NGUYEN, PHUNG HOANG JOSEPH
• Application Number:
  US14/794,251
• Publication Number:
  US 20170013233A1
• Time in Patent Office: 629 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 7/0117 involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method includes transmitting, from an encoding client to a first remote computing system, an encoded video stream that includes a plurality of first-resolution frames and a plurality of second-resolution frames. The method also includes transmitting, from the encoding client to the first remote computing system in the encoded video stream, a first-resolution non-displayed anchor frame. The method also includes receiving, at the encoding client from the first remote computing system, a request to change resolution. In response to receiving the request to change resolution, the method includes transmitting, from the encoding client to the first remote computing system in the encoded video stream, a set of frames that are encoded relative to the first-resolution non-displayed anchor frame.

Claims

1. A method, comprising:

encoding, by an encoding client, a plurality of first-resolution frames and a plurality of second-resolution frames, wherein each of the first-resolution frames and each of the second-resolution frames is encoded with reference to a respective same resolution frame from a previous time period;

transmitting, from the encoding client to a first remote computing system, an encoded video stream that includes the plurality of first-resolution frames and the plurality of second-resolution frames;

transmitting, from the encoding client to the first remote computing system in the encoded video stream, a first-resolution non-displayed anchor frame, wherein the first-resolution non-displayed anchor frame is transmitted periodically during transmission of the encoded video stream;

receiving, at the encoding client from the first remote computing system, a request to change resolution;

in response to receiving the request to change resolution, performing a coding change in which the encoding client encodes a set of frames that are encoded with reference to the non-displayed anchor frame; and

transmitting, from the encoding client to the first remote computing system in the encoded video stream, the set of frames that are encoded relative to the first-resolution non-displayed anchor frame.

2. The method of claim 1, further comprising:

transmitting, from the encoding client to the first remote computing system, a first-resolution intra frame during an initial time period, wherein at least one frame from the plurality of second-resolution frames that is transmitted during the initial time period is encoded relative to the first-resolution intra frame.

3. The method of claim 1, wherein the plurality of first-resolution frames correspond temporally to the plurality of second-resolution frames.

4. The method of claim 1, wherein all of the frames from the set of frames correspond to a single time period.

5. The method of claim 1, wherein the first remote computing system includes a video conference server operable to forward portions of the encoded video stream to each of a plurality of decoding clients.

6. The method of claim 5, wherein the request to change resolution originates at a first decoding client from the plurality of decoding clients and is relayed to the encoding client by the video conference server.

7. The method of claim 5, wherein the video conference server is operable to relay one or more frames from the set of frames to each of the plurality of decoding clients.

8. The method of claim 5, wherein prior to the request to change resolution, the video conference server forwards at least some of the plurality of first-resolution frames to a first decoding client of the plurality of decoding clients, and the video conference server forwards at least some of the plurality of second-resolution frames to a second decoding client of the plurality of decoding clients.

9. The method of claim 5, wherein the first-resolution non-displayed anchor frame is not output for display by any of the decoding clients from the plurality of decoding clients.

10. A method, comprising:

receiving, at a first decoding client from a first remote computing system, an encoded video stream that includes a plurality of first-resolution frames, wherein each of the first-resolution frames is encoded with reference to a respective same resolution frame from a previous time period;

decoding, at the first decoding client, at least some frames from the plurality of first-resolution frames using the respective same resolution frame from the previous time period;

receiving, at the first decoding client from the first remote computing system, a first-resolution non-displayed anchor frame, wherein the first-resolution non-displayed anchor frame is received periodically during reception of the encoded video stream;

decoding, at the first decoding client, the first-resolution non-displayed anchor frame;

subsequent to decoding the first-resolution non-displayed anchor frame, receiving, at the first decoding client, at least one second-resolution frame that is encoded relative to the decoded first-resolution non-displayed anchor frame; and

decoding, at the first decoding client, the at least one second-resolution frame using the decoded first-resolution non-displayed anchor frame.

11. The method of claim 10, further comprising:

outputting, for display by the decoding client, at least some of the at least some frames from the plurality of first-resolution frames and the at least one second-resolution frame.

12. The method of claim 10, wherein the first-resolution non-displayed anchor frame is not output for display by the decoding client.

13. The method of claim 10, further comprising:

prior to receiving the at least one second-resolution frame that is encoded relative to the first-resolution non-displayed anchor frame, transmitting a request to change resolution to the first remote computing system.

14. The method of claim 10, wherein decoding the at least one second-resolution frame using the decoded first-resolution non-displayed anchor frame includes scaling the decoded first-resolution non-displayed anchor frame to increase the resolution of the decoded first-resolution non-displayed anchor frame.

15. The method of claim 10, wherein the at least one second-resolution frame that is encoded relative to the first-resolution non-displayed anchor frame is received at the first decoding client in response to a request to change resolution that is received at the first remote computing system from a second decoding client.

16. The method of claim 10, further comprising:

storing the decoded first-resolution non-displayed anchor frame in a frame buffer at the first decoding client.

17. A method, comprising:

receiving, from an encoding client at a video conference server, an encoded video stream that includes a plurality of first-resolution frames and a plurality of second-resolution frames, wherein each of the first-resolution frames and each of the second-resolution frames is encoded with reference to a respective same resolution frame from a previous time period;

transmitting the plurality of first-resolution frames to at least some decoding clients from the plurality of decoding clients;

transmitting the plurality of second-resolution frames to at least some decoding clients from the plurality of decoding clients;

receiving, from the encoding client at the video conference server in the encoded video stream, a first-resolution non-displayed anchor frame, wherein the first-resolution non-displayed anchor frame is received periodically during receipt of the encoded video stream;

transmitting the first-resolution non-displayed anchor frame to all of the plurality of decoding clients, wherein the first-resolution non-displayed anchor frame is transmitted periodically during transmission of the plurality of first-resolution frames and transmission of the plurality of the second-resolution frames;

transmitting, to the encoding client from the video conference server, a request to change resolution;

in response to receiving the request to change resolution, receiving, from the encoding client at the video conference server in the encoded video stream, a set of frames that are encoded relative to the first-resolution non-displayed anchor frame; and

transmitting one or more frames from the set of frames to each of the plurality of decoding clients.

18. The method of claim 17, further comprising:

receiving, at the video conference server from the encoding client, a first-resolution intra frame during an initial time period, wherein at least one frame from the plurality of second-resolution frames that is transmitted during the initial time period is encoded relative to the first-resolution intra frame.

19. The method of claim 17, wherein the request to change resolution originates at a first decoding client from the plurality of decoding clients and is relayed to the encoding client by the video conference server.