Video encoder with latency control

• US 7,936,819 B2
• Filed: 07/08/2005
• Issued: 05/03/2011
• Est. Priority Date: 07/08/2005
• Status: Expired due to Fees

• Current Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Original Assignee: TAG Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Craig, Robert, Mercer, Clifford Wayne, Sigmund, Ulrich
• Primary Examiner(s): Ustaris; Joseph G
• Assistant Examiner(s): Pe; Geepy
• Application Number:
  US11/178,176
• Publication Number:
  US 20070009029A1
• Time in Patent Office: 2,125 Days
• Field of Search: 375/240.12, 375/240.13, 375/240.01, 375/240.25, 375/240.26
• US Class Current: 375/240.12
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server

Abstract

In a method of transmitting a frame of video, a number of bits corresponding to a frame of video is determined. A length of time needed to transmit the number of bits is determined and is compared with a length of time corresponding to a frame period. A frame period may correspond to an inverse of an average frame update rate. If the length of time exceeds the length of time corresponding to the frame period, one or more P-frames including a plurality of empty macro-blocks may be transmitted and the frame of video may then be transmitted over period of time exceeding one frame period. In some cases, the transmission of the frame of video may equal two or more frame periods. The transmitting may include transmitting an instruction for setting a latency in a client device to less than 50 ms.

Claims

1. A method, comprising:

at a server system comprising one or more processors and memory storing one or more programs configured to be executed by the one or more processors;

determining a number of bits corresponding to a frame of video;

determining a length of time needed to transmit the number of bits;

determining that the length of time exceeds a frame period, wherein the frame period corresponds to an inverse of an average frame update rate; and

in response to determining that the length of time exceeds the frame period;

transmitting one or more P-frames that each comprise a plurality of empty macro-blocks, wherein each of the one or more P-frames is transmitted over a period of time less than the frame period; and

after transmitting the one or more P-frames, transmitting the frame of video over a period of time exceeding one frame period.

5. The method of claim 1, wherein the frame of video is a P-frame.

6. The method of claim 1, wherein the frame of video comprises update information for a previously transmitted frame of video.

7. The method of claim 1, wherein the frame of video is transmitted over 3 to 3.9 frame periods.

8. The method of claim 7, wherein the one or more P-frames comprise three P-frames.

9. The method of claim 1, wherein all macro-blocks of the one or more P-frames are empty.

10. The method of claim 1, wherein each macro-block of the one or more P-frames is either empty or skipped.

11. The method of claim 1, wherein the server system is a video game system comprising one or more video game servers.

12. The method of claim 11, wherein the one or more P-frames and the frame of video are transmitted in response to a user command in a video game.

13. The method of claim 1, further comprising setting a latency less than 50 ms.

2. A method, comprising:

at a server system comprising one or more processors and memory storing one or more programs configured to be executed by the one or more processors;

determining a number of bits corresponding to a frame of video;

determining a length of time needed to transmit the number of bits;

determining that the length of time exceeds a frame period, wherein the frame period corresponds to an inverse of an average frame update rate; and

in response to determining that the length of time exceeds the frame period;

transmitting one or more P-frames that each comprise a plurality of empty macro-blocks, wherein each of the one or more P-frames is transmitted over a period of time less than the frame period; and

after transmitting the one or more P-frames, transmitting the frame of video over two or more frame periods.

3. The method of claim 2, wherein:

transmitting the one or more P-frames and the frame of video comprises setting a latency less than 50 ms; and

setting the latency includes defining a first time and a second time, the first time corresponding to a global clock and the second time corresponding to an offset relative to the global clock, and wherein the latency corresponds to a time difference between an arrival of the number of bits corresponding to the frame of video and decoding of the number of bits.

4. An apparatus, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising;

instructions for determining a number of bits corresponding to a frame of video;

instructions for determining a length of time needed to transmit the number of bits;

instructions for determining that the length of time exceeds a frame period, wherein the frame period corresponds to an inverse of an average frame update rate;

instructions for transmitting, in response to determining that the length of time exceeds the frame period, one or more P-frames that each comprise a plurality of empty macro-blocks, wherein each of the one or more P-frames is to be transmitted over a period of time less than the frame period; and

instructions for transmitting the frame of video after transmitting the one or more P-frames, wherein the frame of video is to be transmitted over a period of time exceeding one frame period.

14. The apparatus of claim 4, wherein the frame of video is a P-frame.

15. The apparatus of claim 4, wherein the instructions for transmitting the frame of video comprise instructions for transmitting the frame of video over two or more frame periods.

16. The apparatus of claim 4, wherein the instructions for transmitting the frame of video comprise instructions for transmitting the frame of video over 3 to 3.9 frame periods.

17. The apparatus of claim 16, wherein the instructions for transmitting the one or more P-frames comprise instructions for transmitting three P-frames.

18. The apparatus of claim 4, wherein each macro-block of the one or more P-frames is either empty or skipped.

19. The apparatus of claim 4, further comprising instructions to set a latency less than 50 ms.