Remote access encoding

• US 9,225,979 B1
• Filed: 01/30/2013
• Issued: 12/29/2015
• Est. Priority Date: 01/30/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jia, Wei, Gu, Qunshan
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): Nawaz, Talha
• Application Number:
  US13/753,632
• Time in Patent Office: 1,063 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/507 using conditional replenishment

Abstract

A method and apparatus for remote access encoding is provided. Remote access encoding may include receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device, rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames, generating an encoded block, and transmitting encoded content to the client device, wherein the encoded content includes the encoded block. Generating the encoded block may include identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames, determining whether the current block is a static block, determining a coding quality for encoding the current block, and determining whether to encode the current block as a skipped block.

Claims

1. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames,

determining whether the current block is a static block, by determining that the current block is a static block on a condition that a difference between the current block and a corresponding raw reference block does not exceed a threshold and determining that the current block is not a static block on a condition that the difference between the current block and the raw reference block exceeds the threshold, wherein determining that the current block is a static block includes associating the current block with a zero motion vector,

determining a coding quality for encoding the current block, and

determining whether to encode the current block as a skipped block; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.

2. The method of claim 1, wherein determining whether the current block is a static block includes determining whether the current block includes static content.

3. The method of claim 1, wherein the raw reference block is one of a plurality of blocks in a raw reference frame, and wherein a position of the raw reference block in the raw reference frame corresponds with a position of the current block in the current frame offset by information indicating movement of a window in the operating environment, and wherein determining that the current block is a static block includes associating the current block with a non-zero motion vector.

4. The method of claim 1, wherein determining the coding quality for encoding the current block includes determining that the encoding quality is high quality on a condition that the current block is a static block.

5. The method of claim 1, wherein determining whether to encode the current block as a skipped block includes encoding the current block as a skipped block and indicating that the skipped block is a high quality block on a condition that a reference block for encoding the current block is a high quality reference block and the current block is a static block.

6. The method of claim 1, wherein determining whether to encode the current block as a skipped block includes encoding the current block as a skipped block and indicating that the skipped block is a low quality block on a condition that a reference block for encoding the current block is a low quality reference block and the current block is a static block.

7. The method of claim 1, wherein determining whether to encode the current block as a skipped block includes determining whether a reference block for encoding the current block is a high quality reference block.

8. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames

determining whether the current block is a static block,

determining a coding quality for encoding the current block, wherein determining the coding quality for encoding the current block includes determining that the encoding quality is high quality on a condition that the current block includes a portion of a topmost window of the operating environment, and

determining whether to encode the current block as a skipped block; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.

9. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames

determining whether the current block is a static block,

determining a coding quality for encoding the current block, wherein determining the coding quality for encoding the current block includes determining that the encoding quality is high quality on a condition that the current block includes a portion of a recently in-focus window of the operating environment, and

determining whether to encode the current block as a skipped block; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.

10. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames

determining whether the current block is a static block,

determining a coding quality for encoding the current block, wherein determining the coding quality for encoding the current block includes determining that the encoding quality is high quality on a condition that the current block includes a portion of a frequently in-focus window of the operating environment, and

determining whether to encode the current block as a skipped block; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.

11. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames

determining whether the current block is a static block,

determining a coding quality for encoding the current block, and

determining whether to encode the current block as a skipped block, wherein

determining whether to encode the current block as a skipped block includes determining whether a reference block for encoding the current block is a high quality reference block, and

wherein determining whether the reference block is a high quality reference block includes determining whether the reference block was encoded using high quality encoding based on a difference between a reference frame and a corresponding reconstructed frame, wherein the reference block is one of a plurality of blocks from the reference frame; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.

12. The method of claim 11, wherein determining whether the reference block was encoded using high quality encoding includes determining a sum of absolute differences in a transform domain or a spatial domain between the reference frame and the corresponding reconstructed frame, such that the reference block was encoded using high quality encoding on a condition that the sum of absolute differences does not exceed a threshold, and the reference block was encoded using low quality encoding on a condition that the sum of absolute differences exceeds the threshold.

13. The method of claim 11, wherein determining whether the reference block is a high quality reference block includes:

determining that the reference block is a high quality reference block on a condition that the reference block was encoded using high quality encoding and is aligned with a block boundary;

determining that the reference block is a high quality reference block on a condition that the reference block overlaps with a plurality of blocks that were encoded using high quality encoding; and

determining that the reference block is a low quality reference block on a condition that the reference block overlaps with a block that was encoded using low quality encoding.

14. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames,

identifying a reference block from a plurality of blocks in a reference frame,

determining whether the current block is a static block, wherein determining whether the current block is a static block includes at least one of determining that the current block is a static block and associating the current block with a zero motion vector, on a condition that a difference between the current block and a corresponding raw reference block does not exceed a threshold, wherein the reference block is a decoded block based on the raw reference block, determining that the current block is a static block and associating the current block with a non-zero motion vector, on a condition that a difference between the current block and a corresponding raw reference block does not exceed a threshold, wherein the raw reference block is one of a plurality of blocks in a raw reference frame, and wherein a position of the raw reference block in the raw reference frame corresponds with a position of the current block in the current frame offset by information indicating movement of a window in the operating environment, or determining that the current block is not a static block on a condition that the difference between the current block and the raw reference block exceeds the threshold,

on a condition that the reference block is a high quality reference block and the current block is a static block, encoding the current block as a skipped block and indicating that the skipped block is a high quality block, and

on a condition that the reference block is a low quality reference block and the current block is a static block, encoding the current block as a skipped block and indicating that the skipped block is a low quality block; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.

15. The method of claim 14, wherein determining whether the current block is a static block includes determining whether the current block includes static content.

16. The method of claim 14, wherein generating the encoded block includes determining whether the reference block is a high quality reference block.

17. A method of remote access encoding, the method comprising:

receiving, at a host device, from a client device, a remote access request indicating a portion of a display area of an operating environment of the host device;

rendering a representation of the portion of the display area, wherein rendering includes generating rendered content including a plurality of frames;

generating an encoded block by;

identifying a current block from a plurality of blocks in a current frame, wherein the current frame is one of the plurality of frames,

identifying a reference block from a plurality of blocks in a reference frame,

determining whether the reference block is a high quality reference block, wherein determining whether the reference block is a high quality reference block includes at least one of determining whether the reference block was encoded using high quality encoding based on a difference between the reference frame and a corresponding reconstructed frame, determining that the reference block is a high quality reference block on a condition that the reference block was encoded using high quality encoding and is aligned with a block boundary, determining that the reference block is a high quality reference block on a condition that the reference block overlaps with a plurality of blocks that were encoded using high quality encoding, or determining that the reference block is a low quality reference block on a condition that the reference block overlaps with a block that was encoded using low quality encoding,

on a condition that the reference block is a high quality reference block and the current block is a static block, encoding the current block as a skipped block and indicating that the skipped block is a high quality block, and

on a condition that the reference block is a low quality reference block and the current block is a static block, encoding the current block as a skipped block and indicating that the skipped block is a low quality block; and

transmitting encoded content to the client device, wherein the encoded content includes the encoded block.