Resource-aware desktop image decimation method and apparatus

• US 9,819,937 B1
• Filed: 04/14/2015
• Issued: 11/14/2017
• Est. Priority Date: 04/14/2015
• Status: Active Grant

• Current Assignee: Teradici Corporation (HP Inc.)
• Original Assignee: Teradici Corporation (HP Inc.)
• Inventors: Helter, Paul Andrew, Savage, Joseph Michel Kenneth
• Primary Examiner(s): Hance, Robert
• Application Number:
  US14/686,320
• Time in Patent Office: 945 Days
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/428 Recompression, e.g. by spatial or temporal decimation
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Abstract

A method and apparatus for transmitting a desktop display comprising classifying a first region of the desktop display as persistently changed and a second region of the desktop display as sporadically changed, adjusting, in relation to a user experience (UX) bias and a resource constraint, a target image quality for the first region, decimating the first region in accordance with a spatial decimation factor to generate a first decimated region, compressing the first decimated region at the target image quality and compressing the second region to generate a plurality of compressed regions and transmitting the plurality of compressed regions to a client via an IP network.

Claims

1. A method for transmitting a desktop display comprising:

classifying a first region of the desktop display as persistently changed and a second region of the desktop display as sporadically changed, wherein the first region is selectively chosen from a sequence of frames of the desktop display to maintain a specified central processing unit (CPU) resource utilization target;

adjusting, in relation to a user experience (UX) bias and a resource constraint, a target image quality for the first region;

decimating the first region in accordance with a spatial decimation factor to generate a first decimated region;

compressing the first decimated region at the target image quality and compressing the second region to generate a plurality of compressed regions; and

transmitting the plurality of compressed regions to a client via an IP network.

2. The method of claim 1, wherein the spatial decimation factor is at least four to one.

3. The method of claim 1, wherein the target image quality is further specified by a quantization value weighted by the UX bias.

4. The method of claim 1, wherein the UX bias comprises a value on a trade-off scale between maximum image quality and maximum frame rate.

5. The method of claim 1, wherein the UX bias is adjusted in response to input received from the client.

6. The method of claim 1, wherein the resource constraint is adjusted for resource utilization requirements of the second region.

7. The method of claim 1, wherein decimating comprises,

representing a block of pixels as a single pixel, the single pixel comprising a gamma-corrected average of the block of pixels.

8. The method of claim 1, wherein compressing further comprises:

transforming the decimated first region and the second region using a block size common to the decimated first region and the second region.

9. The method of claim 1, wherein compressing the second region comprises:

generating a progressive refinement of a previously changed block of pixels.

10. The method of claim 1, further comprising:

demarcating, prior to decimating, a zone in the first region which is excluded from decimating.

11. The method of claim 1, wherein the resource constraint comprises the most significant of i) a bandwidth constraint of the IP network, ii) a client processor utilization target and iii) a host processor utilization target.

12. The method of claim 11, wherein the client processor utilization target is reduced if the first region is determined to be out of focus.

13. The method of claim 11, wherein the host processor utilization target is reduced if the first region is determined to be out of focus.

14. The method of claim 11, wherein the client processor utilization target is adjusted according to power consumption requirements of the client.

15. The method of claim 1 further comprising, prior to decimating and compressing:

predicting resource utilization based on the target image quality and readjusting the target image quality in relation to at least a second resource constraint and the UX bias.

16. The method of claim 15, wherein adjusting the target image quality comprises:

determining a dampened quality value determined respective of a previous image quality of a previous frame and an un-dampened image quality of a current frame being processed for decimation and compression.

17. An apparatus for transmitting a desktop display comprising:

a host processor; and

memory, storing thereon computer executable code for a resource aware encoder that when executed by the host processor performs a method comprising;

classifying a first region of the desktop display as persistently changed and a second region of the desktop display as sporadically changed, wherein the first region is selectively chosen from a sequence of frames of the desktop display to maintain a specified central processing unit (CPU) resource utilization target;

adjusting, in relation to a user experience (UX) bias and a resource constraint, a target image quality for the first region;

decimating the first region in accordance with a spatial decimation factor to generate a first decimated region; and

compressing the first decimated region at the target image quality and compressing the second region to generate a plurality of compressed regions,

wherein the host processor transmits the plurality of compressed regions to a client via an IP network.

18. The apparatus of claim 17, wherein the resource aware encoder further:

demarcates, prior to decimating, a zone in the first region which is excluded from decimating.

19. The apparatus of claim 17 wherein the resource aware encoder, prior to decimating and compressing:

predicts resource utilization based on the target image quality and readjusting the target image quality in relation to at least a second resource constraint and the UX bias.

20. A method for transmitting a desktop display comprising:

classifying a first region of the desktop display as persistently changed and a second region of the desktop display as sporadically changed;

adjusting, in relation to a user experience (UX) bias and a resource constraint, a target image quality for the first region, wherein the resource constraint comprises one of i) a bandwidth constraint of an IP network, ii) a client processor utilization target and iii) a host processor utilization target, and wherein the client processor utilization target is reduced if the first region is determined to be out of focus or adjusted according to power consumption requirements of a client, and the host processor utilization target is reduced if the first region is determined to be out of focus;

decimating the first region in accordance with a spatial decimation factor to generate a first decimated region;

compressing the first decimated region at the target image quality and compressing the second region to generate a plurality of compressed regions; and

transmitting the plurality of compressed regions to the client via an IP network.