RENDER-ASSISTED COMPRESSION FOR REMOTE GRAPHICS

• US 20140028703A1
• Filed: 12/26/2012
• Published: 01/30/2014
• Est. Priority Date: 07/30/2012
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Luebke, David, PAJAK, Dawid STANISLAW, SAULTERS, Scott
• Granted Patent: US 9,576,340 B2
• Time in Patent Office: Days
• US Class Current: 345/619
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 9/004 Predictors, e.g. intraframe, interframe coding
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A technique for efficiently compressing rendered three-dimensional images in a remote rendering system adds a novel render-assisted prediction function to an existing video compression framework, such as the standard H.264/5 framework. Auxiliary rendering information is separated from rendering information used to describe a reference image by a server system. A client system may alter the auxiliary data and generate a new image based on the reference image and rendered scene information from the auxiliary data without creating additional network bandwidth or server workload.

Claims

1. A method for transmitting rendered images to a remote client device, the method comprising:

rendering one or more server rendering elements to generate a reference image;

rendering one or more client rendering elements to generate a client auxiliary image;

compressing the reference image and the client auxiliary image or the reference image and auxiliary data to generate compressed client data; and

transmitting the compressed client data to the remote client device for display.

2. The method of claim 1, wherein each server rendering element comprises one or more graphics operations configured to be executed on a server machine.

3. The method of claim 2, wherein each client rendering element comprises one or more graphics operations configured to be executed on the server machine and the remote client device.

4. The method of claim 3, further comprising organizing a set of application programming interface (API) calls to differentiate the one or more server rendering elements from the one or more client rendering elements.

5. The method of claim 1, wherein the auxiliary data comprises graphics instructions for performing at least one rendering pass on the reference image.

6. The method of claim 1, further comprising comparing an amount of data associated with the client auxiliary image to an amount of data associated with the auxiliary data.

7. The method of claim 6, wherein the reference image and the client auxiliary image are compressed to generate the compressed client data if the amount of data associated with the client auxiliary image is less than the amount of data associated with the auxiliary data.

8. The method of claim 6, wherein the reference image and the auxiliary data are compressed to generate the compressed client data if the amount of data associated with the auxiliary data is less than the amount of data associated with the client auxiliary image.

9. The method of claim 1, wherein compressing the auxiliary data comprises applying a render-assisted prediction function to the auxiliary data.

10. The method of claim 9, wherein compressing the reference image comprises entropy encoding at least a portion of the reference image.

11-21. -21. (canceled)

22. A server system, comprising:

a rendering subsystem configured to;

render one or more server rendering elements to generate a reference image;

render one or more client rendering elements to generate a client auxiliary image;

compress the reference image and the client auxiliary image or the reference image and auxiliary data to generate compressed client data; and

transmit the compressed client data to the remote client device for display.

23. The server system of claim 22, wherein each server rendering element comprises one or more graphics operations configured to be executed on a server machine.

24. The server system of claim 23, wherein each client rendering element comprises one or more graphics operations configured to be executed on the server machine and the remote client device.

26. The server system of claim 22, wherein the auxiliary data comprises graphics instructions for performing at least one rendering pass on the reference image.

27. The server system of claim 22, wherein the rendering subsystem is further configured to compare an amount of data associated with the client auxiliary image to an amount of data associated with the auxiliary data.

28. The server system of claim 27, wherein the reference image and the client auxiliary image are compressed to generate the compressed client data if the amount of data associated with the client auxiliary image is less than the amount of data associated with the auxiliary data.

29. The server system of claim 27, wherein the reference image and the auxiliary data are compressed to generate the compressed client data if the amount of data associated with the auxiliary data is less than the amount of data associated with the client auxiliary image.

30. The server system of claim 22, wherein the rendering subsystem is configured to compress the auxiliary data by applying a render-assisted prediction function to the auxiliary data.

31. The server system of claim 30, wherein the rendering subsystem is configured to compress the reference image by entropy encoding at least a portion of the reference image.

32. The server system of claim 22, wherein the rendering subsystem includes:

a render module configured to generate the reference image and the client auxiliary image; and

a first compression module to compress the reference image and the client auxiliary image or the reference image and the auxiliary data to generate the compressed client data.

33. A computing device, comprising:

a memory configured to store a server application; and

a rendering subsystem coupled to the memory and configured to;

render one or more server rendering elements to generate a reference image;

render one or more client rendering elements to generate a client auxiliary image;

compress the reference image and the client auxiliary image or the reference image and auxiliary data to generate compressed client data; and

transmit the compressed client data to the remote client device for display.