System, method, and computer program product for remote graphics processing

• US 20080049030A1
• Filed: 08/24/2007
• Published: 02/28/2008
• Est. Priority Date: 07/31/2000
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Silicon Graphics Incorporated (Graphics Properties Holdings, Inc.)
• Inventors: Keslin, Phillip
• Granted Patent: US 8,427,491 B2
• Time in Patent Office: Days
• US Class Current: 345/522
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/36 characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A system, method, and computer program product are provided for remote rendering of computer graphics. The system includes a graphics application program resident at a remote server. The graphics application is invoked by a user or process located at a client. The invoked graphics application proceeds to issue graphics instructions. The graphics instructions are received by a remote rendering control system. Given that the client and server differ with respect to graphics context and image processing capability, the remote rendering control system modifies the graphics instructions in order to accommodate these differences. The modified graphics instructions are sent to graphics rendering resources, which produce one or more rendered images. Data representing the rendered images is written to one or more frame buffers. The remote rendering control system then reads this image data from the frame buffers. The image data is transmitted to the client for display or processing. In an embodiment of the system, the image data is compressed before being transmitted to the client. In such an embodiment, the steps of rendering, compression, and transmission can be performed asynchronously in a pipelined manner.

Claims

1. A system for rendering image data, comprising:

a graphics application, at a server, configured to receive commands from a client and to generate graphics instructions, wherein the graphics instructions are formatted as an implementation of an application programming interface for producing an image based upon a mathematical representation of an object; and

a remote rendering control system, at the server, configured to receive the graphics instructions from the graphics application and to generate modified graphics instructions based upon differences, between the client and the server, in at least one of imaging processing capabilities and graphics contexts.

2. The system of claim 1, wherein the application programming interface is OpenGL.

3. The system of claim 1, further comprising:

remote graphics rendering resources, at the server, configured to receive the modified graphics instructions from the remote rendering control system and to render, by a rendering process, the image data.

4. The system of claim 3, wherein the remote rendering control system is further configured to receive the image data from the remote graphics rendering resources and to transmit, by a transmission process, the image data to the client.

5. The system of claim 4, wherein the remote rendering control system is further configured to compress, by a compression process, the image data and wherein the rendering process of a first block of data is configured to occur during a first time period, the compression process of a second block of data is configured to occur during a second time period, and the transmission process of a third block of data is configured to occur during a third time period so that the rendering process, the compression process, and the transmission process are pipelined.

6. The system of claim 5, wherein the remote rendering control system is further configured to discard the first block of data if the rendering process of the first block of data is completed before the compression process of the second block of data and to discard the second block of data if the compression process of the second block of data is completed before the transmission process of the third block of data.

7. The system of claim 1, wherein the remote rendering control system is configured to convert the at least one of the imaging processing capabilities and the graphics contexts appearing in function calls of the graphics application from the at least one of the imaging processing capabilities and the graphics contexts of the server to the at least one of the imaging processing capabilities and the graphics contexts of the client to generate the modified graphics instructions based upon the differences, between the client and the server, in the at least one of the imaging processing capabilities and the graphics contexts.

8. A method for rendering image data, comprising:

(A) receiving, at a server, a command from a client;

(B) generating, at the server, graphics instructions, wherein the graphics instructions are formatted as an implementation of an application programming interface for producing an image based upon a mathematical representation of an object; and

(C) modifying, at the server, the graphics instructions based upon differences, between the client and the server, in at least one of imaging processing capabilities and graphics contexts.

9. The method of claim 8, further comprising:

(D) rendering, by a rendering process at the server, the image data from the modified graphics instructions.

10. The method of claim 9, further comprising:

(E) transmitting, by a transmission process at the server, the image data to the client.

11. The method of claim 10, further comprising:

(F) compressing, by a compression process at the server, the image data, wherein the rendering process of a first block of data is configured to occur during a first time period, the compression process of a second block of data is configured to occur during a second time period, and the transmission process of a third block of data is configured to occur during a third time period so that the rendering process, the compression process, and the transmission process are pipelined.

12. The method of claim 11, further comprising:

(G) discarding, at the server, the first block of data if the rendering process of the first block of data is completed before the compression process of the second block of data; and

(H) discarding, at the server, the second block of data if the compression process of the second block of data is completed before the transmission process of the third block of data.

13. The method of claim 8, wherein the modifying the graphics instructions comprises converting the at least one of the imaging processing capabilities and the graphics contexts appearing in function calls of a graphics application from the at least one of the imaging processing capabilities and the graphics contexts of the server to the at least one of the imaging processing capabilities and the graphics contexts of the client.

14. A computer program product rendering image data, the computer program product having computer program code means embodied in a computer-readable medium, the computer program code means comprising:

a first program code means for causing a server to receive a command from a client;

a second program code means for causing the server to generate graphics instructions, wherein the graphics instructions are formatted as an implementation of an application programming interface for producing an image based upon a mathematical representation of an object; and

a third program code means for causing the server to modify the graphics instructions based upon differences, between the client and the server, in at least one of imaging processing capabilities and graphics contexts.

15. The computer program product of claim 14, wherein the application programming interface is OpenGL.

16. The computer program product of claim 14, further comprising a fourth program code means for causing the server to render, by a rendering process, the image data from the modified graphics instructions.

17. The computer program product of claim 16, further comprising a fifth program code means for causing the server to transmit, by a transmission process, the image data to the client.

18. The computer program product of claim 17, further comprising a sixth program code means for causing the server to compress, by a compression process, the image data, wherein the rendering process of a first block of data is configured to occur during a first time period, the compression process of a second block of data is configured to occur during a second time period, and the transmission process of a third block of data is configured to occur during a third time period so that the rendering process, the compression process, and the transmission process are pipelined.

19. The computer program product of claim 18, further comprising:

a seventh program code means for causing the server to discard the first block of data if the rendering process of the first block of data is completed before the compression process of the second block of data; and

an eighth program code means for causing the server to discard the second block of data if the compression process of the second block of data is completed before the transmission process of the third block of data.

20. The computer program product of claim 14, wherein the third program code means comprises a fourth program code means for causing the server to convert the at least one of the imaging processing capabilities and the graphics contexts appearing in function calls of a graphics application from the at least one of the imaging processing capabilities and the graphics contexts of the server to the at least one of the imaging processing capabilities and the graphics contexts of the client.