System method and computer program product for remote graphics processing

• US 7,274,368 B1
• Filed: 07/31/2000
• Issued: 09/25/2007
• Est. Priority Date: 07/31/2000
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Silicon Graphics Incorporated (Graphics Properties Holdings, Inc.)
• Inventors: Keslin, Phillip C.
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): CUNNINGHAM, GREGORY F
• Application Number:
  US09/629,458
• Time in Patent Office: 2,612 Days
• Field of Search: 345/418, 345/522, 345/530, 345/531, 345/581, 345/619, 345/744, 345/749, 345/FOR.185, 348/211.3, 709/107, 709/203, 709/219, 709/311, 709/217, 709/FOR.106, 709/FOR.112, 725/4, 725/709, 719322-330
• US Class Current: 345/522
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/36 characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A system, method, and computer program product are provided for remote rendering of computer graphics. The system includes a graphics application program resident at a remote server. The graphics application is invoked by a user or process located at a client. The invoked graphics application proceeds to issue graphics instructions. The graphics instructions are received by a remote rendering control system. Given that the client and server differ with respect to graphics context and image processing capability, the remote rendering control system modifies the graphics instructions in order to accommodate these differences. The modified graphics instructions are sent to graphics rendering resources, which produce one or more rendered images. Data representing the rendered images is written to one or more frame buffers. The remote rendering control system then reads this image data from the frame buffers. The image data is transmitted to the client for display or processing. In an embodiment of the system, the image data is compressed before being transmitted to the client. In such an embodiment, the steps of rendering, compression, and transmission can be performed asynchronously in a pipelined manner.

Claims

1. A method of remote graphics rendering on behalf of a client, comprising the steps of:

(A) initializing a graphics rendering session;

(B) starting a graphics application on the basis of a command from the client;

(C) generating graphics instructions;

(D) imposing client parameters to produce modified graphics instructions;

(E) sending the modified graphics instructions to graphics rendering resources;

(F) rendering graphics on the basis of the modified graphics instructions to produce image data in one or more frame buffers;

(G) reading image data from the one or more frame buffers;

(H) enqueuing the image data; and

(I) transmitting the image data to the client.

2. The method of claim 1, further comprising the step of:

(J) compressing the image data, wherein step (J) is performed after step (H) and before step (I).

3. The method of claim 2, wherein steps (F), (J), and (I) are performed in pipeline fashion.

4. The method of claim 3, wherein steps (F), (J), and (I) are asynchronous.

5. The method of claim 1, wherein step (A) comprises the steps of:

(i) performing a client/server handshake;

(ii) receiving a client visual from the client;

(iii) after a user at the client opens a console window at the client and starts the graphics application, opening client and server displays;

(iv) merging the client visual with a server visual to form a merged visual list;

(v) associating the client display with the graphics application;

(vi) overlaying the server visual list with a transparent interface routine;

(vii) enabling the return of a client window to the graphics application;

(viii) enabling the return of an internal context to the graphics application; and

(ix) binding a server context to a server window.

6. The method of claim 5, wherein step (vii) comprises the steps of:

(a) converting the merged visual list into a visual appropriate for the client;

(b) defining the client window;

(c) creating an internal data structure for tracking the displayed location of the client window; and

(d) returning the client window to the graphics application.

7. The method of claim 5, wherein step (viii) comprises the steps of:

(e) converting the merged visual list into a visual appropriate for the server;

(f) creating a server context; and

(g) returning an internal context to the application.

8. The method of claim 5, wherein step (ix) comprises the steps of:

(h) extracting a server context from the internal context;

(i) requesting a window allocation from a session manager; and

(j) associating the server context with a server window.

9. The method of claim 1, wherein step (D) comprises the steps of:

(x) intercepting every function call that includes a visual capability;

(xi) converting the visual capability to a corresponding client visual capability;

(xii) intercepting every reference to a graphics context; and

(xiii) converting every reference to a graphics context to a reference to a graphics context of the client.