Multi-user multi-GPU render server apparatus and methods

• US 9,355,616 B2
• Filed: 11/23/2012
• Issued: 05/31/2016
• Est. Priority Date: 11/23/2007
• Status: Active Grant

• Current Assignee: PME IP Pty., Ltd.
• Original Assignee: PME IP Pty., Ltd.
• Inventors: Westerhoff, Malte, Stalling, Detlev
• Primary Examiner(s): Ma, Tize
• Application Number:
  US13/684,464
• Publication Number:
  US 20130176319A1
• Time in Patent Office: 1,285 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2210/52 Parallel processing
• CPC Class Codes: G09G 2360/06 Use of more than one graphics processor to process data before displaying to one or more screens
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 51/216 Handling conversation history, e.g. grouping of messages in sessions or threads
• CPC Class Codes: H04L 67/08 specially adapted for terminal emulation, e.g. Telnet

Abstract

The invention provides, in some aspects, a system for rendering images, the system having one or more client digital data processors and a server digital data processor in communications coupling with the one or more client digital data processors, the server digital data processor having one or more graphics processing units. The system additionally comprises a render server module executing on the server digital data processor and in communications coupling with the graphics processing units, where the render server module issues a command in response to a request from a first client digital data processor. The graphics processing units on the server digital data processor simultaneously process image data in response to interleaved commands from (i) the render server module on behalf of the first client digital data processor, and (ii) one or more requests from (a) the render server module on behalf of any of the other client digital data processors, and (b) other functionality on the server digital data processor.

Claims

1. A method to render image requests comprising the steps of:

a) executing a render server program;

b) receiving two or more render requests to generate two or more rendered images from one or more clients, where each of the two or more render requests include a time of receipt of the render request;

c) selecting from the two or more render requests the render request requiring the most processing time as a large render request, where one or more render requests are not selected as the large render request;

d) ‘

breaking down’

the large render request into a plurality of partial render requests including at least a first partial render request a second partial render request and remaining partial render requests, where each of the plurality of partial render requests require one or both less compute time and less graphics resources than one or both the requirements for compute time and graphics resources of the large render request;

e) assigning a time of receipt of the first partial render request based on the time of receipt of the large render request;

f) assigning a time of receipt of the second partial render request such that at least one of the one or more render requests is assigned a time of receipt after the time of receipt of the first partial render request and before the time of receipt of the second partial render request;

g) assigning a time of receipt of the remaining partial render request such that at least one of the one or more render requests is assigned a time of receipt after the time of receipt of the second partial render request and before the time of receipt of the remaining partial render requests;

h) determining a sequence of interleaved commands for rendering the one or more render requests and the plurality of partial render requests based on the time of receipt of the one or more render requests and the assigned time of receipt of the first partial render request and the assigned time of receipt of the second partial render request and the assigned time of receipt of the remaining partial render requests;

i) issuing the sequence of interleaved commands to one or more graphics processing units to generate the two or more rendered images; and

j) sending to the one or more clients the two or more rendered images.

2. The method of claim 1, further comprising storing one or more datasets corresponding to one or more of the render requests in a host memory associated with a server digital data processor.

3. The method of claim 1, further comprising maintaining requests received from one or more client digital data processors in one or more queues associated with a server digital data processor, including further prioritizing the requests and removing requests.

4. The method of claim 3, where the further prioritizing step includes any of prioritizing a render request based on at least one of a rendering mode associated therewith, a client associated therewith, and available resources.

5. The method of claim 1, further comprising determining the sequence of interleaved commands to minimize an average wait time of one or more of the two or more render requests.

6. The method of claim 5, further comprising processing with the one or more graphics processing units multiple interleaved commands, each based on the two or more render requests.

7. The method of claim 1, further comprising rendering with the one or more graphics processing units an image at a rendering resolution determined by one or more parameters, including, at least one of a user interaction type, a network speed, and available processing resources.

8. The method of claim 1, further comprising allocating at least a portion of one or more server digital data processor resources in response to one or more requests received from one or more client digital data processors.

9. The method of claim 8, further comprising allocating, as a server digital data processor resource, a graphics memory that is coupled to any of the one or more graphics processing units.

10. The method of claim 1, further comprising rendering an individual image for each of the plurality of partial render requests.

11. The method of claim 10, further comprising choosing the brightest value for a pixel in a rendered image based on the corresponding brightest value for that pixel in the two or more partial rendered images.

12. The method of claim 1, further comprising:

a) associating a pixel in a dataset, corresponding to one or more of the plurality of partial render requests, with a viewing ray;

b) determining one or more voxels intersected by the viewing ray;

c) determining the maximum gray value among the one or more intersected voxels; and

d) assigning the maximum gray value to the pixel in the dataset.

13. The method of claim 1, further comprising determining the sequence of interleaved commands to minimize an average wait time of one or more of the plurality of partial render requests.

14. A method to render image requests comprising the steps of:

a) receiving a plurality of render requests at one or more request times from two or more clients;

b) selecting the render request requiring most processing time as a large render request and the remaining render requests as smaller render requests;

c) breaking down the large render request into a first partial render request and one or more remaining partial render requests, where the first partial render request is assigned the request time of the largest render request and the remaining partial render requests are assigned a time of receipt such that at least one of the smaller render requests is assigned a time of receipt after the time of receipt of the first partial render request and before the time of receipt of the remaining partial render requests;

d) determining a sequence of interleaved commands based on one or more of the time of receipt of the smaller render requests, the assigned time of receipt of the first partial render request and the assigned time of receipt of the remaining partial render requests;

e) issuing the sequence of interleaved commands to one or more graphics processing units to generate a plurality of rendered images; and

f) sending to the two or more clients the plurality of rendered images.

15. The method of claim 14, further comprising where the first partial render request and the one or more remaining partial render requests are rendered independently and choosing the brightest value for a pixel in a rendered image based on the corresponding brightest value for that pixel in the rendered image generated from the first partial render request and the rendered images generated for the one or more remaining partial rendered images.

16. The method of claim 14, further comprising:

a) associating a pixel in a dataset corresponding with the first partial render request and the one or more remaining partial render requests with a viewing ray;

b) determining one or more voxels intersected by the viewing ray;

c) determining the maximum gray value among the one or more intersected voxels; and

d) assigning the maximum gray value to the pixel in the dataset;

b) determining one or more voxels intersected by the viewing ray;

c) determining the maximum gray value among the one or more intersected voxels; and

d) assigning the maximum gray value to the pixel in the dataset.

17. A method to render image requests comprising the steps of:

a) receiving a render request of a dataset from a client;

b) breaking down the render request into a first partial render request and one or more remaining partial render requests;

c) determining a sequence of interleaved commands to render the first partial render request and to render the one or more remaining partial render requests;

d) issuing the sequence of interleaved commands to one or more graphics processing units to generate rendered images corresponding to the first partial render request and the one or more remaining partial render requests, where the first partial render request is assigned the request time of the render request and the one or more remaining partial render requests are assigned a time of receipt such that at least a second render requests is assigned a time of receipt after the time of receipt of the first partial render request and before the time of receipt of the remaining partial render requests;

e) selecting the brightest value for a pixel based on the corresponding brightest value for that pixel in the rendered image generated from the first partial render request and the one or more rendered images generated from the remaining partial render requests; and

f) generating a composite render image based on the brightest value for each pixel.

18. The method of claim 17, where the render request is broken down into a first partial render request, a second partial render request, and a third partial render request and the brightest value for a pixel is selected by comparing the value for that pixel in the rendered image generated from the first partial render request, the rendered image generated from the second partial render request and the rendered image generated from the third partial render request.

19. The method of claim 17, further comprising:

a) associating a pixel in the dataset with a viewing ray;

b) determining one or more voxels intersected by the viewing ray;

c) determining the maximum gray value among the one or more intersected voxels; and

d) assigning the maximum gray value to the pixel in the dataset.