Client-server visualization system with hybrid data processing

• US 10,380,970 B2
• Filed: 07/16/2018
• Issued: 08/13/2019
• Est. Priority Date: 11/23/2007
• Status: Active Grant

• Current Assignee: PME IP Pty., Ltd.
• Original Assignee: PME IP Pty., Ltd.
• Inventors: Westerhoff, Malte, Stalling, Detlev
• Primary Examiner(s): Madamba, Glenford J
• Application Number:
  US16/036,451
• Publication Number:
  US 20180322846A1
• Time in Patent Office: 393 Days
• Field of Search: 709203
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G09G 2360/12 Frame memory handling
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: G09G 5/395 Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen G09G5/399 takes precedence

Abstract

The invention comprises a system of client-server visualization with hybrid data processing, having a server digital data processor, that allows for server side rendering and processing image data, and client digital data processors simultaneously connected to the server, which receives messages from the clients, creates rendered images of data sets or other data processing results and sends those rendered images and results to the clients for display or further processing. Performing certain image rendering operations on either the server or the client according to which is better suited for the tasks requested by the user at any point in time, and possibly adjusting this division of work dynamically, improves rendering speed and application responsiveness on the clients.

Claims

1. A file server method comprising:

(a) monitoring for requests from a client computer, where the client computer comprises one or more local processing resources and a display device;

(b) receiving one or more requests from the client computer for rendering aspects of one or more images, at least one image of the one or more images including at least a first aspect and a second aspect;

(c) responding to the one or more requests by making a file server buffer available for processing by the client computer;

(d) rendering using a render module the first aspect of the at least one image in response to the one or more requests;

(e) storing the first aspect in the file server buffer;

(f) receiving the second aspect of the at least one image from the client computer, where the second aspect was rendered using the one or more local processing resources; and

(g) storing the second aspect in the file server buffer.

2. The method of claim 1, further comprising includes making the file server buffer available for further processing.

3. The method of claim 2, where further processing includes adding one or more textual elements to the file server buffer.

4. The method of claim 1, where the first aspect comprises one or both an acquired and a synthesized image.

5. The method of claim 1, where the first aspect comprises one or more graphical elements.

6. The method of claim 1, where the second aspect comprises one or more textual elements.

7. The method of claim 1, where the second aspect comprises an overlay aspect.

8. The method of claim 1, where the render module responds to multiple application requests for rendering by allocating rendering and compute tasks between the one or more local processing resources and a file server digital data processor.

9. The method of claim 8, where the render module responds to multiple application requests by messaging the file server digital data processor to render a first graphics aspect of a composite image, effecting rendering of a second graphics aspect of the composite image using the one or more local processing resources and further processing includes adding one or more textual elements to the composite image.

10. The method of claim 8, where the render module combines a composite image including:

(i) a graphics aspect rendered by the file server digital data processor;

(ii) a graphics aspect rendered by the one or more local processing resources; and

(iii) a non graphics element rendered by the one or more local processing resources during further processing.

11. The method of claim 8, where the render module:

(i) pastes into the file server buffer the first aspect of a composite image generated by the file server digital data processor;

(ii) adds to the file server buffer the second aspect of the composite image rendered by the one or more local processing resources; and

(iii) adds to the file server buffer a non graphics aspect of the composite image rendered by the one or more local processing resources during further processing.

12. The method of claim 8, where the one or more local processing resources has a first capability to process tasks and the file server digital data processor has a second capability to process tasks.

13. The method of claim 12, where the render module allocates rendering and compute tasks between the one or more local processing resources and the file server digital data processor based on the first capability and the second capability.

14. The method of claim 8, where allocation of rendering and compute tasks places a first load on the one or more local processing resources and a second load on the file server digital data processor.

15. The method of claim 14, where the render module allocates rendering and compute tasks between the one or more local processing resources and the file server digital data processor based on one or both the first load and the second load.

16. The method of claim 15, where the render module identifies a load on a resource based on one or both any communications from the file server digital data processor and rapidity with which the file server digital data processor responds to messages submitted to it by the client computer.

17. The method of claim 8, further comprising a network connecting the client computer and the file server digital data processor.

18. The method of claim 17, where the render module allocates rendering and compute tasks between the one or more local processing resources and the file server digital data processor based on a throughput of the network.

19. The method of claim 8, where the render module allocates rendering and compute tasks between the one or more local processing resources and the file server digital data processor based on image rendering capabilities of the file server digital data processor.

20. A file server method comprising:

(a) monitoring for requests from a client computer, where the client computer comprises one or more local processing resources and a display device;

(b) receiving one or more requests from the client computer for rendering aspects of one or more images, at least one image of the one or more images including at least a first aspect and a second aspect;

(c) responding to the one or more requests by making a file server buffer available for processing by the client computer;

(d) rendering with a render module the first aspect of the at least one image in response to the one or more requests;

(e) storing the first aspect in the file server buffer;

(f) receiving the second aspect of the at least one image from the client computer, where the second aspect was rendered using the one or more local processing resources;

(g) storing the second aspect in the file server buffer; and

(h) sending the at least one image to the client computer for display on the display device.