Client-server visualization system with hybrid data processing

• US 10,043,482 B2
• Filed: 03/06/2017
• Issued: 08/07/2018
• Est. Priority Date: 11/23/2007
• Status: Active Grant

• Current Assignee: PME IP Pty., Ltd.
• Original Assignee: PME IP Pty., Ltd.
• Inventors: Westerhoff, Malte, Stalling, Detlev
• Primary Examiner(s): MADAMBA, GLENFORD J
• Application Number:
  US15/450,888
• Publication Number:
  US 20170178593A1
• Time in Patent Office: 519 Days
• Field of Search: 709203
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G09G 2360/12 Frame memory handling
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: G09G 5/395 Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen G09G5/399 takes precedence

Abstract

The invention comprises a system of client-server visualization with hybrid data processing, having a server digital data processor, that allows for server side rendering and processing image data, and client digital data processors simultaneously connected to the server, which receives messages from the clients, creates rendered images of data sets or other data processing results and sends those rendered images and results to the clients for display or further processing. Performing certain image rendering operations on either the server or the client according to which is better suited for the tasks requested by the user at any point in time, and possibly adjusting this division of work dynamically, improves rendering speed and application responsiveness on the clients.

Claims

1. A method comprising:

(a) executing an application on a client computer comprising one or more local processing resources, a main memory and a display device, where the application resides on the main memory, where the application generates one or more requests for rendering aspects of one or more images, at least one image of the one or more images including at least a first aspect and a second aspect;

(b) contacting a server comprising a server digital data processor, a server graphics processing unit and a server buffer, where the server buffer can be accessed by one or both by the server digital data processor and the client computer, where the server in response to a request from the application renders the first aspect of the at least one image with the server graphics processing unit and stores the first aspect in the server buffer;

rendering the second aspect of the at least one image using the one or more local processing resources;

(c) sending the second aspect to the server buffer; and

(d) making the server buffer available for further processing by the client computer.

2. The method of claim 1, where the further processing includes adding one or more textual elements to the server buffer.

3. The method of claim 1, where the further processing includes display by the display device of the server buffer.

4. The method of claim 1, where the first aspect comprises one or both an acquired and a synthesized image.

5. The method of claim 1, where the first aspect comprises one or more graphical elements.

6. The method of claim 1, where the second aspect comprises one or more textual elements.

7. The method of claim 1, where the second aspect comprises an overlay aspect.

8. The method of claim 1, where the one or more local processing resources includes a render module that responds to multiple application requests for rendering by allocating rendering and compute tasks between the one or more local processing resources and the server digital data processor.

9. The method of claim 8, where the render module responds to multiple application requests by messaging the server digital data processor to render a first graphics aspect of a composite image, effecting rendering of a second graphics aspect of the composite image using the one or more local processing resources and further processing includes adding one or more textual elements to the composite image.

10. The method of claim 8, where the render module combines a composite image including:

(i) a graphics aspect rendered by the server digital data processor;

(ii) a graphics aspect rendered by the one or more local processing resources; and

(iii) a non graphics element rendered by the one or more local processing resources during further processing.

11. The method of claim 8, where the render module:

(i) pastes into the server buffer the first aspect of a composite image generated by the server digital data processor;

(ii) adds to the server buffer the second aspect of the composite image rendered by the one or more local processing resources; and

(iii) adds to the server buffer a non graphics aspect of the composite image rendered by the one or more local processing resources during further processing.

12. The method of claim 8, where the one or more local processing resources has a first capability to process tasks and the server digital data processor has a second capability to process tasks.

13. The method of claim 12, where the render module allocates rendering and compute tasks between the one or more local processing resources and the server digital data processor based on the first capability and the second capability.

14. The method of claim 8, where allocation of rendering and compute tasks places a first load on the one or more local processing resources and a second load on the server digital data processor.

15. The method of claim 14, where the render module allocates rendering and compute tasks between the one or more local processing resources and the server digital data processor based on one or both the first load and the second load.

16. The method of claim 15, where the render module discerns a load on a resource based on one or both any communications from the server digital data processor and rapidity with which the server digital data processor responds to messages submitted to it by the client computer.

17. The method of claim 8, further comprising a network connecting the client computer and the server digital data processor.

18. The method of claim 17, where the render module allocates rendering and compute tasks between the one or more local processing resources and the server digital data processor based on a throughput of the network.

19. The method of claim 8, where the render module allocates rendering and compute tasks between the one or more local processing resources and the server digital data processor based on image rendering capabilities of the server digital data processor.

20. The method of claim 8, where the render module allocates rendering and compute tasks between the one or more local processing resources and the server digital data processor based on one or both processing state of the application and location of data being rendered.