Pipeline for network based server-side 3D image rendering

• US 8,386,560 B2
• Filed: 09/08/2008
• Issued: 02/26/2013
• Est. Priority Date: 09/08/2008
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ma, Ke, Fang, Lijiang, Li, Chao
• Primary Examiner(s): Winder, Patrice
• Assistant Examiner(s): Golabbakhsh, Ebrahim
• Application Number:
  US12/206,151
• Publication Number:
  US 20100063992A1
• Time in Patent Office: 1,632 Days
• Field of Search: 709/203, 709/232, 709/217
• US Class Current: 709/203
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2210/08 Bandwidth reduction
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection

Abstract

The claimed subject matter provides a system and/or a method that facilitates rendering a portion of an image from a server on a client. A portion of an image can be hosted by a server. A client can utilize a device to interact with the portion of an image via a network. The client can established two or more TCP/IP connections with the server in order to interact with the portion of the image. A pipe component can enforce an isolated pipeline and a processing thread for each of a user manipulation of the portion of the image, a transmission of a user request to manipulate the portion of the image via the network, a rendering of the portion of the image on the server, a compression of the portion of the image on the server, a transmission of the compressed portion of the image via the network, a decompression of the portion of the image on the client, and a display of the portion of the image on the client.

Claims

1. A system, comprising:

a processor configured to execute computer-executable components; and

memory storing computer-executable components including;

a client configured to asynchronously communicate with a server for interacting with image data hosted by the server; and

a pipe component configured to, in separate threads corresponding to respective pipeline stages, process an interaction with the image data, process a transmission of requests corresponding to the interaction, process image data received in responses from the server, and process a display of image data;

wherein the requests corresponding to the interaction are queued sequentially and communicated to the server periodically based on a combine interval that specifies an interval between sending consecutive requests, each request includes information representing a manipulation mode and a rendering quality, queued requests are combined if the queued requests are generated within the combine interval and if the queued requests have the same manipulation mode and the same rendering quality, and the combine interval is adjusted by;

calculating a request interval between consecutive requests and a response interval between responses to the consecutive requests,

increasing the combine interval with an increment step if the request interval is less than the response interval,

decreasing the combine interval with a minimum of a decrement step and a difference between the request interval and the response interval if the request interval is not less than the response interval,

setting the combine interval to a maximum value if the combine interval is greater than a maximum value, and

setting the combine interval to a minimum value if the combine interval is less than the minimum value.

2. The system of claim 1, wherein the interaction with the image data includes at least one of a rotation, a zoom, a pan, a window/level, or an edit.

3. The system of claim 1, wherein the image data includes at least one of 2-dimensional (2D) image data, 3-dimensional (3D) image data, medical image data, virtual reality system image data, video game image data, 3D medical image reconstruction data, or 3D medical image Multi-Plane Reconstruction in Picture Archiving and Communication System (PACS) data.

4. The system of claim 1, wherein each request further includes at least one of:

a timestamp and information representing a transformation that can be performed on the image data.

5. The system of claim 1, wherein the server employs a plurality of render engines for the consecutive requests.

6. The system of claim 1, wherein the pipe component is configured to process at least two of the respective pipeline stages substantially simultaneously.

7. The system of claim 1, wherein the server is configured to preprocess at least a portion of the image data into a low resolution volume texture and store the low resolution volume texture on a data store.

8. The system of claim 1, wherein the server is configured to render at least a portion of the image data in a first thread and compress the rendered portion of the image data in a second thread, substantially simultaneously.

9. The system of claim 1, further comprising a request proxy configured to queue the requests.

10. The system of claim 1, wherein the computer-executable components further include a protocol component configured to utilize a protocol to communicate with the server by receiving an image property by an image identification (ID) and rendering requests to the server with the image property and information corresponding to a manipulation.

11. The system of claim 1, wherein the pipe component is configured to support cancellation of queued requests in response to a new request corresponding to user input indicating an end of manipulation of the image data.

12. The system of claim 1, wherein the requests are sent through an asynchronous web service call to the server.

13. The system of claim 1, wherein the consecutive requests have the same manipulation mode and different rendering qualities.

14. The system of claim 1, wherein the consecutive requests have the same rendering quality and different manipulation modes.

15. A computer-readable storage device storing instructions that, when executed by a computing device, cause the computing device to perform a method comprising:

asynchronously communicating with a server for interacting with image data hosted by the server; and

processing in separate threads corresponding to respective pipeline stages;

an interaction with the image data, a transmission of requests corresponding to the interaction, image data received in responses from the server, and a display of image data;

wherein the requests corresponding to the interaction are queued sequentially and communicated to the server periodically based on a combine interval that specifies an interval between sending consecutive requests, each request includes information representing a manipulation mode and a rendering quality, queued requests are combined if the queued requests are generated within the combine interval and if the queued requests have the same manipulation mode and the same rendering quality, and the combine interval is adjusted by;

calculating a request interval between consecutive requests and a response interval between responses to the consecutive requests,

increasing the combine interval with an increment step if the request interval is less than the response interval,

decreasing the combine interval with a minimum of a decrement step and a difference between the request interval and the response interval if the request interval is not less than the response interval,

setting the combine interval to a maximum value if the combine interval is greater than a maximum value, and

setting the combine interval to a minimum value if the combine interval is less than the minimum value.

16. The computer-readable storage device of claim 15, the method further comprising:

canceling queued requests in response to a new request corresponding to user input indicating an end of manipulation of the image data; and

transmitting the new quest to the server.

17. The computer-readable storage device of claim 15, wherein:

the image data includes at least one of;

2-dimensional (2D) image data, 3-dimensional (3D) image data, medical image data, virtual reality system image data, video game image data, 3D medical image reconstruction data, or 3D medical image Multi-Plane Reconstruction in Picture Archiving and Communication System (PACS) data; and

the interaction with the image data includes at least one of;

a rotation, a zoom, a pan, a window/level, or an edit.

18. A computer-implemented method comprising:

asynchronously communicating with a server for interacting with image data hosted by the server; and

processing in separate threads corresponding to respective pipeline stages;

an interaction with the image data, a transmission of requests corresponding to the interaction, image data received in responses from the server, and a display of image data;

wherein the requests corresponding to the interaction are queued sequentially and communicated to the server periodically based on a combine interval that specifies an interval between sending consecutive requests, each request includes information representing a manipulation mode and a rendering quality, queued requests are combined if the queued requests are generated within the combine interval and if the queued requests have the same manipulation mode and the same rendering quality, and the combine interval is adjusted by;

calculating a request interval between consecutive requests and a response interval between responses to the consecutive requests,

increasing the combine interval with an increment step if the request interval is less than the response interval,

decreasing the combine interval with a minimum of a decrement step and a difference between the request interval and the response interval if the request interval is not less than the response interval,

setting the combine interval to a maximum value if the combine interval is greater than a maximum value, and

setting the combine interval to a minimum value if the combine interval is less than the minimum value.

19. The method of claim 18, further comprising:

canceling queued requests in response to a new request corresponding to user input indicating an end of manipulation of the image data; and

transmitting the new quest to the server.

20. The method of claim 18, wherein:

the image data includes at least one of;

2-dimensional (2D) image data, 3-dimensional (3D) image data, medical image data, virtual reality system image data, video game image data, 3D medical image reconstruction data, or 3D medical image Multi-Plane Reconstruction in Picture Archiving and Communication System (PACS) data; and

the interaction with the image data includes at least one of;

a rotation, a zoom, a pan, a window/level, or an edit.