Delivery of a user interface using hypertext transfer protocol

• US 9,600,350 B2
• Filed: 06/16/2011
• Issued: 03/21/2017
• Est. Priority Date: 06/16/2011
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Jooste, Sarel Kobus
• Primary Examiner(s): IPAKCHI, MARYAM M
• Application Number:
  US13/162,365
• Publication Number:
  US 20120324358A1
• Time in Patent Office: 2,105 Days
• Field of Search: 715/733, 715/736, 715/740, 715/749, 715/753, 715/741, 345/418, 709/219
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/545 Gui
• CPC Class Codes: G06F 9/54 Interprogram communication

Abstract

A method is provided to remotely access an application hosted by a server and having a corresponding application graphical user interface (GUI) represented on the server, the method comprising: a client automatically sending GUI display update requests to the server throughout a duration of the access, the requests being HTTP requests over corresponding HTTP connections.

Claims

1. A method in a server for providing remote access to an application, the method comprising:

maintaining a mapping between a plurality of (Uniform Resource Locators) URLs and a corresponding plurality of applications;

receiving a URL from a user executing a user interface on a client device remote from the server and remote from the corresponding plurality of applications;

identifying one application of the corresponding plurality of applications, the one application corresponding to the received URL;

remoting the one application over a sequence of Hypertext Transfer Protocol (HTTP) requests and responses to the user via a remoting protocol, by;

receiving a first HTTP request for a graphical user interface (GUI) of the one application;

encoding a region of a server-side frame buffer containing image data corresponding to the GUI of the one application;

sending the encoded region of the server-side frame buffer to the client device in response to the first HTTP request, wherein the client device renders the encoded region and generates a second HTTP request for an update to the server-side frame buffer in response to completing the rendering of the encoded region;

receiving, from the client device, the second HTTP request subsequent to the first HTTP request for an update to the GUI of the one application;

in response to the second HTTP request, computing a difference between the server-side frame buffer and a client-side frame buffer to identify modified image data;

sending to the client device the modified image data in response to the second HTTP request;

receiving, from the client device, an HTTP user interface (UI) interpretation message that indicates how an Internet browser on the client device interprets contents of the client-side frame buffer, wherein the HTTP UI interpretation message is used to transform a screen capture process on the server according to capabilities of the Internet browser of the client; and

adjusting the screen capture process on the server based on the HTTP UI interpretation message received from the client device, wherein the adjusting the screen capture process includes invoking a request to an operating system and graphics driver on the server to change a capture of a bitmap image in the frame buffer of the server according to information contained in the HTTP UI interpretation message.

2. The method of claim 1 wherein the modified image data includes a portion of an entire GUI of the one application that was modified in response to a user input transmitted from the client device to the server.

3. The method of claim 1 wherein location information is included in cascading style sheet (CSS) instructions included in the response to the first HTTP request, the CSS instructions causing the client device to position the encoded region on an output display at a location consistent with the region in the server-side frame buffer that was encoded into the encoded image.

4. The method of claim 1 wherein the encoding the region of the server-side frame buffer including the modified image data comprises encoding the modified image data into a plain-text encoded image.

5. The method of claim 1 wherein the encoding the region of the server-side frame buffer including the modified image data comprises encoding the modified image data into a binary encoded image.

6. The method of claim 1 wherein the encoded region of at least a portion of the graphical user interface (GUI) generated by the one application comprises an entire desktop view generated by an operating system user interface.

7. The method of claim 1, further comprising:

transmitting executable code to the client device upon an initial connection to the client device, the code causing the client to send the sequence of HTTP requests.

8. The method of claim 7 wherein the transmitted executable code executes in conjunction with an Internet browser executed by the client device.

9. The method of claim 1 wherein the remoting protocol does not require a persistent connection or a session.

10. A server for providing remote access to an application, comprising:

a memory;

a processor;

instructions stored in the memory and configured, when executed by the processor, to;

receive a Uniform Resource Locator (URL) from a user executing a user interface on a client device remote from the server;

identify an application of a plurality of applications that corresponds to the received URL; and

a Graphical User Interface (GUI) update response control module stored in the memory and configured, when executed on the processor, to remote the application to the user via a remoting protocol over a sequence of Hypertext Transfer Protocol (HTTP) requests and responses by;

receiving a first HTTP request for a graphical user interface (GUI) of the application;

encoding a region of a server-side frame buffer containing image data corresponding to the GUI of the application;

sending the encoded region of the server-side frame buffer to the client device in response to the first HTTP request, wherein the client device renders the encoded region and generates a second HTTP request for an update to the server-side frame buffer in response to completing the rendering of the encoded region;

receiving a second HTTP request for an update to the GUI of the application, the second HTTP request subsequent to the first HTTP request;

in response to the second HTTP request, computing a difference between the server-side frame buffer and a client-side frame buffer to identify modified image data;

sending to the client device, in response to the second HTTP request, the modified image data;

receiving, from the client device, an HTTP user interface (UI) interpretation message that indicates how an Internet browser on the client device interprets contents of the client-side frame buffer, wherein the HTTP UI interpretation message is used to transform a screen capture process on the server according to capabilities of the Internet browser of the client; and

adjusting the screen capture process on the server based on the HTTP UI interpretation message received from the client device, wherein the adjusting the screen capture process includes invoking a request to an operating system and graphics driver on the server to change a capture of a bitmap image in the frame buffer of the server according to information contained in the HTTP UI interpretation message.

11. The server of claim 10 wherein the remoting protocol does not require a persistent communications connection or a session.

12. The server of claim 10 wherein the encoded region includes one of:

text-coded image data or binary coded image data.

13. A method in a client for providing remote access to an application running on a server, the method comprising:

causing a hypertext transfer protocol (HTTP) client module to transmit a first HTTP request for an update to a frame buffer of the server;

receiving an encoded region of the frame buffer containing image data corresponding to the graphical user interface (GUI) of the application executing on the server;

decoding and rendering the encoded region of the frame buffer on a display coupled to the client device by a rendering module;

monitoring the rendering module to intercept completion status of the rendering module to detect when rendering the encoded region of the frame buffer is complete;

causing the HTTP client module to transmit a second HTTP request for another update to the frame buffer in response to detecting that rendering the encoded region of the frame buffer is complete;

receiving, from the client device, an HTTP user interface (UI) interpretation message that indicates how an Internet browser on the client device interprets contents of the client-side frame buffer, wherein the HTTP UI interpretation message is used to transform a screen capture process on the server according to capabilities of the Internet browser of the client; and

adjusting the screen capture process on the server based on the HTTP UI interpretation message received from the client device, wherein the adjusting the screen capture process includes invoking a request to an operating system and graphics driver on the server to change a capture of a bitmap image in the frame buffer of the server according to information contained in the HTTP UI interpretation message.

14. The method of claim 13, further comprising:

receiving a second encoded region of the frame buffer from the server in response to the second HTTP request; and

combining pixel data of the second encoded region of the frame buffer received from the server with pre-update pixel data received in previous HTTP responses from the server in order to render the update on the display coupled to the client.

15. The method of claim 14, further comprising:

determining a limitation related to a number of image layers that can be composited on the client by combining the pixel data with pre-update pixel data;

transmitting the limitation from the client to the server; and

in response to transmitting the limitation, receiving a new full bitmap from the server before the client reaches the number of image layers specified by the limitation.

16. The method of claim 13, further comprising:

receiving a second encoded region of the frame buffer from the server in response to the second HTTP request; and

causing the HTTP client module to transmit a third HTTP request to obtain a location of the second encoded region in the frame buffer prior to rendering the second encoded region, such that content of the second encoded region and the location of the content are received in separate HTTP responses.

17. The method of claim 13, further comprising:

reducing a sampling rate of mouse events of a browser on the client, wherein the sampling rate is determined based on a speed at which the server is capable of accepting events.

18. The method of claim 17, wherein reducing the sampling rate further comprises:

transmitting one or more HTTP UI input device messages to the server indicating mouse input events;

receiving HTTP acknowledgements from the server in response to the one or more HTTP UI input device messages; and

filtering, by the client, one or more mouse events from being reported to the server based on a delay in receiving the HTTP acknowledgments from the server.