Multi-user display proxy server

• US 8,112,513 B2
• Filed: 11/30/2005
• Issued: 02/07/2012
• Est. Priority Date: 11/30/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Margulis, Neal D.
• Primary Examiner(s): Pwu, Jeffrey
• Assistant Examiner(s): Waqas, Saad A
• Application Number:
  US11/289,983
• Publication Number:
  US 20070124474A1
• Time in Patent Office: 2,260 Days
• Field of Search: 711/E12.025, 711/E12.031, 711/E12.026, 711/152, 711/E12.032, 709/224, 709/226, 709/214, 714/E11.072, 713/1, 713/2, 713/100, 713/310, 713/300, 726/23, 710/1
• US Class Current: 709/224
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

A multi-user host computer system comprises processor blades combined with terminal services blades to provide acceleration and proxy server functions for supporting a variety of remote terminals. For each remote terminal, the terminal services blade and proxy server functions may improve the video and graphics performance. This allows the multi-user host computer system to more efficiently support multiple users. The terminal services blade may include a graphics processor that manages a virtual display for each remote terminal and provides selective updates of sub frame data. Where appropriate, the sub frame data is encoded and transmitted over the network to the remote terminals. The terminal services processor also offloads and optimizes video data streams for the intended remote terminals and their respective network connections. Processor blades may include a baseboard management controller that utilizes advanced features for supporting remote KVM administration.

Claims

1. A display proxy server system configured to support multiple remote terminals, comprising:

a graphics and display subsystem having a display memory which can store display frames for multiple terminals, the graphics and display subsystem being configured to;

generate display frames corresponding to display frames at one or more remote terminals, the display frames comprising sub frames wherein a size for each sub frame is dynamically configurable during run-time by the display proxy server system as a multiple of a minimum pixel block size of an algorithm selected to encode the sub frames, the algorithm selected during runtime based on characteristic indicators of content within the sub frames;

track modified sub frames of the display frames and perform selective updates from the display memory based on the tracking, the tracking comprising generating signatures corresponding to the modified sub frames, performing signature checks for the modified sub frames, updating status bits to indicate that a sub frame requires selective updating, determining a type of data contained in the modified sub frames, and selecting an encoding algorithm based on the type; and

respond to commands and requests from virtual machines via a virtualization abstraction layer including graphics drivers corresponding to operating systems executing on the virtual machines;

the display proxy server system being configured to;

communicatively couple said display proxy server system to one or more host CPUs;

communicate with one or more host CPUs running one or more virtual machines configured to request graphics operations, locally manage the multiple virtual machine graphics requests, utilize the virtual machine graphics requests to produce virtual displays for requesting virtual machines, and transmit encoded updates of said virtual displays over a network to corresponding ones of said remote terminals;

communicate with said one or more host CPUs in one of;

an operating system virtual machine mode to perform operating system functions of said graphics processing operations, and

a protected management virtual machine mode which is isolated from said operating system virtual machine mode, said protected management virtual machine mode being configured to manage some of the update functions for the remote terminals; and

manage the selective updates from the display memory so that only the selective updates will be transferred via a network subsystem to corresponding ones of said remote terminals.

2. The system of claim 1 wherein said remote terminals are configured to execute an Internet browser and request web updates via said display proxy server system which is configured to perform a proxy function to request and receive the web updates from a web server and translate said web updates into encoded partial frame updates and provide said encoded partial frame updates to the Internet browser.

3. The system of claim 1 wherein said remote terminals are configured to execute an Internet browser and perform content based frame requests to said display proxy server system and said display proxy server system is configured to composite, encode and supply partial frame updates based on what portions of the frame have changed using data already cached at said display proxy server system.

4. The system of claim 1 wherein said display proxy server system is implemented in a blade system and said communicatively couple is provided over a system backplane.

5. The system of claim 1 wherein said display proxy server system is implemented as an appliance and is attached by both a network connection and a display output path to a server system.

6. The system of claim 1 wherein said display proxy server system is configured to communicate with a host CPU running an application, respond to the application as if it were a client, and perform selective display updates to the remote terminal.

7. The system of claim 1 wherein said display proxy server system provides a DNS lookup function for said multiple remote terminals when performing web accesses.

8. A display management system configured to support one or more remote terminals, comprising:

a graphics controller and display subsystem configured to;

perform graphics processor operations in a display memory, at the request of a host CPU, to generate display frame data;

track modified sub frames of display frames, perform selective updates based on the modified sub frames, generate signatures corresponding to the modified sub frames, perform signature checks for the modified sub frames, updating status bits to indicate that a sub frame requires selective updating, determining a type of data contained in the modified sub frames, and selecting an encoding algorithm based on the type, the size for the sub frames being dynamically configurable during run-time by the display management as a multiple of a minimum pixel block size of an algorithm selected to encode the sub frames, the algorithm selected during runtime based on characteristic indicators of content within the sub frames;

encode said selective updates such that extraneous video artifacts due to alignment of a display window and surrounding pixels are substantially minimized; and

manage the selective updates from said display memory so that only the selective updates will be transferred via a network subsystem to corresponding ones of said one or more remote terminals;

the display management system being further configured to;

communicate with one or more host CPUs running one or more virtual machines each requesting graphics operations, locally manage the multiple virtual machine graphics requests and utilize the virtual machine graphics requests to produce virtual displays for requesting virtual machines, and transmit encoded updates of said virtual displays over a network to corresponding ones of said remote terminals; and

communicate with said one or more host CPUs in one of;

an operating system virtual machine mode to perform operating system functions of said graphics operations, and

in a protected management virtual machine mode which is isolated from said operating system virtual machine mode, said protected management virtual machine mode being configured to manage some of the update functions for the remote terminals.

9. The display management system of claim 8 wherein said selective updates are based on subframes of fixed tile sizes.

10. The display management system of claim 8 wherein said display management system uses an S-Buffer to manage selective updates.

11. A blade based multi-user system configured to support multiple remote terminals, comprising:

one or more processor blades connected to a backplane;

one or more terminal services accelerator blades including a graphics and display subsystem having a display memory which can store display frames for multiple terminals, the graphics and display subsystem being configured to;

generate display frames corresponding to display frames at a remote terminal, the display frames comprising sub frames, the size for the sub frames being dynamically configurable during run-time by the graphics and display subsystem as a multiple of a minimum pixel block size of an algorithm selected to encode the sub frames;

track modified sub frames;

perform selective updates from the display memory based on the tracking, wherein the tracking comprises generating signatures corresponding to the modified sub frames, performing signature checks for the modified sub frames, and updating status bits to indicate that a sub frame requires selective updating;

identify a type of data contained in the modified sub frames and determine an encoding technique as a function of characteristic indicators for said type; and

respond to commands and requests from one or more virtual machines using a virtualization abstraction layer including graphics drivers corresponding to operating systems executing on the one or more virtual machines; and

an offload subsystem configured to;

to intercept graphics commands or video data prior to being processed by said one or more processor blades and prior to sending graphics commands or video data to the remote terminals such that the one or more processor blades do not process the graphics commands or video data; and

process said intercepted graphics commands or video data, said processing process performed as a function of network conditions, display characteristics, and video decoding capabilities at said multiple remote terminals, said comprising at least one of transcoding from one encoded format to another encoded format, transrating from one bit rate to another bit rate, changing a frame rate, changing a display format, and changing a resolution;

communicatively couple said processor blades to one or more terminal services accelerator blades;

the blade based multi-user system being configured to;

manage the selective updates from the display memory so that only the selective updates will be transferred via a network subsystem to corresponding ones of said remote terminals; and

communicate with one or more host CPUs running one or more virtual machines each requesting graphics operations, locally manage the multiple virtual machine graphics requests and utilize the virtual machine graphics requests to produce virtual displays for requesting virtual machines, and transmit encoded updates of said virtual displays over a network to corresponding ones of the remote terminals; and

communicate with said one or more host CPUs in one of;

an operating system virtual machine mode to perform operating system functions of said graphics operations, and

in a protected management virtual machine mode which is isolated from said operating system virtual machine mode, said protected management virtual machine mode being configured to manage some of the update functions for the remote terminals.

12. The system of claim 11 wherein said one or more terminal services accelerator blade is configured to communicate with a host CPU which is running application software and which is configured to:

utilize a multi-user remote client protocol that virtualizes the graphics and display subsystem;

perform virtual graphics functions as the remote user client; and

perform the selective display updates from the virtual display subsystem to the remote terminal.

13. The system of claim 11 wherein said terminal services blade is configured to communicate with one or more CPUs running one or more virtual machines each of which requests graphics operations as if the terminal services blade included a respective graphics controller dedicated to each virtual machine.

14. The system of claim 11 wherein said terminal services blade utilizes said graphics and display subsystem to locally manage virtual machine graphics requests from multiple independent virtual machine mode CPUs to produce virtual displays for virtual machines and then transmits encoded updates of said virtual displays over a network to remote terminals, and said multiple independent virtual machine mode CPUs are not aware of each other.

15. The system of claim 11 wherein said processor blades operate as a web server.

16. The system of claim 11 wherein said blade based multi-user system provides an Internet services management functions to multiple remote terminals which access multiple web servers.