Multi-user terminal services accelerator

• US 20070097130A1
• Filed: 11/01/2005
• Published: 05/03/2007
• Est. Priority Date: 11/01/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Digital Display Innovations, LLC
• Inventors: Margulis, Neal
• Granted Patent: US 7,899,864 B2
• Time in Patent Office: Days
• US Class Current: 345/501
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G09G 2320/0252 Improving the response speed
• CPC Class Codes: G09G 2370/025 LAN communication management
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

A multi-user host computer system comprises a host computer that processes applications and the desktop environments for multiple remote terminals. The host computer includes a terminal services offload processor to supplement the processing of the host CPU. For each remote terminal, the terminal services offload processor may improve the video and graphics performance and allow the multi-user host computer system to more efficiently support multiple users. The host computer may include a graphics processor that manages a virtual display for each remote terminal and provides selective updates of sub frame data. Where appropriate, the sub frame data is encoded and transmitted over the network to the remote terminals. Video data streams are also optimized by the terminal services offload processor and optimized for the intended remote terminals and their respective network connections. The terminal services offload processor may be implemented in programmable hardware and may be a separate subsystem or may also be used to perform other offload tasks.

Claims

1. A host computer system capable of supporting multiple remote terminals, comprising:

a graphics and display subsystem having

a display memory which can store display frames for multiple terminals;

one or more drawing engines for generating display frames which may each correspond to a display frame at a remote terminal; and

means for tracking modified sub frames of display frames and for performing selective updates from the display memory based on this tracking;

means for connecting said graphics and display subsystem to a host CPU in said host computer system; and

means for managing the selective updates from the display memory so that only the selective updates will be transferred via a network subsystem to corresponding ones of said remote terminals.

2. The system of claim 1 wherein said graphics and display subsystem utilizes display memory to manage status bits to track which sub frames have been modified and where said selective updates check these status bits before transferring the selective update output.

3. The system of claim 1 wherein said tracked sub frames that require selective updating are first encoded to reduce bandwidth requirements prior to being transferred via said network subsystem.

4. The system of claim 1 wherein said graphics and display subsystem outputs said selective updates over a private connection to an encoding subsystem that encodes said selective updates prior to their transfer to said network subsystem.

5. The system of claim 4 wherein, beyond the tracking performed by said graphics and display subsystem, said encoding subsystem performs additional tracking to determine if said selective updates are required to be transferred to said remote terminal.

6. The system of claim 4 wherein said encoding subsystem receives graphics commands either from said graphics and display subsystem or from said host CPU and processes said graphics commands for transfer via said network subsystem to said corresponding remote terminal.

7. The system of claim 1 wherein said sub frames are fixed size tiles and said tracking is performed on said fixed size tiles.

8. The system of claim 7 wherein said graphics and display subsystem generates signatures on said tiles and manages said signatures of subsequent tiles to determine if said tiles are to be selectively updated and sent to said encoding subsystem.

9. A host computer system capable of supporting multiple remote terminals, comprising:

a host CPU connected via a system bus to other subsystems within said host computer system;

an offload subsystem for managing said remote terminals, having

means to intercept graphics commands or video data on said bus from being processed by said host CPU; and

means to encode, transcode or otherwise process said intercepted graphics commands or video data; and

means to manage said intercepted and processed graphics commands or video data for transfer via a network subsystem to corresponding ones of said remote terminals.

10. The system of claim 9 wherein said graphics commands are intercepted and blocks of data associated with said graphics commands are encoded by said offload subsystem to reduce bandwidth required by said network transfer.

11. The system of claim 9 wherein said video data is intercepted by said offload subsystem and said video data is processed to match the network performance and decode capabilities of individual ones of said remote terminals and wherein said processing may include changing the bit rate (Transrating), frame rate, resolution or the coding algorithm (Transcoding).

12. The system of claim 9 wherein said means to intercept includes a tracking software layer running on said host CPU.

13. The system of claim 9 wherein said offload subsystem includes a means for connecting to one or more display output paths from a graphics and display controller.

14. The system of claim 13 where said graphics and display controller produces output for one local display and remote management can be performed via said remote terminal.

15. The system of claim 13 where said offload subsystem performs encoding of said host system graphics and display output prior to network transmission to one of said remote terminals.

16. The system of claim 13 wherein said graphics and display controller supports multiple RTs and said connection provides said offload system with selective sub frame updates corresponding to different sub frames in multiple remote terminals.

17. A method for operating a multi-user host system having multiple remote terminals, comprising the steps of:

using a host computer that includes software, a main CPU and an offload engine to assist the CPU in processing graphics commands or video data;

processing said video data by changing bit rate, changing resolution, changing frame rate or changing encoding algorithm;

processing said graphics commands by encapsulating and encoding data associated with the graphics commands;

determining which of said remote terminals is the destination for said processed graphics commands or said processed video data; and

propagating said processed graphics commands or processed video data according to network protocol techniques through a network interface.

18. The method of claim 17 wherein said multi-user host system includes a local graphics

processor with frame memory corresponding to one or more of said remote terminals and wherein said local graphics processor performs the steps of:

rendering the graphics commands into display frames;

determining on a sub frame basis which sub frames need to be selectively updated on each of said remote terminals; and

transmitting said selective updates to said offload engine.

19. The method of claim 18 said offload engine performs encoding of said selective update sub frames.

20. The method of claim 17 wherein said offload engine is also used to offload one or more other processing tasks from said main CPU.