Multi-user terminal services accelerator

• US 7,899,864 B2
• Filed: 11/01/2005
• Issued: 03/01/2011
• Est. Priority Date: 11/01/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Margulis, Neal D.
• Primary Examiner(s): Bruckart; Benjamin R
• Assistant Examiner(s): Waqas; Saad A
• Application Number:
  US11/264,269
• Publication Number:
  US 20070097130A1
• Time in Patent Office: 1,946 Days
• Field of Search: 709/204, 709/220, 709/224, 709/208, 709/211, 709/223, 345/504, 345/501, 345/502, 345/503, 345/520, 345/530, 340/540
• US Class Current: 709/204
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G09G 2320/0252 Improving the response speed
• CPC Class Codes: G09G 2370/025 LAN communication management
• CPC Class Codes: G09G 5/363 Graphics controllers

Abstract

A multi-user host computer system comprises a host computer that processes applications and the desktop environments for multiple remote terminals. The host computer includes a terminal services offload processor to supplement the processing of the host CPU. For each remote terminal, the terminal services offload processor may improve the video and graphics performance and allow the multi-user host computer system to more efficiently support multiple users. The host computer may include a graphics processor that manages a virtual display for each remote terminal and provides selective updates of sub frame data. Where appropriate, the sub frame data is encoded and transmitted over the network to the remote terminals. Video data streams are also optimized by the terminal services offload processor and optimized for the intended remote terminals and their respective network connections. The terminal services offload processor may be implemented in programmable hardware and may be a separate subsystem or may also be used to perform other offload tasks.

Claims

1. A host computer system configured to support multiple remote terminals, comprising:

a graphics and display subsystem having

a display memory which can store display frames for multiple terminals;

one or more drawing engines for generating display frames corresponding to display frames at a remote terminal, the display frames comprising sub frames wherein a size for each sub frame is configurable during run-time by the host computer system as a multiple of a minimum pixel block size of an algorithm for encoding the sub frames executing in the graphics and display subsystem; and

means for tracking modified sub frames of the display frames and for performing selective updates based on this tracking, wherein said tracking comprises generating signatures and status bits identifying the modified sub frames, reducing the sub frames into smaller sub frames based on the signatures, determining a type of data contained in the modified sub frames, and selecting the algorithm for encoding the sub frames based on the type;

means for communicatively coupling said graphics and display subsystem to a host CPU in said host computer system and

means for managing the selective updates so that only the selective updates will be transferred via a network subsystem to corresponding ones of said remote terminals.

2. The system of claim 1 wherein modified sub frames are encoded to reduce bandwidth requirements prior to being transferred via said network subsystem.

3. The system of claim 1 wherein said graphics and display subsystem is configured to output said selective updates over a private connection to an encoding subsystem that encodes said selective updates prior to their transfer to said network subsystem.

4. The system of claim 3 wherein said encoding subsystem is further configured to perform additional tracking to determine if said selective updates are required to be transferred to said remote terminal.

5. The system of claim 3 wherein said encoding subsystem is configured to receive graphics commands either from said graphics and display subsystem or from said host CPU and process said graphics commands for transfer via said network subsystem to said corresponding remote terminal.

6. The system of claim 1 wherein said sub frames are fixed size tiles and said tracking is performed on said fixed size tiles.

7. A host computer system configured to support multiple remote terminals, comprising:

a host CPU connected via a system bus to other subsystems within said host computer system;

an offload subsystem for managing said remote terminals configured to;

intercept graphics commands or video data on said bus prior to being processed by said host CPU and prior to sending graphics commands or video data to the remote terminals such that the host CPU does not process the graphics commands or video data; and

process said intercepted graphics commands or video data as a function of video decoding capabilities at said multiple remote terminals and support remote terminals of different display sizes, further comprising at least one of transcoding, transrating, changing a frame rate, changing a display format, and changing a resolution; and

a management subsystem configured to manage said intercepted and processed graphics commands or video data for transfer via a network subsystem to corresponding ones of said remote terminals; and

a graphics and display subsystem configured to;

generate display frames corresponding to display frames at the remote terminals, the display frames comprising sub frames wherein a size for each sub frame is configurable during runtime by the host computer system as a multiple of a minimum pixel block size of an algorithm for encoding the sub frames;

track modified sub frames by generating signatures and status bits identifying the modified sub frames;

reduce the sub frames into smaller sub frames based on the signatures; and

perform selective updates based on this tracking, further comprising determining a type of data contained in the modified sub frames and selecting an encoding algorithm based on the type.

8. The system of claim 7 wherein said graphics commands are intercepted and blocks of data associated with said graphics commands are encoded by said offload subsystem to reduce bandwidth required by said network transfer.

9. The system of claim 7 wherein said video data is intercepted by said offload subsystem and said video data is processed to match the network performance and decode capabilities of individual ones of said remote terminals and wherein said processing may include changing the bit rate (Transrating), frame rate, resolution or the coding algorithm (Transcoding).

10. The system of claim 7 wherein said further comprises a tracking software layer running on said host CPU.

11. The system of claim 7 wherein said offload subsystem includes a means for connecting to one or more display output paths from a graphics and display controller.

12. The system of claim 11 where said graphics and display controller is configured to produce output for one local display and remote management can be performed via said remote terminal.

13. The system of claim 11 where said offload subsystem is configured to perform encoding of said host system graphics and display output prior to network transmission to one of said remote terminals.

14. The system of claim 11 wherein said graphics and display controller supports multiple remote terminals and said connection provides said offload system with selective sub frame updates corresponding to different sub frames in multiple remote terminals.

15. In a system comprising a host computer including software, a main CPU, a graphics and display subsystem, and an offload engine, a method for operating a multi-user host system having multiple remote terminals, comprising:

using said offload engine to assist the main CPU in processing graphics commands and video data, wherein said processing said video data comprises at least one of changing bit rate, changing resolution, changing frame rate and changing encoding algorithm and said processing said graphics commands comprises encapsulating and encoding data associated with the graphics commands, said encoding executed such that extraneous video artifacts are substantially minimized regardless of an alignment of a current display window;

using said graphics and display subsystem to;

generate display frames corresponding to display frames at the remote terminals, the display frames comprising sub frames wherein a size for each sub frame is configurable during runtime by the host computer system as a multiple of a minimum pixel block size of an algorithm for encoding the sub frames;

track modified sub frames by generating signatures and status bits identifying the modified sub frames;

reduce the sub frames into smaller sub frames based on the signatures; and

perform selective updates based on this tracking, further comprising determining a type of data contained in the modified sub frames and selecting an encoding algorithm based on the type;

determining which of said remote terminals is the destination for said processed graphics commands and said processed video data; and

propagating said processed graphics commands and processed video data according to network protocol techniques through a network interface.

16. The method of claim 15 wherein said offload engine is configured to perform encoding of said selective update sub frames.

17. The method of claim 15 wherein said offload engine is also used to offload one or more other processing tasks from said main CPU.