User-mode based remote desktop protocol (RDP) encoding architecture

• US 8,180,905 B2
• Filed: 12/09/2008
• Issued: 05/15/2012
• Est. Priority Date: 12/09/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Schmieder, Wilhelm R., Srinivas, Nelamangal Krishnaswamy, Hagiu, Costin, Abdo, Nadim Y., Stoyanov, Vladimir K., Tolba, Ahmed M., Swaminathan, Gautam, Neerudu, Srinivasa Reddy
• Primary Examiner(s): Pwu, Jeffrey
• Assistant Examiner(s): Taha, Shaq
• Application Number:
  US12/331,298
• Publication Number:
  US 20100146127A1
• Time in Patent Office: 1,253 Days
• Field of Search: 709/228
• US Class Current: 709/228
• CPC Class Codes: G06F 9/545 where tasks reside in different layers, e.g. user- and kernel-space

Abstract

Techniques are disclosed for a user-mode based remote desktop protocol (RDP) encoding architecture. A user mode desktop application and user mode virtual channel application run in user-mode session space. Virtual channel data from the virtual channel application is marshaled and sent to a RDP encoder process in user-mode system space. There it is converted to RDP protocol data units (PDU) and sent to a remote client across a communications network. Graphics data from the desktop application is sent to a display driver in kernel-mode session space and then to a graphics reflector that marshals the graphics data and sends it to the RDP encoder for a similar transformation.

Claims

1. A method for encoding graphics data in the user-mode address space of a computer, comprising:

sending graphics data corresponding to a request from a client from a user application executing in user-mode session space to a display driver executing in kernel mode;

converting the graphics data by the display driver into at least one graphics command in a format understandable by a RDP encoder process, the at least one graphics command in a format understandable by the RDP encoder process comprising a protocol data unit (PDU);

marshaling the PDU;

sending the marshaled PDU across a graphics reflector, the graphics reflector comprising shared memory, from the display driver executing in kernel mode to the RDP encoder processes executing in user-mode system space;

unmarshaling the PDU;

compressing, by the RDP encoder process, the PDU; and

sending the compressed PDU from the RDP encoder processes to the client over a communications network.

2. The method of claim 1, wherein the graphics data is sent to the RDP encoder process via a shared memory mechanism.

3. The method of claim 2, further comprising:

transforming a memory representation of the graphics data prior from a first format to a second format to sending it to the RDP encoder process.

4. The method of claim 1, wherein the user application executes in a user-mode session space of a guest operating system (OS) executing on a virtual machine (VM) of the computer, and the display driver executes in a kernel-mode session space of the guest OS.

5. The method of claim 1, wherein the display driver executes in kernel-mode session space.

6. The method of claim 1, wherein the graphics data comprises data in Graphics Device Interface (GDI) format, Desktop Window Manager (DWM) format, or DirectX (DX) format.

7. The method of claim 1, further comprising:

sending virtual channel data corresponding to a request from the client from a virtual channel application executing in user-mode session space to the RDP encoder process, without sending the virtual channel data to a kernel-mode process; and

sending the virtual channel data from the RDP encoder processes to the client over a communications network.

8. The method of claim 7, wherein the virtual channel data is sent to the RDP encoder process via a named pipe.

9. The method of claim 8, further comprising:

transforming a memory representation of the virtual channel data from a first format to a second format prior to sending it to the RDP encoder process.

10. The method of claim 7, wherein the virtual channel data comprises text or image data copied to a clipboard from the user application.

11. The method of claim 1, wherein the RDP encoder process performs at least one from a set on the graphics data, the set comprising:

framing the graphics data, and encrypting the graphics data.

12. The method of claim 1, further comprising:

encoding the graphics data by the display driver into at least one command that can be understood by the RDP encoder process before sending it to the RDP encoder process.

13. A system for encoding graphics data in the user-mode address space of a computer, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

send graphics data corresponding to a request from a client from a user application executing in user-mode session space to a display driver executing in kernel mode;

convert the graphics data by the display driver into at least one graphics command in a format understandable by a RDP encoder process, the at least one graphics command in a format understandable by the RDP encoder process comprising a protocol data unit (PDU);

marshal the PDU;

send the marshaled PDU across a graphics reflector, the graphics reflector comprising shared memory, from the display driver executing in kernel mode to the RDP encoder processes executing in user-mode system space;

unmarshal the PDU;

compress, by the RDP encoder process, the PDU; and

send the compressed PDU from the RDP encoder processes to the client over a communications network.

14. The system of claim 13, wherein the memory further bears processor-executable instructions that, when executed on the processor, causes the system to at least:

receive virtual channel data from a virtual channel application executing in user-mode session space, without the virtual channel data being sent through a kernel-mode process, encode the virtual channel data, and transmit the encoded virtual channel data across the communications network to the client.

15. The system of claim 14, wherein the memory further bears processor-executable instructions that, when executed on the processor, causes the system to at least:

receive the virtual channel data across a virtual channel reflector, the virtual channel reflector comprising a named pipe.

16. The system of claim 13, wherein the memory further bears processor-executable instructions that, when executed on the processor, causes the system to at least:

receive the graphics data across a graphics reflector, the graphics reflector comprising a shared memory.

17. The system of claim 13, wherein the user application executes in a user-mode session space of a guest operating system (OS) executing on a virtual machine (VM) of the computer, and the display driver executes in a kernel-mode session space of the guest OS.

18. The system of claim 13, wherein the encoded graphics data comprises graphics data in a remote desktop protocol (RDP) format.

19. A tangible computer-readable storage medium, bearing computer-readable instructions for encoding graphics data in the user-mode address space of a computer, that upon execution by a computer, cause the computer to perform comprising:

sending graphics data corresponding to a request from a client from a user application executing in user-mode session space to a display driver executing in kernel mode;

converting the graphics data by the display driver into at least one graphics command in a format understandable by a RDP encoder process, the at least one graphics command in a format understandable by the RDP encoder process comprising a protocol data unit (PDU);

marshaling the PDU;

sending the marshaled PDU across a graphics reflector, the graphics reflector comprising shared memory, from the display driver executing in kernel mode to the RDP encoder processes executing in user-mode system space;

unmarshaling the PDU;

compressing, by the RDP encoder process, each PDU;

sending the compressed PDU from the RDP encoder processes to the client over a communications network;

marshaling virtual channel data corresponding to a request from the client from a virtual channel application executing in user-mode session space;

sending the marshaled virtual channel data to the RDP encoder process across a virtual channel reflector, the virtual channel reflector comprising a named pipe;

unmarshaling the virtual channel data;

converting the virtual channel data into at least one virtual channel RDP PDU;

compressing each virtual channel RDP PDU; and

sending each compressed virtual channel RDP PDU from the RDP encoder processes to the client over a communications network.

20. The tangible computer-readable storage medium of claim 19, wherein the user application executes in a user-mode session space of a guest operating system (OS) executing on a virtual machine (VM) of the computer, and the display driver executes in a kernel-mode session space of the guest OS.