Methods and systems for remoting three dimensional graphical data

• US 8,638,336 B2
• Filed: 10/20/2008
• Issued: 01/28/2014
• Est. Priority Date: 10/20/2007
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Bullard, Justin, Peart, Franklyn, Treder, Terry, Thorslund, Derek, Anderson, Brad
• Primary Examiner(s): Welch, David T
• Application Number:
  US12/254,810
• Publication Number:
  US 20090102838A1
• Time in Patent Office: 1,926 Days
• Field of Search: 345/501, 345/522, 718/104, 718/105
• US Class Current: 345/522
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2210/08 Bandwidth reduction

Abstract

Systems and methods for remotely displaying three dimensional graphical data, include a local computing machine that communicates with a remote computing machine via a connection. An application executing on the local computing machine, generates three dimensional graphics commands which are intercepted and transferred to either the remote computing machine or a computing element on the local computing machine. Determining where to transfer the three dimensional graphics commands requires: an analysis of the application to obtain application specific characteristics; an analysis of the remote computing machine to obtain the capabilities of the remote computing machine; an analysis of the local computing machine to obtain the capabilities of the local computing machine; and an analysis of the connection to determine characteristics of the connection. A determination is made in response to obtaining the application characteristics, the remote computing machine capabilities, the local computing machine capabilities and the connection characteristics.

Claims

1. A method for remoting three dimensional graphical data, the method comprising:

intercepting, by a local computing machine in a distributed computing system, three dimensional graphics commands generated in a session by an application executing on the local computing machine prior to the processing of the three dimensional graphics commands by a display driver executing on the local computing machine;

obtaining application specific characteristics of the application by analyzing the application;

obtaining capabilities of a remote computing machine in the distributed computing system by analyzing the remote computing machine;

obtaining capabilities of the local computing machine by analyzing the local computing machine;

obtaining characteristics of a connection installed between the local computing machine and the remote computing machine by analyzing the connection;

determining, based on the application characteristics, the remote computing machine capabilities, the local computing machine capabilities, and the connection characteristics, to which one of the remote computing machine and a computing element on the local machine that receives as input three dimensional graphics commands to transfer the intercepted three dimensional graphics commands;

transferring the three dimensional graphics commands to one of the remote computing machine and the computing element on the local computing machine responsive to determining where to transfer the intercepted three dimensional graphics commands;

waiting a predetermined period of time within the session;

re-obtaining each of the application specific characteristics, the remote computing machine capabilities, the local computing machine capabilities, and the connection characteristics; and

re-determining, based on the application characteristics, the remote computing machine capabilities, the local computing machine capabilities, and the connection characteristics, to which of the remote computing machine and the computing element on the local machine to transfer the intercepted three dimensional graphics commands; and

transferring the three dimensional graphics commands to one of the remote computing machine and the computing element on the local computing machine responsive to re-determining where to transfer the intercepted three dimensional graphics commands.

2. The method of claim 1, wherein intercepting three dimensional graphics commands further comprises intercepting primitive three dimensional graphics commands.

3. The method of claim 1, wherein intercepting three dimensional graphics commands further comprises intercepting high level three dimensional graphics commands.

4. The method of claim 1, wherein analyzing the remote computing machine further comprises determining whether the remote computing machine has a computing element having the capability to receive three dimensional graphics commands and output three dimensional graphical data.

5. The method of claim 1, wherein analyzing the remote computing machine further comprises determining a score associated with the remote computing machine, the score associated with a hardware and software configuration of the remote computing machine.

6. The method of claim 1, wherein analyzing the remote computing machine further comprises identifying codecs available on the remote computing machine.

7. The method of claim 1, wherein analyzing the local computing machine further comprises determining whether the local computing machine has a computing element having the capability to receive three dimensional graphics commands and output three dimensional graphical data.

8. The method of claim 1, wherein analyzing the connection further comprises determining an amount of bandwidth available in the connection.

9. The method of claim 1, wherein transferring further comprises transferring three dimensional graphics commands to the computing element on the local computing machine.

10. The method of claim 9, further comprising:

rendering, by the computing element, three dimensional graphical data from the three dimensional graphics commands; and

transmitting, by the local computing machine, the rendered three dimensional graphical data to the remote computing machine.

11. The method of claim 9, wherein transferring to a computing element on the local computing machine further comprises transferring to a graphics engine on the local computing machine.

12. The method of claim 9, wherein transferring to a computing element on the local computing machine further comprises transferring to a processor on the local computing machine, the processor having the capability to receive three dimensional graphics commands and output three dimensional graphical data.

13. The method of claim 9, wherein transferring to a computing element on the local computing machine further comprises transferring to an application executing on the local computing machine, the application receiving three dimensional graphics commands and outputting three dimensional graphical data.

14. The method of claim 10, further comprising:

receiving, by the remote computing machine, the three dimensional graphical data; and

displaying, by the remote computing machine, the received three dimensional graphical data.

15. The method of claim 1, wherein transferring further comprises transferring three dimensional graphics commands to the remote computing machine having a computing element.

16. The method of claim 15, further comprising:

receiving, by the computing element, the three dimensional graphics commands; and

outputting, by the computing element, three dimensional graphical data.

17. The method of claim 15, wherein transferring to a computing element on the remote computing machine further comprises transferring to a graphics engine on the remote computing machine.

18. The method of claim 15, wherein transferring to a computing element on the remote computing machine further comprises transferring to a processor on the remote computing machine, the processor having the capability to receive three dimensional graphics commands and output three dimensional graphical data.

19. The method of claim 15, wherein transferring to a computing element on the remote computing machine further comprises transferring to an application executing on the remote computing machine, the application receiving three dimensional graphics commands and outputting three dimensional graphical data.

20. The method of claim 15 further comprising displaying, by the remote computing machine, the outputted three dimensional graphical data.

21. The method of claim 1 further comprising storing the three dimensional graphics commands prior to transmission.

22. The method of claim 1, wherein transferring further comprises transferring three dimensional graphics commands to the computing element on the local computing machine, the computing element comprising a computer hardware element capable of rendering three dimensional graphical data from graphics commands.

23. The method of claim 1, wherein transferring further comprises transferring three dimensional graphics commands to the remote computing machine having a computing element, the computing element comprising a computer hardware element capable of rendering three dimensional graphical data from graphics commands.

24. The method of claim 1, wherein the local computing machine is a mobile device.

25. A system for remoting three dimensional graphical data, the system comprising:

a remote computing machine within a distributed computing system;

a connection installed between the remote computing machine and a local computing machine within the distributed computing system, the remote computing machine communicating with the local computing machine via the connection;

an application executing within a session on the local computing machine, the application generating three dimensional graphics commands; and

the local computing machine having a computing element that receives as input three dimensional graphics commands and outputs three dimensional graphical data, the local computing machine executing an application to;

intercept the three dimensional graphics commands prior to the processing of the three dimensional graphics commands by a display driver executing on the local computing machine,

analyze the application to obtain application specific characteristics of the application,

analyze the remote computing machine to obtain capabilities of the remote computing machine, analyze the local computing machine to obtain capabilities of the local computing machine,

analyze the connection to obtain characteristics of the connection,

determine, based on the application characteristics, the remote computing machine capabilities, the local computing machine capabilities, and the connection characteristics, to transfer the intercepted three dimensional graphics commands to one of the remote computing machine and the computing element on the local computing machine; and

transfer the three dimensional graphics commands to one of the remote computing machine and the computing element on the local computing machine responsive to determining where to transfer the intercepted three dimensional graphics commands;

wait a predetermined period of time within the session;

re-obtain each of the application specific characteristics, the remote computing machine capabilities, the local computing machine capabilities, and the connection characteristics;

re-determine, based on the application characteristics, the remote computing machine capabilities, the local computing machine capabilities, and the connection characteristics, to which of the remote computing machine and the computing element on the local machine to transfer the intercepted three dimensional graphics commands; and

transfer the three dimensional graphics commands to one of the remote computing machine and the computing element on the local computing machine responsive to re-determining where to transfer the intercepted three dimensional graphics commands.

26. The system of claim 25, wherein the local computing machine intercepts high level three dimensional graphics commands.

27. The system of claim 25, wherein the local computing machine intercepts primitive three dimensional graphics commands.

28. The system of claim 25, wherein the local computing machine analyzes the remote computing machine by determining whether the remote computing machine has a computing element capable to receive three dimensional graphics commands and output three dimensional graphical data.

29. The system of claim 25, wherein the local computing machine analyzes the remote computing machine by determining a score associated with the remote computing machine, the score associated with a hardware and software configuration of the remote computing machine.

30. The system of claim 25, wherein the local computing machine analyzes the remote computing machine by identifying codecs available on the remote computing machine.

31. The system of claim 25, wherein the local computing machine analyzes the local computing machine to determine whether the computing element on the local computing machine has the capability to receive three dimensional graphics commands and output three dimensional graphical data.

32. The system of claim 25, wherein the local computing machine analyzes the connection to determine an amount of bandwidth available in the connection.

33. The system of claim 25, wherein transferring further comprises transferring three dimensional graphics commands to the computing element on the local computing machine.

34. The system of claim 33, further comprising:

the computing element on the local computing machine, rendering three dimensional graphical data from the three dimensional graphics commands; and

the local computing machine transmitting the rendered three dimensional graphical data to the remote computing machine.

35. The system of claim 33, further comprising the local computing machine transferring to a graphics engine on the local computing machine.

36. The system of claim 33, further comprising the local computing machine transferring to a processor on the local computing machine, the processor having the capability to receive three dimensional graphics commands and output three dimensional graphical data.

37. The system of claim 33, further comprising the local computing machine transferring to an application executing on the local computing machine, the application receiving three dimensional graphics commands and outputting three dimensional graphical data.

38. The system of claim 34 further comprising the remote computing machine receiving the three dimensional graphical data, and displaying the three dimensional graphical data.

39. The system of claim 25 wherein the local computing machine transfers the three dimensional graphics commands to the remote computing machine having a computing element.

40. The system of claim 39 further comprising:

the computing element receiving the three dimensional graphics commands and outputting three dimensional graphical data.

41. The system of claim 39 wherein the local computing machine transfers the three dimensional graphics commands to a graphics engine on the remote computing machine.

42. The system of claim 39 wherein the local computing machine transfers the three dimensional graphics commands to a processor on the remote computing machine, the processor having the capability to receive three dimensional graphics commands and output three dimensional graphical data.

43. The system of claim 39 wherein the local computing machine transfers the three dimensional graphics commands to an application executing on the remote computing machine, the application having the capability to receive three dimensional graphics commands as input and output three dimensional graphical data.

44. The system of claim 39, further comprising the remote computing machine displaying the outputted three dimensional graphical data.

45. The system of claim 25 wherein the local computing machine stores the three dimensional graphics commands prior to transmission.

46. The system of claim 25, wherein the local computing machine is a mobile device.