System and method for providing a remote user interface for an application executing on a computing device

• US 20070043550A1
• Filed: 08/16/2005
• Published: 02/22/2007
• Est. Priority Date: 08/16/2005
• Status: Active Grant

• Current Assignee: Exent Technologies Incorporated
• Original Assignee: Exent Technologies Incorporated
• Inventors: Tzruya, Yoav
• Granted Patent: US 7,844,442 B2
• Time in Patent Office: Days
• US Class Current: 703/24
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131

Abstract

A system and method for providing a remote user interface for an application, such as a video game, executing on a computing device. The system includes a computing device configured to execute a software application and at least one remote user interface (UI) communicatively coupled to the computing device via a data communication network. The remote UI includes at least one hardware device such as a video, audio or user input/output (I/O) device. The computing device is further configured to emulate the hardware device locally and to redirect function calls generated by the software application for the emulated local hardware device to the remote UI for processing by the hardware device.

Claims

1. A method for providing a remote user interface for a computing device, comprising:

emulating a local hardware device on the computing device;

executing a software application on the computing device; and

redirecting commands generated by the software application for the emulated local hardware device to a remote user interface (UI).

2. The method of claim 1, wherein emulating a local hardware device comprises emulating a graphics device and wherein redirecting commands comprises redirecting graphics commands.

3. The method of claim 1, wherein emulating a local hardware device comprises emulating an audio device and wherein redirecting commands comprises redirecting audio commands.

4. The method of claim 1, wherein emulating a local hardware device comprises emulating a user input/output (I/O) device and wherein redirecting commands comprises redirecting control commands.

5. The method of claim 4, further comprising:

receiving control commands from the remote UI; and

providing the received control commands to the software application.

6. The method of claim 1, wherein redirecting commands generated by the software application for the emulated local hardware device comprises:

intercepting the commands by a software hook installed in a software stack of the computing device.

7. The method of claim 1, further comprising:

receiving capability information from the remote UI; and

allocating functionality between the computing device and the remote UI based on the capability information.

8. The method of claim 7, wherein allocating functionality between the computing device and the remote UI based on the capability information comprises:

installing a software hook in a software stack of the computing device.

9. The method of claim 7, further comprising:

dynamically receiving updated capability information from the remote UI; and

dynamically reallocating functionality between the computing device and the remote UI based on the updated capability information.

10. The method of claim 1, further comprising:

emulating a second local hardware device on the computing device;

executing a second software application on the computing device; and

redirecting commands generated by the second software application for the emulated second local hardware device to a second remote user interface (UI).

11. A computing device, comprising:

means for emulating a local hardware device;

means for executing a software application; and

means for redirecting commands generated by the software application for the emulated local hardware device to a remote user interface (UI).

12. The computing device of claim 11, wherein the means for emulating a local hardware device comprises means for emulating a graphics device and wherein the means for redirecting commands comprises means for redirecting graphics commands.

13. The computing device of claim 11, wherein the means for emulating a local hardware device comprises means for emulating an audio device and wherein the means for redirecting commands comprises means for redirecting audio commands.

14. The computing device of claim 11, wherein the means for emulating a local hardware device comprises means for emulating a user input/output (I/O) device and wherein the means for redirecting commands comprises means for redirecting control commands.

15. The computing device of claim 14, further comprising:

means for receiving control commands from the remote UI; and

means for providing the received control commands to the software application.

16. The computing device of claim 11, wherein the means for redirecting commands generated by the software application for the emulated local hardware device comprises:

a software hook installed in a software stack of the computing device.

17. The computing device of claim 11, further comprising:

means for receiving capability information from the remote UI; and

means for allocating functionality between the computing device and the remote UI based on the capability information.

18. The computing device of claim 17, wherein the means for allocating functionality between the computing device and the remote UI based on the capability information comprises:

means for installing a software hook in a software stack of the computing device based on the capability information.

19. The computing device of claim 17, further comprising:

means for dynamically receiving updated capability information from the remote UI; and

means for dynamically reallocating functionality between the computing device and the remote UI based on the updated capability information.

20. The computing device of claim 11, further comprising:

means for emulating a second local hardware device on the computing device;

means for executing a second software application on the computing device; and

means redirecting commands generated by the second software application for the emulated second local hardware device to a second remote user interface (UI).

21. A computer program product comprising a computer useable medium having computer program logic recorded thereon, said computer program logic comprising:

means for enabling a computing device to emulate a local hardware device; and

means for enabling the computing device to redirect commands generated by a software application for the emulated local hardware device to a remote user interface (UI).

22. The computing program product of claim 21, wherein the means for enabling the computing device to emulate a local hardware device comprises means for enabling the computing device to emulate a graphics device and wherein the means for enabling the computing device to redirect commands comprises means for enabling the computing device to redirect graphics commands.

23. The computer program product of claim 21, wherein the means for enabling the computing device to emulate a local hardware device comprises means for enabling the computing device to emulate an audio device and wherein the means for enabling the computing device to redirect commands comprises means for enabling the computing device to redirect audio commands.

24. The computer program product of claim 21, wherein the means for enabling the computing device to emulate a local hardware device comprises means for enabling the computing device to emulate a user input/output (I/O) device and wherein the means for enabling the computing device to redirect commands comprises means for enabling the computing device to redirect control commands.

25. The computer program product of claim 24, further comprising:

means for enabling the computing device to receive control commands from the remote UI; and

means for enabling the computing device to provide the received control commands to the software application.

26. The computer program product of claim 21, wherein the means for enabling the computing device to redirect commands generated by the software application for the emulated hardware device comprises:

means for enabling the computing device to intercept commands directed to a software stack of the computing device.

27. The computer program product of claim 21. further comprising:

means for enabling the computing device to receive capability information from the remote UI; and

means for enabling the computing device to allocate functionality between the computing device and the remote UI based on the capability information.

28. The computer program product of claim 27, wherein the means for enabling the computing device to allocate functionality between the computing device and the remote UI based on the capability information comprises:

means for enabling the computing device to install a software hook in a software stack of the computing device based on the capability information.

29. The computer program product of claim 27, further comprising:

means for enabling the computing device to dynamically receive updated capability information from the remote UI; and

means for enabling the computing device to dynamically reallocate functionality between the computing device and the remote UI based on the updated capability information.

30. The computer program product of claim 21, further comprising:

means for enabling the computing device to emulate a second local hardware device on the computing device; and

means for enabling the computing device to redirect commands generated by a second software application for the emulated second local hardware device to a second remote user interface (UI).

31. A method for operating a remote user interface (UI) for a computing device comprising:

publishing graphics capability information to the computing device over a data communication network;

receiving graphics commands from the computing device over the data communication network, wherein the format of the graphics commands received from the computing device is consistent with the graphics capability information; and

processing the graphics commands in a graphics device to generate video content therefrom.

32. The method of claim 31, further comprising:

rendering and displaying the video content.

33. The method of claim 31, wherein publishing graphics capability information comprises publishing graphics capability information in accordance with a UPnP protocol.

34. The method of claim 31, wherein publishing graphics capability information to the computing device over a data communication network comprises publishing graphics capability information to the computing device over a local area network.

35. The method of claim 31, wherein publishing graphics capability information to the computing device over a data communication network comprises publishing graphics capability information to the computing device over a wide area network.

36. The method of claim 31, wherein receiving graphics commands comprises receiving one of OpenGL commands, DirectX commands, or Graphics Device Interface commands.

37. The method of claim 31, wherein receiving graphics commands comprises receiving Pre-Rendering Code (PRC) commands.

38. A remote user interface (UI) for a computing device comprising:

control logic; and

a graphics device coupled to the control logic;

wherein the control logic is configured to publish graphics capability information to the computing device over a data communication network and to receive graphics commands from the computing device over the data communication network, wherein the format of the graphics commands received from the computing device is consistent with the graphics capability information; and

wherein the graphics device processes the graphics commands to generate video content therefrom.

39. The remote UI of claim 38, further comprising:

a display that renders and displays the video content.

40. The remote UI of claim 38, wherein the control logic is configured to publish the graphics capability information in accordance with a UPnP protocol.

41. The remote UI of claim 38, wherein the control logic is configured to publish the graphics capability information to the computing device over a local area network.

42. The remote UI of claim 38, wherein the control logic is configured to publish the graphics capability information to the computing device over a wide area network.

43. The remote UI of claim 38, wherein the control logic is configured to receive one of OpenGL commands, DirectX commands, or Graphics Device Interface commands.

44. The remote UI of claim 38, wherein the control logic is configured to receive Pre-Rendering Code (PRC) commands.

45. A method for operating a remote user interface (UI) for a computing device comprising:

publishing audio capability information to the computing device over a data communication network;

receiving audio commands from the computing device over the data communication network, wherein the format of the audio commands received from the computing device is consistent with the audio capability information; and

processing the audio commands in an audio device to generate audio content therefrom.

46. The method of claim 45, further comprising:

playing the audio content.

47. The method of claim 45, wherein publishing audio capability information comprises publishing audio capability information in accordance with a UPnP protocol.

48. The method of claim 45, wherein publishing audio capability information to the computing device over a data communication network comprises publishing audio capability information to the computing device over a local area network.

49. The method of claim 45, wherein publishing audio capability information to the computing device over a data communication network comprises publishing audio capability information to the computing device over a wide area network.

50. The method of claim 45, wherein receiving audio commands comprises receiving DirectX commands.

51. A remote user interface (UI) for a computing device comprising:

control logic; and

an audio device coupled to the control logic;

wherein the control logic is configured to publish audio capability information to the computing device over a data communication network and to receive audio commands from the computing device over the data communication network, wherein the format of the audio commands received from the computing device is consistent with the audio capability information; and

wherein the audio device processes the audio commands to generate audio content therefrom.

52. The remote UI of claim 51, further comprising:

one or more speakers that play the audio content.

53. The remote UI of claim 51, wherein the control logic is configured to publish the audio capability information in accordance with a UPnP protocol.

54. The remote UI of claim 51, wherein the control logic is configured to publish the audio capability information to the computing device over a local area network.

55. The remote UI of claim 51, wherein the control logic is configured to publish the audio capability information to the computing device over a wide area network.

56. The remote UI of claim 51, wherein the control logic is configured to receive DirectX commands.

57. A method for operating a remote user interface (UI) for a computing device comprising:

publishing user input/output (I/O) device capability information to the computing device over a data communication network;

receiving control commands from the computing device over the data communication network, wherein the format of the control commands received from the computing device is consistent with the user I/O device capability information; and

processing the control commands in a user I/O device to generate output to a user.

58. The method of claim 57, further comprising:

processing input from a user in the user I/O device to generate control commands; and

transmitting the generated control commands to the computing device over the data communication network.

59. The method of claim 57, wherein publishing user I/O device capability information comprises publishing user I/O device capability information in accordance with a UPnP protocol.

60. The method of claim 57, wherein publishing user I/O device capability information to the computing device over a data communication network comprises publishing user I/O device capability information to the computing device over a local area network.

61. The method of claim 57, wherein publishing user I/O device capability information to the computing device over a data communication network comprises publishing user I/O device capability information to the computing device over a wide area network.

62. The method of claim 57, wherein receiving control commands comprises receiving DirectX commands.

63. A remote user interface (UI) for a computing device comprising:

control logic; and

a user input/output (I/O) device coupled to the control logic;

wherein the control logic is configured to publish user I/O device capability information to the computing device over a data communication network and to receive control commands from the computing device over the data communication network, wherein the format of the control commands received from the computing device is consistent with the user I/O device capability information; and

wherein the user I/O device processes the control commands to generate output for a user.

64. The remote UI of claim 63, wherein the user I/O device processes input from a user to generate control commands;

and

wherein the control logic is further configured to transmit the generated control commands to the computing device over the data communication network.

65. The remote UI of claim 63, wherein the control logic is configured to publish the user I/O device capability information in accordance with a UPnP protocol.

66. The remote UI of claim 63, wherein the control logic is configured to publish the user I/O device capability information to the computing device over a local area network.

67. The remote UI of claim 63, wherein the control logic is configured to publish the user I/O device capability information to the computing device over a wide area network.

68. The remote UI of claim 63, wherein the control logic is configured to receive DirectX commands.

69. A method for operating a remote user interface (UI) for a computing device comprising:

publishing graphics and audio capability information to the computing device over a data communication network;

receiving graphics and audio commands from the computing device over the data communication network, wherein the format of the graphics commands received from the computing device is consistent with the graphics capability information and the format of the audio commands received from the computing device is consistent with the audio capability information;

processing the graphics commands in a graphics device to generate video content therefrom; and

processing the audio commands in an audio device to generate audio content therefrom.

70. The method of claim 69, further comprising:

publishing user input/output (I/O) device capability information to the computing device over the data communication network; and

receiving control commands from the computing device over the data communication network, wherein the format of the control commands received from the computing device is consistent with the user I/O device capability information; and

processing the control commands in a user I/O device to generate output for a user.

71. A remote user interface (UI) for a computing device comprising:

control logic;

a graphics device coupled to the control logic; and

an audio device coupled to the control logic;

wherein the control logic is configured to publish graphics and audio capability information to the computing device over a data communication network and to receive graphics and audio commands from the computing device over the data communication network, wherein the format of the graphics commands received from the computing device is consistent with the graphics capability information and the format of the audio commands received from the computing device is consistent with the audio capability information;

wherein the graphics device processes the graphics commands to generate video content therefrom; and

wherein the audio device processes the audio commands to generate audio content therefrom.

72. The remote UI of claim 71, further comprising:

a user input/output (I/O) device coupled to the control logic;

wherein the control logic is further configured to publish user I/O device capability information to the computing device over the data communication network and to receive control commands from the computing device over the data communication network, wherein the format of the control commands received from the computing device is consistent with the user I/O device capability information; and

wherein the user I/O device processes the control commands to generate output for a user.

73. A system comprising:

a computing device configured to execute a software application;

a remote user interface (UI) communicatively coupled to the computing device via a data communication network;

wherein the computing device is further configured to emulate a local hardware device and to redirect commands generated by the software application for the emulated local hardware device to the remote UI.

74. The system of claim 73, wherein the remote UI includes a video device, the emulated local hardware device comprises an emulated video device, and wherein the redirected commands are processed by the video device within the remote UI to generate video content.

75. The system of claim 74, wherein the remote UI further includes a display that displays the video content.

76. The system of claim 74, further comprising:

a display device communicatively coupled to the remote UI that displays the video content.

77. The system of claim 73, wherein the remote UI includes an audio device, the emulated local hardware device comprises an emulated audio device, and wherein the redirected commands are processed by the audio device within the remote UI to generate audio content.

78. The system of claim 77, wherein the remote UI further includes one or more speakers that play the audio content.

79. The system of claim 78, further comprising:

a device communicatively coupled to the remote UI that plays the audio content.

80. The system of claim 73, wherein the remote UI includes a user input/output (I/O) device, the emulated local hardware device comprises an emulated user I/O device, and wherein the redirected commands are processed by the user I/O device.

81. The system of claim 80, wherein the computing device is further configured to receive control commands from the remote UI, the control commands originating from the user I/O device, and to redirect the control commands to the software application.

82. The system of claim 73, wherein the remote UI is communicatively coupled to the computing device via a local area network.

83. The system of claim 73, wherein the remote UI is communicatively coupled to the computing device via a wide area network.

84. The system of claim 73, wherein the remote UI is configured to publish capability information over the data communication network and wherein the computing device is further configured to receive the capability information and to allocate functionality between the computing device and the remote UI based on the capability information.

85. The system of claim 84, wherein the published capability information comprises one or more of graphics capability information, audio capability information, and user I/O device capability information.

86. The system of claim 84, wherein the emulation of hardware devices on the computing device is determined based upon the published capabilities of the remote UI.

87. The system of claim 73, further comprising:

a second remote UI communicatively coupled to the computing device via the data communication network;

wherein the computing device is further configured to execute a second software application, to emulate a second local hardware device, and to redirect commands generated by the second software application for the second emulated local hardware device to the second remote UI.

88. The system of claim 73, wherein each of the emulated local hardware devices have different capabilities based on respective published capabilities of each of the remote UIs.