Methods and apparatus for generating graphical and media displays at a client

• US 20050198189A1
• Filed: 03/14/2003
• Published: 09/08/2005
• Est. Priority Date: 03/14/2002
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Panasyuk, Anatoliy, Laborczfalvi, Lee George, Roychoudhry, Anil, Robinson, David, Semaan, Pierre, Duursma, Martin
• Granted Patent: US 8,131,816 B2
• Time in Patent Office: Days
• US Class Current: 709/217
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G09G 2320/10 Special adaptations of display systems for operation with variable images
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display

Abstract

The invention generally relates to generating a display having graphical and/or media components at a client. In one aspect, a method for generating a graphical display at a client includes transmitting output from an application program executing on a server to the client, identifying a non-textual element within the application output, retrieving a compressed data format associated with the non-textual element, and transmitting to the client the compressed data format in place of the non-textual element. In another aspect, a method for generating a media presentation at a client includes transmitting output from an application program executing on a server to the client, identifying a media stream within the application output, intercepting an original compressed data set representing at least a portion of the media stream before processing by the application program, and transmitting the original compressed data set to the client.

Claims

1. A method for generating a graphical display at a client, the method comprising:

transmitting output from an application program executing on a server to the client;

identifying a non-textual element within the application output;

retrieving a compressed data format associated with the non-textual element; and

transmitting to the client the compressed data format in place of the non-textual element.

2. The method of claim 1 further comprising:

receiving the compressed data format at the client; and

generating a display at the client using the compressed data format.

3. The method of claim 1 further comprising:

identifying a textual element within the application output; and

transmitting to the client the textual element.

4. The method of claim 3 further comprising:

receiving the compressed data format and the textual element at the client; and

generating a display at the client using the compressed data format and the textual element.

5. The method of claim 1 wherein the compressed data format is transmitted using at least one presentation layer protocol packet.

6. The method of claim 5 wherein the at least one presentation layer protocol packet is transmitted using a command for transmitting a file in its native format.

7. The method of claim 5 wherein the at least one presentation layer protocol packet conforms to a remote access protocol.

8. The method of claim 1 wherein the non-textual element is a bitmap representation and wherein the step of transmitting the compressed data format comprises replacing the bitmap representation with the compressed data format.

9. The method of claim 1 further comprising determining the capability of the client to render the non-textual element using the compressed data format.

10. The method of claim 9 further comprising, upon determination that the client cannot render the non-textual element using the compressed data format, transmitting an image-rendering library capable of rendering the non-textual element using the compressed data format.

11. The method of claim 1 wherein the step of identifying further comprises:

intercepting the application output; and

inspecting the intercepted output for a bitmap representation of the non-textual element.

12. The method of claim 1 wherein the step of retrieving further comprises:

calculating a first check value for a bitmap representation of the non-textual element;

and searching an image store for the compressed data format having a check value identical to the first check value.

13. A method for generating a graphical display at a client, the method comprising:

transmitting output from an application program executing on a server to the client;

identifying a bitmap representation within the application output;

determining a check value for the bitmap representation;

retrieving a compressed data format of the bitmap representation using at least in part the check value; and

transmitting to the client the compressed data format in place of the bitmap representation.

14. The method of claim 13 further comprising:

identifying a command associated with the bitmap representation within the application output; and

transmitting to the client the command along with the compressed data format.

15. The method of claim 14 further comprising:

receiving the compressed data format and the command at the client;

decompressing the compressed data format to generate the bitmap representation; and

generating a display using the bitmap representation and the command.

16. The method of claim 13 wherein the compressed data format and a command associated with displaying the bitmap representation are transmitted using a presentation layer protocol packet.

17. The method of claim 16 wherein the presentation layer protocol packet is transmitted using a protocol command established for transmitting a file in its compressed data format.

18. The method of claim 16 wherein the presentation layer protocol packet is conformed to a presentation protocol.

19. The method of claim 13 former comprising determining the capability of the client to render the bitmap representation using the compressed data format.

20. The method of claim 19 further comprising, upon determination that the client cannot render the bitmap representation using the compressed data format, transmitting an image-rendering library capable of rendering the bitmap representation using the compressed data format.

21. The method of claim 13 wherein the step of identifying further comprises:

intercepting the application output; and

inspecting the intercepted output for a bitmap representation of one or more non-textual elements.

22. The method of claim 13 wherein the step of determining further comprises:

calculating a CRC based on the bitmap representation;

and using the calculated CRC as the check value.

23. A system for generating a graphical display at a client, the system comprising:

an output filter module configured to intercept output produced by an application program, identify a non-textual element of the output and retrieve a compressed data format associated with the non-textual element; and

a server agent configured to transmit to the client the compressed data format in place of the non-textual element.

24. The system of claim 23 further comprising a server node, the server node including the server agent and the output filter module.

25. The system of claim 23 further comprising:

a client agent configured to receive the compressed data format and to generate a display of the non-textual element using the received compressed data format.

26. The system of claim 25 further comprising a client node, the client node including the client agent and a display.

27. The system of claim 23 wherein the server agent is further configured to transmit the compressed data format using a presentation layer protocol packet.

28. The system of claim 27 wherein the server agent is further configured to include a command in the presentation layer protocol packet for transmitting the compressed data format in place of the non-textual element.

29. The system of claim 23 wherein the server agent is further configured to conform to a thin client protocol.

30. The system of claim 23 wherein the non-textual element is a bitmap representation.

31. The system of claim 23 wherein the output filter module is further configured to determine whether the client is capable of rendering the non-textual element using the compressed data format.

32. The system of claim 31 wherein the server agent is further configured to transmit an image-rendering library capable of rendering the non-textual element using the compressed data format upon determination that the client cannot render the non-textual element using the compressed data format.

33. The system of claim 23 wherein the output filter module is further configured to intercept the output and inspect the intercepted output for a bitmap representation of a non-textual element.

34. A system for generating a graphical display at a client, the system comprising:

a network;

a server in communication with the network, the server including, an output filter module configured to intercept output produced by an application program, identify a non-textual element of the output and retrieve a compressed data format associated with the non-textual element; and

a server agent in communication with the output filter module, the server agent configured to transmit to the client the compressed data format hi place of the non-textual element; and

the client in communication with the network, the client including,

a client agent in communication with the server agent, the client agent configured to receive the compressed data format and to generate a display of the non-textual element using the received compressed data format.

35. An article of manufacture having computer-readable program means embodied therein for generating a graphical display at a client, the article comprising:

computer-readable program means for intercepting output produced by an application program executing on a server;

computer-readable program means for identifying a non-textual element of the output;

computer-readable program means for retrieving a compressed data format associated with the non-textual element; and

computer-readable program means for transmitting to the client the compressed data format in place of the non-textual element.

36. An article of manufacture having computer-readable program means embodied therein for generating a graphical display at a client, the article comprising:

computer-readable program means for intercepting output produced by an application program executing on a server;

computer-readable program means for identifying a bitmap representation within the output produced by the application program;

computer-readable program means for determining a check value for the bitmap representation;

computer-readable program means for retrieving a compressed data format of the bitmap representation using at least in part the check value; and

computer-readable program means for transmitting to the client the compressed data format in place of the bitmap representation.

37. A method for generating a media presentation at a client, the method comprising:

(a) transmitting output from an application program executing on a server to the client;

(b) identifying a media stream within the application output;

(c) intercepting an original compressed data set representing at least a portion of the media stream before processing by the application program; and

(d) transmitting the original compressed data set to the client.

38. The method of claim 37 further comprising determining that the application program utilizes external codecs.

39. The method of claim 37 further comprising:

receiving the original compressed data set at the client;

decompressing the original compressed data set at the client to generate a decompressed data set; and

generating the media presentation at the client using the decompressed data set.

40. The method of claim 37 further comprising:

capturing timing information associated with the media stream;

and transmitting the timing information to the client.

41. The method of claim 40 further comprising:

receiving the original compressed data set and the timing information at the client;

decompressing the original compressed data set at the client to generate a decompressed data set; and

generating the media presentation at the client using the decompressed data set and the timing information.

42. The method of claim 39 wherein the media stream comprises a video stream and the media presentation comprises a graphical display.

43. The method of claim 42 further comprising:

transmitting to the client information for locating images of the video stream on a display screen;

receiving the information for locating at the client; and

generating the media presentation at the client using the decompressed data set and the information for locating.

44. The method of claim 39 further comprising:

transmitting non-media graphical information from the application output to the client;

receiving the non-media graphical information at the client; and

generating the media presentation at the client using the decompressed data set and the non-media graphical information.

45. The method of claim 39 wherein the media stream comprises an audio stream and the media presentation comprises an audio presentation.

46. The method of claim 37 further comprising informing the server of at least one media format supported by a client agent installed on the client.

47. The method of claim 46 wherein the at least one media format is determined by enumerating at least one codec installed on the client.

48. The method of claim 46 wherein the at least one media format is determined by enumerating at least one codec embedded in the client agent installed on the client.

49. The method of claim 46 wherein a globally unique identifier is used to inform the server of the at least one media format.

50. The method of claim 46 wherein a four character code is used to inform the server of the at least one media format.

51. A method for generating a media presentation at a client, the method comprising:

(a) transmitting output from an application program executing on a server to the client;

(b) identifying a media stream within the application output;

(c) intercepting a first decompressed data set representing at least a portion of the media sream;

(d) compressing the intercepted first decompressed data set; and

(e) transmitting the compressed data set to the client in place of the first decompressed data set.

52. The method of claim 51 further comprising determining that the application program utilizes embedded codecs.

53. The method of claim 51 further comprising:

receiving the compressed data set at the client;

decompressing the compressed data set at the client to generate a second decompressed data set; and

generating the media presentation at the client using the second decompressed data set.

54. The method of claim 51 further comprising:

capturing timing information associated with the media stream;

and transmitting the tuning information to the client.

55. The method of claim 54 further comprising:

receiving the compressed data set and the timing information at the client;

decompressing the compressed data set at the client to generate a second decompressed data set; and

generating the media presentation at the client using the second decompressed data set and the timing information.

56. The method of claim 53 wherein the media stream comprises a video stream and the media presentation comprises a graphical display.

57. The method of claim 56 further comprising:

transmitting to the client information for locating images of the video stream on a display screen;

receiving the information for locating at the client; and

generating the media presentation at the client using the second decompressed data set and the information for locating.

58. The method of claim 53 further comprising:

transmitting non-media graphical information from the application output to the client;

receiving the non-media graphical information at the client; and

generating the media presentation at the client using the second decompressed data set and the non-media graphical information.

59. The method of claim 56 wherein (d) comprises compressing the intercepted first decompressed data set of the video stream using a lightweight glossy video encoding algorithm.

60. The method of claim 59 wherein the lightweight glossy video encoding algorithm comprises MJPEG compression.

61. The method of claim 59 wherein (d) further comprises choosing a compression ratio for the lightweight glossy video encoding algorithm.

62. The method of claim 51 further comprising informing the server of at least one media format supported by a client agent installed on the client

63. The method of claim 62 wherein the at least one media format is determined by enumerating at least one codec installed on the client.

64. The method of claim 62 wherein the at least one media format is determined by enumerating at least one codec embedded in the client agent installed on the client.

65. The method of claim 62 wherein a globally unique identifier is used to inform the server of the at least one media format.

66. The method of claim 62 wherein a four character code is used to inform the server of the at least one media format.

67. A method for generating a media presentation at a client, the method comprising:

(a) informing a server of at least one media format supported by a client agent installed on the client;

(b) receiving a compressed data set representing at least a portion of a media stream at the client;

(c) decompressing the compressed data set at the client to generate a decompressed data set; and

(d) generating the media presentation at the client using the decompressed data set.

68. The method of claim 67 further comprising:

receiving timing information associated with the media stream;

and using the timing information to generate the media presentation.

69. The method of claim 67 wherein the media stream comprises a video stream and the media presentation comprises a graphical display.

70. The method of claim 69 further comprising:

receiving at the client information for locating images of the video stream on a display screen; and

generating the media presentation at the client using the decompressed data set and the information for locating.

71. The method of claim 67 further comprising:

receiving non-media graphical information at the client; and

generating the media presentation at the client using the decompressed data set and the non-media graphical information.

72. The method of claim 67 wherein the media stream comprises an audio stream and the media presentation comprises an audio presentation.

73. The method of claim 67 wherein the at least one media format is determined by enumerating at least one codec installed on the client.

74. The method of claim 67 wherein the at least one media format is determined by enumerating at least one codec embedded in the client agent installed on the client.

75. The method of claim 67 wherein a globally unique identifier is used to inform the server of the at least one media format.

76. The method of claim 67 wherein a four character code is used to inform the server of the at least one media format.

77. A system for generating a media presentation at a client, the system comprising:

an application program configured to identify a media stream within output produced by the application program; and

an output filter module configured to intercept an original compressed data set representing at least a portion of the media stream before processing by the application program and transmit the original compressed data set to the client.

78. The system of claim 77 further comprising a client agent configured to receive the original compressed data set, decompress the original compressed data set at the client to generate a decompressed data set, and generate the media presentation at the client using the decompressed data set.

79. The system of claim 77 wherein the output filter module is further configured to capture timing information associated with the media stream and to transmit the timing information to the client.

80. The system of claim 79 further comprising a client agent configured to receive the original compressed data set and the timing information, decompress the original compressed data set to generate a decompressed data set, and generate the media presentation using the decompressed data set and the timing information.

81. The system of claim 78 wherein the media stream is a video stream and the client agent is further configured to receive information for locating images of the video stream on a display screen and to generate the media presentation at the client using the decompressed data set and the information for locating.

82. The system of claim 78 wherein the client agent is further configured to receive non-media graphical information and to generate the media presentation at the client using the decompressed data set and the non-media graphical information.

83. A system for generating a media presentation at a client, the system comprising:

an application program configured to identify a media stream within output produced by the application program; and

an output filter module configured to intercept a first decompressed data set representing at least a portion of the media stream, compress the intercepted first decompressed data set of the media stream, and transmit the compressed data set in place of the first decompressed data set to the client.

84. The system of claim 83 further comprising a client agent configured to receive the compressed data set, decompress the compressed data set at the client to generate a second decompressed data set, and generate the media presentation at the client using the second decompressed data set.

85. The system of claim 83 wherein the output filter module is further configured to capture timing information associated with the media stream and to transmit the timing information to the client.

86. The system of claim 85 further comprising a client agent configured to receive the compressed data set and the timing information, decompress the compressed data set to generate a second decompressed data set, and generate the media presentation using the second decompressed data set and the timing information.

87. The system of claim 84 wherein the media stream is a video stream and the client agent is further configured to receive information for locating images of the video stream on a display screen and to generate the media presentation at the client using the second decompressed data set and the information for locating.

88. The system of claim 84 wherein the client agent is further configured to receive non-media graphical information and to generate the media presentation at the client using the second decompressed data set and the non-media graphical”

information.

89. The system of claim 83 wherein the media stream comprises a video stream and the output filter module is further configured to compress the intercepted first decompressed data set of the video stream using a lightweight glossy video encoding algorithm.

90. The system of claim 89 wherein the lightweight glossy video encoding algorithm comprises MJPEG compression.

91. The system of claim 89 wherein the output filter module is further configured to determine a compression ratio for the lightweight glossy video encoding algorithm.

92. A system for generating a media presentation at a client, the system comprising:

a server; and

the client in communication with the server, the client comprising a client agent configured to inform the server of at least one media format supported by the client agent, receive a compressed data set representing at least a portion of a media stream;

decompress the compressed data set at the client to generate a decompressed data set, and generate the media presentation using the decompressed data set.

93. The system of claim 92 wherein the client agent is further configured to receive timing information associated with the media stream and to use the timing information to generate the media presentation.

94. The system of claim 92 wherein the media stream is a video stream and the client agent is further configured to receive information for locating images of the video stream on a display screen and to generate the media presentation at the client using the decompressed data set and the information for locating.

95. The system of claim 92 wherein the client agent is further configured to receive non-media graphical information and generate the media presentation using the decompressed data set and the non-media graphical information.

96. An article of manufacture having computer-readable program means embodied therein for generating a media presentation at a client, the article comprising:

computer-readable program means for transmitting output from an application program executing on a server to the client;

computer-readable program means for identifying a media stream within the application output;

computer-readable program means for intercepting an original compressed data set representing at least a portion of the media stream before processing by the application program; and

computer-readable program means for transmitting the original compressed data set to the client.

97. (canceled)

98. (canceled)

99. (canceled)

100. (canceled)

101. (canceled)

102. (canceled)

103. (canceled)

104. (canceled)

105. (canceled)

106. (canceled)

107. (canceled)

108. An article of manufacture having computer-readable program means embodied therein for generating a media presentation at a client, the article comprising:

computer-readable program means for transmitting output from an application program executing on a server to the client;

computer-readable program means for identifying a media stream within the application output;

computer-readable program means for intercepting a first decompressed data set representing at least a portion of the media stream;

computer-readable program means for compressing the intercepted first decompressed data set; and

computer-readable program means for transmitting the compressed data set to the client in place of the first decompressed data set.

109. (canceled)

110. (canceled)

111. (canceled)

112. (canceled)

113. (canceled)

114. (canceled)

115. (canceled)

116. (canceled)

117. (canceled)

118. (canceled)

119. (canceled)

120. (canceled)

121. (canceled)

122. (canceled)

123. An article of manufacture having computer-readable program means embodied therein for generating a media presentation at a client, the article comprising:

computer-readable program means for informing a server of at least one media format supported by a client agent installed on the client;

computer-readable program means for receiving a compressed data set representing at least a portion of a media stream at the client;

computer-readable program means for decompressing the compressed data set at the client to generate a decompressed data set; and

computer-readable program means for generating the media presentation at the client using the decompressed data set.

124. (canceled)

125. (canceled)

126. (canceled)

127. (canceled)

128. (canceled)

129. (canceled)

130. (canceled)

131. A system for generating a media presentation at a client, the system comprising:

a network;

a server in communication with the network, the server comprising;

an application program configured to identify a media stream within output produced by the application program; and

at least one output filter module configured to obtain a compressed data set representing at least a portion of the media stream before processing by the application program and transmit the compressed data set to the client; and

the client in communication with the network, the client comprising;

a client agent configured to inform the server of at least one media format supported by the client agent, receive the compressed data set, decompress the compressed data set at the client to generate a decompressed data set, and generate the media presentation at the client using the decompressed data set.

132. An article of manufacture having computer-readable program means embodied therein for generating a media presentation at a client, the article comprising:

computer-readable program means for intercepting an original compressed data set of a media stream; and

computer-readable program means for transmitting the original compressed data set to the client using a thin client protocol such as ICA or RDP.

133. An article of manufacture having computer-readable program means embodied therein for generating a media presentation at a client, the article comprising:

computer-readable program means for intercepting a decompressed data set of a media stream;

computer-readable program means for compressing the intercepted decompressed data set; and

computer-readable program means for transmitting the compressed data set to the client using a thin client protocol such as ICA or RDP.