Methods and apparatus for enabling multiple remote displays

• US 8,766,993 B1
• Filed: 03/30/2006
• Issued: 07/01/2014
• Est. Priority Date: 04/06/2005
• Status: Active Grant

• Current Assignee: Teradici Corporation (HP Inc.)
• Original Assignee: Teradici Corporation (HP Inc.)
• Inventors: Hobbs, David V.
• Primary Examiner(s): Merouan, Abderrahim
• Application Number:
  US11/278,128
• Time in Patent Office: 3,015 Days
• Field of Search: 345/536, 345/537, 345/538, 345/539, 345/540
• US Class Current: 345/536
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A method of transmitting visual data from a host computer to multiple displays across a computer network is disclosed. Visual data is stored in a plurality of frame buffers, each frame buffer associated with a separate display. A frame buffer update sequence is determined, with operations to be performed on frame buffers in the plurality. The data stored in the plurality of frame buffers is encoded as specified by the buffer update sequence to yield encoded images and each encoded image is sent across a computer network to the separate display associated with the frame buffer from which the encoded image was derived.

Claims

1. A host computer comprising:

at least one processing unit comprising a processor memory having instructions executed by the at least one processing unit;

a display encoder memory independent of the at least one processing unit;

an Ethernet network connection independent of the at least one processing unit; and

a multi-method display encoder module comprising a lossless image encoder and a lossy image encoder, the multi-method display encoder module independent of the at least one processing unit and in communication with the display encoder memory and the at least one processing unit, the display encoder module in communication with clients over a standard corporate Ethernet network via the Ethernet network connection,

wherein the multi-method display encoder module is enabled to obtain visual data from the at least one processing unit,

wherein the display encoder memory comprises a plurality of encoder frame buffers configured to store the visual data obtained from the at least one processing unit, each encoder frame buffer in the plurality of encoder frame buffers associated with a different client display in a plurality of client displays connected to the host computer via the standard corporate network, each client display associated with one of the clients,

wherein the multi-method display encoder module compresses the visual data stored in each encoder frame buffer in the plurality of encoder frame buffers using at least one of a set comprising the lossless image encoder and the lossy image encoder to form a plurality of encoded images, and sends each in the plurality of encoded image across the standard corporate Ethernet network to the associated client display, and

wherein the at least one processing unit executes the instructions stored in the processor memory to render at least one different image for each encoder frame buffer in the plurality of encoder frame buffers, each rendered image placed in at least one of the plurality of encoder frame buffers at a pixel resolution and size for which the image is to be displayed on the encoder frame buffer'"'"'s associated client display, the size accommodating the rectangular dimension of an associated client'"'"'s display'"'"'s display area.

2. The host computer of claim 1, wherein the independent of the at least one processing unit for the multi-method display encoder module comprises, communications between the at least one processing unit and the multi-method display encoder module primarily enabled to communicate digital image data.

3. The host computer of claim 1, wherein the at least one processing unit comprises a graphics processing unit.

4. The host computer of claim 1, further comprising a second Ethernet network connection, the second Ethernet network connection disposed between the at least one processing unit and the standard corporate Ethernet network, wherein the multi-method display encoder sends each of the plurality of encoded image across the standard corporate Ethernet network to the associated client display independent of the second Ethernet network connection.

5. A method of transmitting visual data from a host computer to a plurality of displays across a standard corporate Ethernet network, comprising:

storing visual data in a plurality of frame buffers, each frame buffer of the plurality of frame buffers associated with one from the plurality of displays;

determining a frame buffer update sequence, the frame buffer update sequence comprising compression operations for the stored visual data in the each frame buffer in the plurality of frame buffers;

reading the stored visual data in the plurality of frame buffers in accordance with the determined frame buffer update sequence;

compressing the read visual data to yield a plurality of packetized encoded images; and

sending each of the plurality of packetized encoded images across the standard corporate Ethernet network to the one from the plurality of displays associated with the frame buffer from which the compressed image was derived.

6. The method of claim 5, wherein compressing the visual data comprises encoding a set of frame buffers, the set of frame buffers comprising spatially separate portions of a divided image, each of the spatially separate portions selected to match a resolution and size of the associated display.

7. The method of claim 5, wherein:

determining the frame buffer update sequence comprises determining individual update rates for individual frame buffers in the plurality of frame buffers, and

compressing the visual data comprises compressing the visual data stored in the individual frame buffers in accordance with the individual update rates.

8. The method of claim 7, further comprising adjusting a compression technique for at least one from the plurality of frame buffers to accommodate an available Ethernet network bandwidth between the host computer and the one from the plurality of displays associated with the one from the plurality of frame buffers.

9. The method of claim 5, wherein determining the frame buffer update sequence comprises determining image content of the visual data stored in the plurality of frame buffers and setting the frame buffer update sequence in accordance with the determined image content.

10. The method of claim 9, wherein:

the determined image content comprises video content and text content, the video content comprising an image in a video sequence, the text content comprising computer generated text.

11. The method of claim 5, wherein:

a one of the plurality of frame buffers comprises a first visual data portion and a second visual data portion;

determining the frame buffer update sequence comprises determining different service qualities for the first and the second visual data portions; and

compressing the visual data stored in the one from the plurality of frame buffers comprises encoding the first visual data portion and delayed encoding of the second visual data portion.

12. The method of claim 5, further comprising:

obtaining the visual data from a second plurality of frame buffers, the second plurality of frame buffers used in the rendering of the visual data.

13. A host computer comprising:

at least one local processing unit comprising a plurality of rendering frame buffers; and

a local encoder comprising;

an interface to the at least one local processing unit;

a plurality of display controller register sets presented to the processing unit, via the interface to the at least one local processing unit, as a plurality of independent display controller register sets when activated, each of the plurality of rendering frame buffers associated with one from the plurality of display controller register sets; and

a processor interface controller enabled to read the plurality of rendering frame buffers based on contents of the plurality of display controller register sets;

wherein the local encoder is enabled to read one from the plurality of rendering frame buffers to obtain a display image, compress the image and transmit the compressed image over a standard corporate Ethernet network to a remote display system.

14. The host computer of claim 13, wherein:

the local encoder is enabled to activate one of the plurality of display controller registers sets under control of virtualization software running on the at least one processing unit, the activation used to co-ordinate the one of the plurality of display controllers register sets with a virtual machine on the at least processing unit.

15. The host computer of claim 13, wherein the at least one processing unit comprises a graphics processing unit.

16. A host computer configured for displaying a plurality of images on a plurality of remote displays, said host computer comprising:

a processing system comprising a plurality of frame buffers, the plurality of images rendered into the plurality of frame buffers by the processing system; and

an encoder, independent of the processing system, said encoder comprising;

a processor expansion interface communicatively coupled to the processing system, the processor expansion interface enabled to support Direct Memory Access (DMA) to the plurality of frame buffers;

an encoder control enabled to read a selective plurality of image frame portions of the plurality of images via the processor expansion interface;

a digital image decomposition module comprising a digital image filter, the digital image filter enabled to classify each of the selective plurality of image frame portions into at least one image layer based on determining image types of each of the selective plurality of image frame portions;

a multi-method encoder comprising a lossy image encoder and a lossless image encoder, the multi-method encoder enabled to compress the selective plurality of image frame portions by selecting at least one of a set comprising the lossy image encoder and the lossless image encoder based on the at least one image layer of each selective plurality of image frame portions to generate encoded image frame portions; and

an IP Ethernet network interface communicatively coupled to the plurality of remote displays, the IP Ethernet network interface enable to transmit the encoded image frame portions to associated remote displays of the plurality of remote displays.

17. The system of claim 16, wherein:

the encoder further comprises an encoder memory enabled to store copies of the selective plurality of image frame portions, the copies used by the multi-method encoder to generate the encoded image frame portions, the encoder control enable to create the copies from the read selective plurality of image frame portions.

18. The system of claim 16, wherein:

the encoder comprises a first instruction set for encoding images independent of a second instruction set of the processing system.

19. The system of claim 16, wherein:

each in the plurality of frame buffers comprises an allocation of memory in the processing system, the allocation of memory associated with a display resolution of an associated one from the plurality of remote displays.

20. The system of claim 16, wherein:

the encoder control is further enabled to determine the selective plurality of image frame portions by reading a table comprising identification of changed portions of the plurality of frame buffers.

21. The system of claim 20, wherein:

the encoder control is further enabled to clear entries in the table to indicate that the identified changed portions of the plurality of frame buffers have been read by the encoder control.

22. The system of claim 20, wherein:

the processing system is enable to determine the changes to the plurality of frame buffers and update the table to identify the changed portions of the plurality of frame buffers.

23. The system of claim 16, wherein:

the encoder is enabled to repeatedly encode each of the plurality of frame buffers at an associated update rate,

the encoder is further enabled to determine a change in bandwidth of the IP Ethernet network associated with the frame buffer and to adjust the associated update rate based on the change in the bandwidth of the IP Ethernet network associated with the frame buffer.

24. The system of claim 16, wherein:

the image decomposition module is further enabled to determine the image types by obtaining interpretations of drawing commands used in the rendering of the plurality of images.

25. The system of claim 16, wherein:

the processor expansion interface comprises a plurality of registers sets, each in the plurality of frame buffers associated with a one in the plurality of registers sets, each of the plurality of register sets enabled to present an equivalent interface to a one of a plurality of virtual machines in the processing system, each of the plurality of register sets substantially dedicated to an associated one of the plurality of virtual machines.

26. The encoder of claim 16 wherein:

the image filter is enabled to identify natural images in the plurality of image frame portions;

the lossy image encoder comprises a discrete transform encoder;

the multi-method encoder encodes the identified natural images with the discrete transform encoder;

the image filter is enabled to identify at least one of a set comprising computer generated text and graphics images in the plurality of image frame portions; and

the multi-method encoder encodes the identified at least one of the set comprising computer generated text and graphics images with the lossless image encoder.

27. The host computer of claim 16 wherein:

the processor expansion interface comprises at least one of a set comprising;

a computer expansion card interface, and

a Peripheral Component Interconnect Express (PCI Express);

wherein the processor expansion interface enables read access to the plurality of frame buffers by the encoder.