Computer system for supporting multiple remote displays

• US 7,667,707 B1
• Filed: 05/05/2005
• Issued: 02/23/2010
• Est. Priority Date: 05/05/2005
• Status: Active Grant

• Current Assignee: III Holdings 1, LLC (Intellectual Ventures LLC)
• Original Assignee: Digital Display Innovations, LLC
• Inventors: Margulis, Neal D.
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): Guertin, Aaron M
• Application Number:
  US11/122,457
• Time in Patent Office: 1,755 Days
• Field of Search: 345 11- 13, 345 21- 3, 345 31- 4, 345/204, 345/418, 345/433, 345/441, 345/519, 345/521, 345/537, 345/545, 345/212, 345/5, 345/6, 345501-506, 345/520, 345/522, 364/900, 379/93.21, 709/222, 709/223, 709/225, 725/37, 725/42, 725/43, 714/807
• US Class Current: 345/519
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06F 3/1438 using more than one graphics controller
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/440218 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 conversion of standards in analogue television systems H04N7/01
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A multi-display computer system comprises a host computer system that processes windowed desktop environments for multiple remote displays, multiple users or a combination of the two. For each display and for each frame, the multi-display processor responsively manages each necessary portion of a windowed desktop environment. The necessary portions of the windowed desktop environment are further processed, encoded, and where necessary, transmitted over the network to the remote display for each user. Embodiments integrate the multi-display processor with the graphics processing unit, network controller, main memory controller or a combination of the three. The encoding process is optimized for network traffic and special attention is made to assure that all users have low latency interactive capabilities.

Claims

1. A graphics and display system capable of supporting multiple displays, comprising:

a System-On-Chip (SOC) integrated circuit including

2D and 3D drawing engines, video processing hardware, and a display controller that share a memory subsystem, that concurrently generate multiple display frames which may each correspond to a display frame at a respective remote display system, and that support

a first number of local display devices via local display paths, and

a second number plurality of remote display systems, not limited by the first number, each enabling a respective remote user to provide user inputs;

a frame buffer which stores display frames organized as sub frame memory areas representing display frame tiles for said remote display systems;

frame compare means that tracks on a frame or sub-frame basis which portions of a display have changed based on a checksum of a grid of a prior frame'"'"'s display data; and

display data encoder means that compresses the display data from the frame buffer;

means for connecting said SOC to a host CPU in a host computer system running software that works with said SOC to identify modified display frames or sub frames so that only modified display data will be transferred via a network subsystem to corresponding ones of said remote display systems; and

means for connecting said SOC to a network subsystem in communication with said second number of remote display systems and for enabling said plurality of remote display systems to transmit user inputs to said SOC.

2. The system of claim 1 wherein said Data encoder function effectively changes the resolution and color depth from the existing display format with a graceful degradation of the display quality and without the need to reconfigure the remote display device.

3. The system of claim 1 wherein said tracking software or other utilities within the graphics and display controller tracks which frames of which remote display systems are being updated.

4. The system of claim 1 wherein said CPU software swaps static displays out of the shared frame buffer and into main system RAM in order to process a larger number of displays than could be active in said frame buffer at one time.

5. The system of claim 1 wherein to assist said CPU software in identification of modified display frames, said display and graphics controller includes hardware which tracks when pixels are changed in the display memory area.

6. The system of claim 1 wherein said CPU software assists in choosing the method for encoding (the) display frames that have been modified.

7. A graphics and display system capable of supporting a first number of local display devices and an independent second number of remote display systems, comprising:

means to connect said graphics and display system to a host computer system which includes

a CPU subsystem running software that supports multiple remote users;

a display controller capable of supporting

a first number of local display devices by supplying display frames via first local display output paths, and

a second number of remote display systems not limited by the first number; and

a shared frame buffer which stores display frames organized as sub frame memory areas representing the display frame tiles for said remote display systems; and

2D and 3D drawing engines and video processing hardware for generating multiple display frames which may each correspond to a display frame at a remote display system; and

a data encoder which accesses the shared frame buffer and utilizes sub band encoding techniques to encodes sub frame data into wavelet transform multi-resolution compressed representations for transmission to update said remote display systems; and

means to connect managed and encoded output of said display controller to a network controller which in turn can be connected to said second number of remote display systems, each of which receives one or more of the multi-resolution compressed representations of the sub frame data.

8. The system of claim 7 wherein said display data encoder and network controller perform source based forward error correction techniques, transmit lower resolution sub bands of said wavelet transform multi-resolution data in a heavily protected manner, and transmit higher resolution sub bands in a less protected manner.

9. The system of claim 7 wherein

said 2D, 3D and video processing constructs sub frame images into display planes of data,

said display controller is configured to generate multiple composite display planes of data within said shared frame buffer, and

said multi-display processor processes said composite frames uniquely for each of the multiple remote display systems.

10. The system of claim 7 wherein said multi-display processor receives frame display data and stores said frame display data to RAM before a frame manager and a frame comparer perform precinct-based processing, and wherein said data encoder performs different types of encoding algorithms based on the type of frame data within the subframe and operates only on required subframes which have changed from the prior display frame for the corresponding said remote display system.

11. The system of claim 7 wherein said network controller is integrated with said multi-display processor and shares a common RAM.

12. The system of claim 7 wherein said network controller feeds back network transmission information to said data encoder and said data encoder responsively adjusts said data encoding to match said data encoding to the network characteristics.

13. The system of claim 7 wherein said display data encoder and network controller transmit the different resolution sub bands of said wavelet transform multi-resolution data over different communication channels.

14. The system of claim 7 wherein said network controller utilizes a transmission protocol such that if errors occur in certain of the frame data packets then the erroneous packets are not retransmitted.