Multiple remote display system

• US 20060282855A1
• Filed: 05/27/2005
• Published: 12/14/2006
• Est. Priority Date: 05/05/2005
• Status: Abandoned Application

• Current Assignee: III Holdings 1, LLC
• Original Assignee: Digital Display Innovations, LLC
• Inventors: Margulis, Neal D.
• Application Number:
  US11/139,149
• Publication Number:
  US 20060282855A1
• Time in Patent Office: Days
• US Class Current: 725/43
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06F 3/1438 using more than one graphics controller
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4363 Adapting the video stream to a specific local network, e.g. a Bluetooth® network
• CPC Class Codes: H04N 21/4402 involving reformatting operations of video signals for household redistribution, storage or real-time display details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44227 Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network configuring of peripheral devices in general G06F9/4411; monitoring connectivity in data switched networks H04L43/0811
• CPC Class Codes: H04N 21/4435 Memory management allocation of memory to service a request G06F9/5016; addressing or allocating within memory systems or architectures G06F12/02
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A multi-display system includes a host system that supports both graphics and video based frames for multiple remote displays, multiple users or a combination of the two. For each display and for each frame, a multi-display processor responsively manages each necessary aspect of the remote display frame. The necessary portions of the remote display frame are further processed, encoded and where necessary, transmitted over a network to the remote display for each user. In some embodiments, the host system manages a remote desktop protocol and can still transmit encoded video or encoded frame information where the encoded video may be generated within the host system or provided from an external program source. Embodiments integrate the multi-display processor with either the video decoder unit, graphics processing unit, network controller, main memory controller, or any combination thereof The encoding process is optimized for network traffic and attention is paid to assure that all users have low latency interactive capabilities.

Claims

1. A graphics and video display controller capable of supporting multiple displays, comprising:

a display controller for supporting

a first number of local display devices via local display paths, and

a second number of remote display systems, not limited by the first number;

a 2D drawing engine for generating display frames which may each correspond to a display frame at a remote display system;

a video processor for processing one or more formats of video streams;

means for connecting to a CPU subsystem that explicitly tracks which of said display frames are modified so that only modified display frames will be transmitted via a network subsystem to remote display systems;

means for connecting to external program sources that provide video program data; and

means for connecting to a network controller which utilizes network paths to communicate with said remote display systems.

2. The system of claim 1 wherein said video processor encodes said modified display frames to reduce the network bandwidth of an encoded display stream output for said remote display systems.

3. The system of claim 1 wherein video content from said external program sources is processed by said host system and is transmitted over said network subsystem in an encoded format such that it can be decoded by one or more of said remote display systems.

4. A graphics and video multi-display system capable of supporting a first number of local display devices and an independent second number of remote display systems etc, comprising:

means to connect said graphics and video multi-display system to a CPU subsystem;

a graphics and video display controller capable of supporting a first number of local display devices by supplying display frames via a local display output path;

a multi-display processor that receives display frames from said local display output path, and that has

a frame manager and a frame comparer which process each received display frame, and

a data encoder which encodes frame data for transmission to update said remote display systems;

means to connect said graphics and video display system to external program sources which provide video program data; and

means to connect said graphics and video multi-display system to a network controller which in turn can be connected to said remote display systems.

5. The system of claim 4 wherein said graphics and video display controller is configured to composite multiple display planes of data prior to the frames being output on said local display paths, and wherein said multi-display processor processes said composite frames for multiple remote display systems.

6. The system of claim 4 wherein video program data from said external program sources can be processed by said multi-display processor to include source based encoding prior to transmission to said remote display systems.

7. A graphics and video multi-display system with integrated multiple display support, comprising:

a multi-display controller capable of supporting a large number of independent display frames tiled in memory;

a display data encoder capable of encoding display frame data;

means for connecting to a CPU subsystem; and

means for connecting to a network controller which in turn can be connected to a number of remote display systems.

8. The system of claim 7 wherein said graphics and video multi-display system composites surfaces into frames for each of said remote display systems, and further comprising a multi-display frame manager that implicitly tracks what display frame data is transmitted to each said remote display system, then responsively utilizes said display data encoder to encode the changed frames for transmission via said network controller.

9. The system of claim 7 wherein said multi-display system composites surfaces into frames for each of said remote display devices, tracks what display frame data is transmitted to each said remote display device, compares precincts of new frame data with preceding frame data and uses said display data encoder to encode changed precincts for transmission via said network controller.

10. The system of claim 7 wherein said host system transmits encoded video program data streams from said external program sources without changing the video encoding format.

11. A remote display system for use in a multi-display system, comprising:

a network controller for interfacing said remote display system to a host system in said multi-display system;

a data decoder and frame manager for decoding data received from said host system;

a local graphics and video controller including a local processor which runs a remote display protocol which performs graphics commands initiated by said host system;

and

a display controller for using the most recently reassembled frame of data to refresh a display screen;

where said local processor manages functions of said remote display system.

12. The system of claim 11 wherein said local processor, said display controller, and said data decoder are implemented together as an integrated circuit.

13. The system of claim 11 wherein said data decoder includes a display data decoder and frame manager which support data resiliency based on source based encoding to conceal errors that occur in data transmission.

14. The system of claim 11 wherein said data decoder includes a video decoder for decoding video and providing said decoded video to said display controller for updating a display screen.

15. A host system for supporting multiple displays, comprising:

means to connect external program sources;

a CPU subsystem having a CPU which performs remote display procedures for controlling a graphics and video controller at a remote display system;

a local graphics and video processor for performing 2D, 3D and video processing functions;

a network subsystem providing a coupling through which said host system may be connected to multiple remote display systems; and

means to transmit an encoded bit stream for decoding at one or more of said remote display systems.

16. The system of claim 15 wherein said transmitted encoded bit stream includes compressed video data encoded in a compressed video format that can be decoded at said remote display system and where the encoding includes source based encoding techniques.

17. The system of claim 15 wherein said encoded bit stream is generated from the output from said local graphics processor that is encoded and transmitted via said network subsystem for decode at said remote display systems, and wherein said network subsystem performs UDP-based packet transmissions with a protocol for handshaking with retransmission capability.

18. A method for operating a multi-display system, comprising the steps of:

processing a set of graphics operations, using a host system and a graphics and video display controller, for drawing one or more display surfaces;

processing video program data from external program sources to translate said external program source video program data onto one or more display surfaces;

compositing the surfaces into display frames;

duplicating said drawing and compositing steps to generate display refresh stream frame data for each of multiple remote display systems;

encoding said display refresh stream frame data to produce an encoded display update network stream; and

propagating said encoded display update network stream according to network protocol techniques through a network interface.

19. The method of claim 18 wherein said compositing step utilizes a display controller capable of combining display surfaces with various overlay and keying techniques.

20. The method of claim 18 wherein said network protocol techniques utilize a UDP-based protocol that also includes handshaking and retransmit capability.

21. The method of claim 18 wherein said encoding step is performed by data encoding processing blocks based on source based encoding techniques.

22. A method for operating a multi-display system, comprising the steps of:

processing a set of graphics operations using a host system that includes a CPU, and translating said graphics operations to a remote display protocol for execution on a remote display system;

translating video program data received from external program sources into an encoded network stream; and

propagating said remote display protocol and said encoded network stream according to network protocol techniques through a network interface.

23. The method of claim 22, wherein said remote display system is capable of remote execution of said graphics operations and of decoding and displaying said encoded network streams of encoded video data.

24. The method of claim 22, wherein said remote display system is capable of remote execution of said graphics operations and of decoding and displaying said encoded network streams of encoded display frame data.

25. A method for receiving display updates from a host system and displaying video program data, comprising the steps of:

receiving, through a network subsystem from two or more data sources, display updates which may include remote display protocol graphics commands, encoded video bitstreams or encoded data display frames;

decoding said received encoded video bitstream and producing a new display frame of data for display; and

outputting said display frame of data to refresh a display screen and continuing to refresh said display screen with current display data until a new display frame of data becomes available.